Adaptive Comparative Judgement

Where it started

image2

'Humans are better at judgements when comparing two things against each other rather than absolutely.'

(L.Thurstone, 1927)

Thurstone defined a mathematical way of modelling the chance that one object will 'beat' another in a comparison, Comparative Judgement was born.

It is by far the most reliable way known to score essays or more complex performances. It is much simpler than marking, and is known to reducing teacher workload.

What is ACJ?

image3

Adaptive Comparative Judgement (ACJ) is a low stakes approach to assessment, where several judges compare writing/work/competencies and then, using a computer algorithm, arrive at the Collaborative Professional Consensus (CPC) or rank order.  

ACJ is a highly reliable and valid way to assess subject competencies (like writing or performance) in education.

RM Compare is the only available software for schools that uses Adaptive Comparative Judgement.


Collaborative Professional Consensus (CPC)

image4

 Judging in our benchmarked sessions means that your student's writing will be largely judged by teachers (approximately 20) from outside your school.  A wide range of teacher's professional judgements elicit your CPC.

As several judges from outside your school define the CPC order, it has less bias that can be associated with other CJ systems.

Time

image5

 It normally takes less than an hour to assess a year group's writing, initially 60 seconds per judgement, as your teachers use the system their decision making improves reducing time, and workload, to approx 20 secs per judgement. The more judges, the less time needed to judge and the better the whole school CPD.

Assessment

image6

ACJ as assessment is low stakes and not seen as a test or assessment by the students taking part, as the work that your students complete in lesson time is judged. Removing the criteria and rubrics usually used for assessment for a writing task, for instance, often liberates the writer. 

Benefits

image7

There are several benefits to be gained from using ACJ to assess subject competencies over using rubrics, frameworks or criterion assessments: Read on for more information

Bias

image8

Halo effect and Order effect have an impact on the decisions that we make,  this is reduced as you only judge, anonymously, work from other schools when taking part in our ACJ group sessions.  

Ambiguity

image9

20 other professionals have judged each students work, not just the class teacher, as the teachers come from outside the school you can be confident that there can be little disagreement with the outcome.

Professional consensus about quality

image10

Teachers love being empowered to make professional decisions about the subjects they teach. More often than not they'll agree with each other too making ACJ a highly reliable assessment.

Marker Fatigue

image11

Marking a class worth of books against rubrics and criteria can take a couple of hours.  By the time your marking the last set of books you're quite mentally fatigued, those judgements suffer from and are affected by that fatigue.  With ACJ as you are only making a comparative judgement against two pieces of writing your judgements are equally affected for each judgement you make.

CPD / teacher learning

image12

Get all of your teacher across your school to judge in a staff meeting (1hr) for every year group so that teachers across the whole school understand what good and better looks like within the work range.  Use the judging session in the staff meetings to stimulate discussions between all staff to support writing improvement. Set writing targets and talk about pedagogy  (30 mins) free professional led CPD.

The scope...

image13

RM Compare accepts PDF, MP4 voice recording, Video recording, HTML pages and Website links (You Tube etc) all of which can be compared in an ACJ session.

Brilliant to also assess PE, Music, Art, Drama, speaking competitions and the like in a completely natural way.