Adaptive Comparative Judgement

Where it started

image437

'Humans are better at judgements when comparing two things against each other rather than absolutely.'

(L.Thurstone, 1927)

Thurstone defined a mathematical way of modelling the chance that one object will 'beat' another in a comparison, Comparative Judgement was born.

It is by far the most reliable way known to score essays or more complex performances. It is much simpler than marking, and is known to reducing teacher workload.

What is ACJ?

image438

Adaptive Comparative Judgement (ACJ) is a low stakes approach to assessment, where several judges compare writing/work/competencies and then, using a computer algorithm, arrive at the Collaborative Professional Consensus (CPC) or rank order.  

ACJ is a highly reliable and valid way to assess subject competencies (like writing or performance) in education.

RM Compare is the only available software for schools that uses Adaptive Comparative Judgement.


Collaborative Professional Consensus (CPC)

image439

 Judging in our benchmarked sessions means that your student's writing will be largely judged by teachers (approximately 20) from outside your school.  A wide range of teacher's professional judgements elicit your CPC.

As several judges from outside your school define the CPC order, it has less bias that can be associated with other CJ systems.

Time

image440

 It normally takes less than an hour to assess a year group's writing, initially 60 seconds per judgement, as your teachers use the system their decision making improves reducing time, and workload, to approx 20 secs per judgement. The more judges, the less time needed to judge and the better the whole school CPD.

Assessment

image441

ACJ as assessment is low stakes and not seen as a test or assessment by the students taking part, as the work that your students complete in lesson time is judged. Removing the criteria and rubrics usually used for assessment for a writing task, for instance, often liberates the writer. 

Benefits

image442

There are several benefits to be gained from using ACJ to assess subject competencies over using rubrics, frameworks or criterion assessments: Read on for more information

Bias

image443

Halo effect and Order effect have an impact on the decisions that we make,  this is reduced as you only judge, anonymously, work from other schools when taking part in our ACJ group sessions.  

Ambiguity

image444

20 other professionals have judged each students work, not just the class teacher, as the teachers come from outside the school you can be confident that there can be little disagreement with the outcome.

Professional consensus about quality

image445

Teachers love being empowered to make professional decisions about the subjects they teach. More often than not they'll agree with each other too making ACJ a highly reliable assessment.

Marker Fatigue

image446

Marking a class worth of books against rubrics and criteria can take a couple of hours.  By the time your marking the last set of books you're quite mentally fatigued, those judgements suffer from and are affected by that fatigue.  With ACJ as you are only making a comparative judgement against two pieces of writing your judgements are equally affected for each judgement you make.

CPD / teacher learning

image447

Get all of your teacher across your school to judge in a staff meeting (1hr) for every year group so that teachers across the whole school understand what good and better looks like within the work range.  Use the judging session in the staff meetings to stimulate discussions between all staff to support writing improvement. Set writing targets and talk about pedagogy  (30 mins) free professional led CPD.

The scope...

image448

RM Compare accepts PDF, MP4 voice recording, Video recording, HTML pages and Website links (You Tube etc) all of which can be compared in an ACJ session.

Brilliant to also assess PE, Music, Art, Drama, speaking competitions and the like in a completely natural way.