We have some encouraging accuracy data for 20 question types on Examiner AI for Edexcel GCSE History.

Every week we measure the accuracy of Examiner AI against our database of thousands internally-standardised answers for different question types. That way we can be sure that Examiner AI is marking consistently, answer after answer. But, how accurate is it?

We focus on two key metrics: Weighted Kappa and mean bias.

Weighted Kappa: the gold standard measure for marking reliability. Above 0.80 is classified as "Almost Perfect" agreement - and even experienced human examiners typically only achieve 0.6-0.8 with each other.

Mean Bias tells us whether, on average, Examiner AI is more or less generous than our human examiners, with a positive (+) number indicating the average mark generosity or (-) lack of generosity!

Edexcel GCSE History has been very busy on Examiner AI since we launched the question types with almost 16,000 answers marked. The latest accuracy data for the four most commonly-marked answers is:

• Modern Depth Study - 4 Marks 'What is the main difference between...?' Weighted Kappa: 0.76 | Mean Bias -0.09
• Thematic Study - 8 Marks 'How useful are the sources...?' Weighted Kappa: 0.74 | Mean Bias: -0.45
• British Depth Study - 12 Marks 'Explain why' Weighted Kappa: 0.85 | Meamn Bias -1.00
• British Depth Study - 16 Marks 'How far do you agree?' Weighted Kappa: 0.86 | Mean Bias: 0.44
• Period Study - 16 Marks 'Explain two of the following...' (8+8) Weighted Kappa: 0.76 | Mean Bias -1.17

In a nutshell, Examiner AI delivers marks that basically agree with our experienced team of Edexcel GCSE History examiners. However, in all four question types above Examiner AI is marking slightly harsher than our human examiners, for example by 1.0 marks in a 12 mark British Depth Study answer.

If you haven't tried it yet, come given Examiner AI a go here (and share the link with your students so they can use it in their mocks revision).

Examiner AI for GCSE History