UK Edition
Ministers and Ofqual ignored multiple warnings that the exam regulator's algorithm for grading A-levels and GCSEs was heading for catastrophe, according to examiners, advisers, academics, former civil servants and MPs.
UCL was among universities to warn Ofqual as long as four months ago that its assessment model would penalise high-achieving children in disadvantaged schools.
Heads at some of Britain's top schools also submitted evidence to the regulator that there were serious risks of children's results being downgraded, leading to "inaccurate and unfair" grades.
Even Ofqual's own expert advisers warned that it was doomed to fail as long as ministers insisted on a system that would prevent grade inflation and ensure results were consistent with previous years.
The disclosures follow schools minister Nick Gibb's admission on Thursday that he had been told by Sir Jon Coles, a former director-general at the Department for Education, that the algorithm would be only 75 per cent accurate – meaning thousands of children would get wrong grades.
Mr Gibb (see video below) said he had called a meeting with Ofqual to discuss those "very concerns" but said was "reassured" the algorithm would not have that effect.
Despite this, Ofqual ploughed ahead with its algorithm, which led to around 40 per cent of last week's A-level results being downgraded, disproportionately hitting disadvantaged pupils and leading to a public outcry and concerns about GCSEs.
Ofqual rejected demand for algorithm to be tested by experts
Robert Halfon, the chairman of the Commons education committee – who warned ministers of the impending unfairnesses of the regulator's plans in a July report – said the MPs' recommendations to the Government and Ofqual were "for the most part ignored".
In particular, he said Ofqual rejected their demand for the algorithm to be published "immediately" so it could be stress-tested by experts and statisticians to prevent injustices. It was not published until after the results.
Even an offer by the Royal Statistical Society (RSS) to provide two leading university statisticians to advise on ways of preventing "grade volatility" and effectively assessing pupils' prior attainment ended up being thwarted after Ofqual demanded they sign non-disclosure agreements.
The regulator also rejected the MPs' calls for a fairer and less prescriptive appeals system that was not tilted in favour of better-off pupils and parents.
On Thursday, Ofqual was due to meet school leaders and teachers' groups, who want answers to their questions about what went wrong. But the regulator cancelled the meeting as it put up what Mr Halfon described as "a Berlin Wall around themselves".
Schools had believed Ofqual's algorithm would take account of teachers' assessments of individual pupils and their prior achievement but, in the end, it was largely based on students being put in rank order and the school's subject-by-subject historic performance.
Williamson's mixed messages meant process 'doomed to fail'
David Blow, who sat on the External Advisory Group on Exam Grading, set up in April to guide Ofqual, said that the process was "doomed to fail" from the moment Gavin Williamson wrote to Sally Collier, Ofqual's chief regulator, on March 31 giving his instructions for results.
"Mr Williamson said that firstly no child should be disadvantaged and secondly that marks ought to be in line with previous years," Mr Blow told The Telegraph. "Those are fundamentally incompatible objectives, as to make sure no child is disadvantaged you need to take an optimistic view of what each could achieve.
"There had been mixed messages – two different questions were posed from the outset, which is the fundamental problem."
Because of the mixed messages, some schools prioritised ensuring their pupils were not disadvantaged and gave more lenient grades, whilst a third gave grades in line with their past performances, Mr Blow said.
Ofqual modelled a number of algorithms but, when it had received all the teachers' predicted grades by the end of June, realised these were too high to use the preferred system and had to use one which would bring down grade inflation – leading to 40 per cent being downgraded (see video below).
However, it had been clear that grade inflation would be an issue by June 1, when predicted GCSE grades submitted by 1,900 schools – over half of all state secondaries in England – to the FTT education data lab showed massive inflation.
A blog by the lab's statisticians on June 15 revealed that "in every subject we've looked at, the average grade proposed for 2020 is higher than the average grade awarded last year. In most subjects, the difference is between 0.3 and 0.6 grades".
Ofqual 'should have pushed back more against impossible task'
Several of those who work with the regulator told The Telegraph that, as a supposedly independent body, it could and should have done more to push back against the impossible task that had been set by a Secretary of State demanding that it prevent grade inflation.
In the responses to the consultation on the grading system, which ran between April 15 and 29, Ofqual was repeatedly warned of the unfairness in the system.
UCL's Centre for Education Policy and Equalising Opportunities wrote on April 29 to warn that "the use of historic performance data for standardisation could penalise 'atypical' students such as high achievers in historically low-performing schools".
This was four months before Mr Williamson (seen defending the algorithm in the video below) "started noticing such outliers at the weekend", said Professor Lindsey Macmillan, the head of the UCL centre, who noted that there was no solution to the problem as long as Ofqual based its algorithm on school-level data rather than individual pupils.
At the same time, the the Headmasters' and Headmistresses' Conference (HMC), which represents Britain's top independent schools, wrote to Ofqual to warn that failure to use "objective evidence" of past performance by pupils risked them being graded "downwards", producing "inaccurate and unfair results in August".
One educational trust told the regulator that using an algorithm to make grades fit with a school's historic results was "a flawed system" that was"disingenuous, dangerous and fundamentally unfair".
Foreseeing that private schools would come out on top, the chief executive said: "The winners will be those already advantaged by the current system; the losers will be those already disadvantaged by the current system."
An exam board even foresaw the backlash, telling the regulator that "a disparity between a centre's estimations and the final results might also cause an increase in appeals, public criticism and a lack of public confidence in the summer 2020 results".
The problems were acknowledged by Ofqual, which admitted that some respondents to its consultation had said its approach would be "unfair to individuals who might have excelled this year and whose grades would be affected by the poor performance of their predecessors".
One insider said: "Ofqual wanted all the data in from schools on their pupils as quickly as possible, but not enough time was devoted at the beginning to thinking about what the purpose was and what it was going to achieve.
"Because Nick Gibb was so obsessed with preventing grade inflation, and consistency year on year, those two things came together to mean that was the only thing they were thinking about. They were not thinking about fairness to individual students and the path this was all going down."
There was growing alarm by July, with Mr Halfon's education select committee warning that it was "unconvinced that safeguards – such as additional guidance and practical recommendations – put in place by Ofqual will be sufficient to protect against bias and inaccuracy in calculated grades".
It culminated in the intervention in early July by former DFE mandarin Sir Jon, now the chief Executive of United Learning, a group of more than 70 independent schools.
In a letter to Mr Williamson, he raised particular concerns about using teachers' predicted grades for small groups of pupils but leaving larger groups reliant on the algorithm. He said this would lead to unfairness in the system.
Are ministers off the hook?
On Thursday, Mr Williamson's deputy Mr Gibb drew a distinction between the overall model designed to avoid grade inflation and its flawed implementation.
"The model was a good model, and we continued to refine it,"he said, defending the Government's approach to grading. "It became clear that there were some results that were being published on Thursday and Friday that were just not right, and they were not what the model had intended."
This distinction matters because ministers are responsible for the overall model, and Ofqual devised the implementation.
However, that does not let ministers off the hook.
As is now increasingly becoming apparent, problems and concerns had been flagged for months, and Tory MPs who have privately questioned whether Mr Williamson should remain in his job believe he should have seen them coming.
One senior Tory MP said: "The date for the publication of the results has been in the diary for at least a year. There is no excuse for this s***show."
Ofqual said it had consulted widely and tested different models before selecting the "fairest solution". This had been developed with input from expert advisers, including statistical experts, while it had worked closely with the DFE.