Submissions of incorrect scores, missing documents mar CSEC, CAPE

…training of teachers, registrars recommended

Chairman of CXC, Professor Sir Hilary Beckles

The Independent Review Team tasked with reviewing the administration of the modified July/August 2020 Caribbean Secondary Education Certificate (CSEC) and the Caribbean Advanced Proficiency Examinations (CAPE) has pointed to a number of school-related issues during the moderation of the regional examinations.
These are: inadequate number of School Based Assessments (SBAs) submitted, incorrect assignment of candidates to groups for group SBAs, zero entered instead of “No SBA”, incorrect registration of students by schools, non-submission of scores/ submission of incorrect scores, missing documents, and corrupt files.
The Review Team convened by Chairman of the Caribbean Examination Council (CXC), Professor Sir Hilary Beckles, released its 46-page report on Wednesday. The panel was chaired by Professor Hazel Simmons-McDonald, Professor Emerita and retired Pro Vice-Chancellor and Principal of the Open Campus, The University of the West Indies (UWI).

CXC Registrar,
Dr Wayne Wesley

The other members are Professor Andrew Downes, retired Pro Vice-Chancellor, Planning and Development, (UWI); Professor Francis De Lanoy, President of the University of Curacao; Harrilal Seecharan, retired Chief Education Officer, Ministry of Education, Republic of Trinidad and Tobago; and Michael S Browne, Minister of Education in Antigua and Barbuda, and Chair of the Caricom Council for Human and Social Development (COHSOD).
While the report specifically focused on the modified approach for the administration of this year’s examinations, the moderation process applied to SBAs and the grading process for them, it also touched a number of other issues.
According to the report, complaints from a number of territories of students/ schools being ungraded in many instances can be attributed to the school level and internal issues identified by CXC. The report said that CXC experienced issues with being unable to convert files, removal of cover sheets, and challenges in uploading files and additional files.
Students with missing SBAs are ungraded, a policy of CXC.
The report revealed that a total of 11,578 subject entries in 2020, compared to 8062 in 2019, was ungraded. While the number of subject entries for which students were reported absent in 2019 was higher (39,958 compared to 30,108 in 2020), students were reported ungraded in 2020 (11,575) for more subject entries compared to 2019 (8,062).
“The review of the IT infrastructure and further training and communication of the requirements for uploading SBA scores are also critical to the reduction of instances of students receiving ungraded scores,” the report noted.

Reliability of SBA scores
The Review Team found that the modified examination process in 2020 resulted in a number of issues and concerns being raised by various stakeholder groups. This included the lowering of scores awarded by teachers on SBAs following moderation by CXC.
“…scores from the expanded moderation process were generally lower than in previous years. While there was an overall lowering of SBA scores resulting from the expanded process, the improved performance of students in the Paper I Examination resulted in the overall grade distribution of students in 2020 being comparable to previous years.”
They therefore held: “In addition, for on-site moderation within country and across country, standardisation is also essential for comparability of scores. On-site moderation is managed at the territory level by Local Registrars. The training of teachers and local registrars should therefore be examined and addressed by CXC to improve the reliability of scores obtained from SBAs.”
According to the Review Team, in any examination, teachers’ and students’ expectations of grades expected and grades obtained will vary. Further, the award of grades is a complex process requiring the bringing together of scores from items on different papers and components of the examination.

Having examined the changes; namely, the submission of 100 percent of the SBAs from all candidates and schools for all subjects, the Review Team said, the increase in the number of samples moderated and the deployment of an application to facilitate moderation for on-site moderation were all found to be measures which did not fundamentally change the requirements for moderation and the moderation process.
Rather, it stated, the changes served to increase the rigor and improve the reliability of the process in 2020, compared to previous years. The report said teacher training on the requirements for planning, administration and scoring of SBAs is vital to ensure comparability of teacher and moderator scores.

Better explained
The Review Team concluded that the processes which CXC proposed to use for grading and moderation could have been better explained to its stakeholders.
“…more extensive communication between CXC and educators in particular would have helped to clarify any misunderstandings that may have existed about the administration of the modified approach. The processes which CXC proposed to use for grading and moderation could have been better explained to its stakeholders.”
In fact, they noted, “In addition, the extension of the review process, the strengthening of the communications strategy with key stakeholders, must be treated with highest priority. It is the expectation that the recommendations will initiate steps leading to solutions of the problems that were underscored this year.”
The Review Team said it sought answers on whether the modified approach and the administration of the examination affected students’ performance.
In regard to the grading model, the Review Team found that it was adequate, given the circumstance, as it was the least disruptive in terms of parameters (weights and cut points for grades). The Team, however, added that a simulation exercise (with and without Paper 2), using data from previous years highlighted limitations of the model; namely, shifting in the distribution of grades (reduction in Grades I to IV).
“At the technical level, the limitations of the grading model and the extensive moderation used in the SBAs could have resulted in less than expected performance in some subjects at both CAPE and CSEC levels.”
The report highlighted that the issue of using the teachers’ predicted grades for each subject was considered as one option in the initial grading model, but was excluded in the final analysis because a statistical analysis of data for previous years found a weak correlation between the final grade for the three papers and the teachers’ predicted grade in general.
According to the report, incorporating the teachers’ predicted grades would have also raised the issue of the weighting of such grades in the overall grading process, given the CXC standards.