THE Primary Exit Profile (PEP) results are out! Congratulations to the hard-working students, dedicated teachers, and supportive parents.

However, concerns remain about the validity of these results. The reliability of PEP scores hinges on the robustness and transparency of the assessment process. The Ministry of Education must ensure that all assessment processes and results are transparent by producing and disseminating detailed technical documentation on the development, validation, and implementation of these assessments. To improve educational outcomes we must enhance how we measure learning by using valid and reliable methods, adopting various assessment types, and ensuring transparency. This will lead to a more accurate and comprehensive understanding of student learning and ultimately, better educational outcomes for all.

Provide the evidence

The Ministry of Education should produce the technical documentation used in the development of these exams, to ensure transparency and build public confidence in the assessment system. Technical documentation provides a detailed account of the methodologies and processes involved in creating, validating, and implementing the assessments. This includes information on item development, field testing, statistical equating, scaling procedures, and psychometric analyses. Without such documentation stakeholders — including educators, students, and parents — cannot verify that the exams are designed and administered using sound, scientifically validated techniques. This lack of transparency can lead to scepticism and distrust in the fairness, accuracy, and reliability of the test scores, undermining the credibility of the entire educational assessment system.

Build accountability

Moreover, the technical documentation serves as a critical tool for continuous improvement. By thoroughly documenting the test development process the ministry can identify areas for enhancement and make data-driven decisions to refine future assessments. External experts can review the documentation to provide additional validation, ensuring that the exams meet national and international standards. This process not only enhances the quality of the assessments but also fosters a culture of accountability within the education system.

Build equity

Providing access to technical documentation demonstrates a commitment to ethical practices and educational equity. It ensures that all stakeholders have a clear understanding of how the assessments are constructed and assures them that the tests are free from bias and appropriate for all students. In the absence of this documentation there can be no assurance that the necessary measures have been taken to develop fair and equitable assessments, potentially disadvantaging certain groups of students.

Baseless claims

As an expert in psychometrics I view recent claims and reports by the ministry with much scepticism. In the PEP press conference held on June 21, 2024, permanent secretary in the Ministry of Education and Youth Dr Kasan Troupe emphasised that the examinations branch worked hard to ensure that the PEP exam can “stand up to scrutiny and is fair, credible, and valid”. Claims like these cannot be made loosely without the requisite evidence. Validity rests with the interpretation of the test score — therefore, it is incorrect to say that an exam is valid. Likewise, Minister Faval Williams in her address described the ‘descriptors for performance’. These descriptors must be scientifically developed; where is the evidence for this? Also, she made comparisons between PEP results of 2019, 2022, 2023 and 2024. Test scores cannot be compared from one year to the other without specific actions taken during the test development phase; this is part of the evidence to be provided in the technical documentation. For clarity, to compare scores on any two tests there must be either the same students or the same test items.

Conclusion

In conclusion, to take the PEP results seriously and improve educational outcomes we must enhance the ways we measure learning. There is a need to strive for evidence-based decision-making. Without evidence that the assessment processes are rigorous and scientifically validated, the results cannot be trusted. Calls for the data to be released for independent analysis are growing louder, as transparency can help build trust and enable external experts to verify the validity and reliability of the assessments. However, even with access to the data, if the exam was not developed with sound methodologies and best practices then secondary analysis would be futile. As the adage goes, “garbage in, garbage out”— inaccurate or poorly constructed assessments will yield misleading conclusions, no matter how thoroughly the data is analysed. Therefore, ensuring that the development and implementation of these assessments adhere to the highest standards is paramount for their results to be meaningful and actionable.

Jeneve Swaby is a measurement specialist. She is the founder and CEO of Psychometric Associates, offering professional development courses to teachers in educational assessment, and conducts psychometric and data analysis on all forms of assessment — educational, medical, psychological, credentialing, and workplace. Jeneve may be contacted at psychometric.associates@gmail.com