Best Research Award Finalists sponsored by TCS iON
This award honours outstanding research and its impact within the e-assessment community.
The 2024 Finalists are:
- AlphaPlus Consultancy Ltd with Welsh Government National Reading and Numeracy Onscreen Personalised Assessments (OPAs)
- Pearson with "I can read without letters doing backflips": understanding the SEND learner experience and shaping inclusive digital assessments.
- Duolingo with Measuring Variability in Proctor Decision Making on High-Stakes Assessments: Improving Test Security in the Digital Age.
- University of Massachusetts Amherst with The Massachusetts Adult Proficiency Tests.
Watch the summary videos below and read what being shortlisted means to the organisation and project.
AlphaPlus Consultancy Ltd with Welsh Government National Reading and Numeracy Onscreen Personalised Assessments (OPAs)
AlphaPlus has been working with the Welsh Government since 2016 to deliver personalised assessments to all children in Wales aged 6 to 14. To date, over 3.5 million assessments have been delivered in over 1,400 schools across Wales. AlphaPlus is extremely proud of our work on this innovative project. However, delivering an equivalent to a paper assessment via a technology platform is not an end in itself – our team and our client are focussed on trying to improve educational outcomes and understanding the complex relationships between assessment, progression and attainment.
Therefore the delivery of the personalised assessments and the outcomes achieved has been informed by and subject to research from the start and as we continually refine our approach.
Pearson with "I can read without letters doing backflips": understanding the SEND learner experience and shaping inclusive digital assessments.
Having our work officially recognised in this way speaks to the importance of accessibility and inclusion when developing digital assessments and putting the experiences of learners with SEND and additional needs at the heart of our findings and the decisions we make. Every student’s learning and assessment experience is unique. As the needs and experiences of learners with SEND and additional needs have historically been overlooked or marginalised in educational assessment research, the opportunity to showcase the lived experiences of students more fully is a privilege. Already, these experiences are driving tangible, positive changes in how we harness technology to further improve accessibility and inclusion within our assessments. Alongside these insights that enhance our understanding and open up more ways for students to best show what they know and can do in exams, we hope that our research will also inspire wider conversations and support work across the entire assessment community.
Duolingo with Measuring Variability in Proctor Decision Making on High-Stakes Assessments: Improving Test Security in the Digital Age.
This nomination underscores the importance of empirically evaluating and improving proctor decision making for high-stakes online exams. Despite this research topic being under-researched to date, we believe it is not only critical for securing online tests, but also for supporting fairness and credibility among test takers. Our study, aimed at measuring and increasing the reliability of proctor decision making, aligns with Duolingo's mission to provide accessible, equitable education globally. We believe that the future of education depends on the ability to offer affordable and secure assessments to learners everywhere, regardless of their geographical or economic constraints. By evaluating and reducing variability in proctor decision making and publishing our work, we want test users (e.g. university admissions officers) to understand how the DET supports the integrity of test scores through research, and we want test takers to trust that the proctoring process will be fair, regardless of which proctor reviews their test session.
University of Massachusetts Amherst with The Massachusetts Adult Proficiency Tests (MAPT)
Educational tests must justify they are valid for their intended purposes through a strong and comprehensive system of research. The Technical Manual for the MAPT outlines our validity research framework, the studies we conducted, and the results, that allow all interested users and stakeholders with important information regarding the quality, fairness, validity, and reliability of the MAPT. We believe educational tests must be supported by research, and testing agencies should be transparent in all testing activities, which include validation. Being a digital-first, multistage-adaptive exam requires innovations in test development, standard setting,, scoring, and validation, all of which are present in our documentation of MAPT research. We hope this research provides a helpful illustration of how to build and evaluate valid 21-century educational assessments.
Our Awards Sponsors
TCS iON is a strategic business unit of Tata Consultancy Services focused on enabling institutions, government departments and organizations from multiple industry sectors to be efficient in their recruitment/admissions process, learning and skilling and overall business operations with the use of ‘Phygital’ platforms. These are platforms that overlay digital technologies over physical assets. TCS iON delivers this with a unique IT-as-a-Service model that provides easy-to-use, secured, integrated, and hosted solutions in a build-as-you-grow and pay-as-you-use business model. TCS iON has been serving clients with the help of best practices gained through TCS’ global experience, deep domestic market exposure and industry-leading technical expertise.
Awards Headline Sponsor, The British Council
With a presence on the ground in over 100 countries, the British Council plays a pivotal role in transforming lives and fostering partnerships in the UK and globally. They offer opportunities for learning, gaining internationally recognised qualifications and building creative and professional networks making a positive impact on individuals and communities worldwide. Every year, the British Council’s global exams services team manages the delivery of over 4.5 million exams for schools, universities and professional bodies.
