Smith, Ben, Morris, Stephen P. ORCID: https://orcid.org/0000-0001-6869-8933 and Armitage, Harry
(2022)
Using pupils’ grade obtained in national examinations as an outcome measure in evaluations : some considerations for the design of randomised controlled trials.
Research Papers in Education.
ISSN 0267-1522
|
Accepted Version
Available under License Creative Commons Attribution Non-commercial. Download (437kB) | Preview |
Abstract
It is not uncommon for randomised trials in education to have the performance of sample members in national examinations as their primary outcome. In many cases examination results are available as summary measures only. Taking the example of GCSE examination results in England, this paper shows that using summary measures of an underlying score or mark, such as exam grade, complicates the design of trials and can lead to under-powered studies. Simple simulations are used to explore the consequences of powering trials to detect a difference assuming grade or summary measures are the only outcome metric available, where the effects of an intervention is primarily captured in the unknown mark or score distribution. The analysis draws on data that relate the entire distribution of marks in English language and mathematics examinations to grades. Recommendations are made in order to address this problem.
Impact and Reach
Statistics
Additional statistics for this dataset are available via IRStats2.