This study sought to investigate validity arguments related to IELTS reading score interpretations and use, exploring issues of consequential validity, the intersection of contextual validity and cognitive validity, as well as scoring validity.
Through four phases, the present study sought to explore and better understand international students’ perceptions regarding their language proficiency and preparedness for academic demands. We developed reading skill mastery profiles to investigate the possibility of enhancing test score users’ interpretations of scores through the use of descriptors developed through cognitive diagnostic modeling and through scale anchoring.
The study results show that both students and faculty/staff have limited knowledge regarding what the IELTS test scores mean. Differences between the IELTS reading texts and the texts that students encounter in their first year of undergraduate study contribute negatively towards students’ interpretations of test scores and sense of preparedness and further, to their attitude towards the test. Three reading attributes used to develop reader profiles and proficiency-level skill descriptors include basic comprehension, summarising main ideas, and inferential reasoning. Students who met the local cut-off score (6.5) lack mastery of inferential reasoning at the text level. Our analyses show that there are relatively fewer items measuring text-level critical thinking skills, which may explain why students who met the cut-off score lack inferential reasoning. This finding needs to be further investigated to determine if it is due to a lack of items or if the given local cut-off score is not appropriate for expecting successful academic performance requiring higher-order thinking skills such as inferential reasoning at the text level.
Test score users found the IELTS reading skill descriptors informative and useful for planning future actions to improve reading proficiency and support.