This report describes a study of reading test text selection, item writing and editing processes, with particular reference to these areas of test production for the IELTS Academic Reading test. Based on retrospective reports and direct observation, the report compares how trained and untrained item writers select and edit reading texts to make them suitable for a task-based test of reading and how they generate the accompanying items. Both individual and collective test editing processes are investigated.
For Phase 1 of the study, item writers were invited to respond to a questionnaire on their academic and language teaching and testing background, experience of IELTS and comments on its reading module (see Appendix B). Two groups of participants were selected: four officially-trained IELTS item writers (the experienced group) and three teachers of English for academic purposes who had prepared students to take IELTS, but had no previous experience of item writing for the IELTS Academic Reading module (the non-experienced group).
In Phase 2 of the project both groups were asked to select and prepare texts and accompanying items for an IELTS Academic Reading test, and to bring their texts and items to separate interview and focus group sessions. In the first of these sessions, participants were interviewed on how they had selected and edited their texts and how they had generated the items. In a second session, the item writers worked in their two groups to further refine the texts and items to make them more suitable for the test (as the trained item writers would normally do in a test editing meeting).
The analyses of the texts and accompanying items produced by each group, and of the discussions at all the Phase 2 sessions have produced valuable insights into the processes of text selection, adaptation and item writing. The differences observed between the experienced and non-experienced groups help to highlight the skills required for effective item writing for the IELTS Academic Reading test, while at the same time suggesting improvements that could be made to the item production process so that it might more fully operationalise the IELTS reading construct.