In 2001 the IELTS interview format and criteria were revised. A major change was the shift from a single global scale to a set of four analytic scales focusing on different aspects of oral proficiency. This study is concerned with the validity of the analytic rating scales. It aims to verify the descriptors used to define the score points on the scales by providing empirical evidence for the criteria in terms of their overall focus, and their ability to distinguish levels of performance.
The Speaking Test band descriptors and criteria key indicators were analysed in order to identify relevant analytic categories for each of the four band scales: fluency, grammatical range and accuracy, lexical resource and pronunciation. Twenty interviews drawn from operational IELTS administrations in a range of countries, and representing a range of proficiency levels, were analysed with respect to these categories.
The analysis found that most of the measures displayed increases in the expected direction over the levels, which appears to confirm the validity of the criteria. However, for all measures the standard deviations tended to be large, relative to the differences between levels. This indicates a high level of variation amongst candidates assessed at the same level, and a high degree of overlap between levels, even for those measures which produced significant findings. In addition, for most measures the differences between levels were greater at some boundaries between two bands than at others.
Overall, the findings indicate that while all the measures relating to one scale contribute in some way to the assessment on that scale, no one measure drives the rating; rather a range of performance features contribute to the overall impression of the candidate’s proficiency.