Test performance 2021

Each year, multiple versions of each of the six IELTS sections (Listening, Academic Reading, General Training Reading, Academic Writing, General Training Writing, and Speaking) are released for use by centres testing IELTS test takers. Reliability estimates for the tests used in 2021 are reported below.

Reliability of Listening and Reading sections

The reliability of Listening and Reading tests is reported using Cronbach's alpha, a reliability estimate which measures the internal consistency of the 40-item test. The following Listening and Reading material released in 2021 had sufficient test taker responses to estimate and report meaningful reliability values as follows:

Section (All Academic and General Training tests) Version Alpha
Listening version 55262 0.919
Listening version
59322 0.926
Listening version 54744
0.913
Listening version
51192
0.913
Listening version
56295
0.932
Listening version 52728
0.929
Listening version
55986
0.877
Listening version
51960
0.931
Listening version
56496
0.907
Listening version
59656
0.915
Listening version
51727
0.91
Listening version 59269
0.928
Listening version
50004
0.924
Listening version
51955
0.924
Listening version
52184
0.918
Listening version
54477
0.899
Average Alpha across versions   0.917
 
Section Version Alpha
Academic Reading version 54109 0.916
Academic Reading version
55788
0.928
Academic Reading version
51252
0.909
Academic Reading version
54915
0.901
Academic Reading version
51608
0.904
Academic Reading version
57302
0.909
Academic Reading version
53353
0.874
Academic Reading version
54613
0.845
Academic Reading version
55303
0.903
Academic Reading version
54486
0.912
Academic Reading version
56630
0.927
Academic Reading version
51773
0.917
Academic Reading version
51549
0.908
Academic Reading version
52938
0.848
Academic Reading version
50902
0.915
Academic Reading version
51810
0.929
Academic Reading version
54109
0.916
Average Alpha across versions   0.903

  

Section Version Alpha
General Training Reading version 57658 0.909
General Training Reading version 55048
0.88
General Training Reading version 59296
0.923
General Training Reading version
50577
0.921
General Training Reading version
57187
0.899
General Training Reading version
57870
0.923
General Training Reading version
50155
0.913
General Training Reading version
56898
0.919
General Training Reading version
50666
0.914
General Training Reading version
58722
0.917
General Training Reading version
56363
0.907
General Training Reading version
55202
0.924
General Training Reading version
51552
0.929
General Training Reading version
56158
0.919
General Training Reading version
50911
0.918
General Training Reading version
53049
0.926
General Training Reading version
57658
0.909
General Training Reading version
55048
0.88
Average Alpha across versions
0.915

The figures reported for Listening and Reading sections indicate the expected levels of reliability for tests containing 40 items. On the basis of these reliability figures, an estimate of the standard error of measurement (SEM) may be calculated for these sections using the following formula:

Standard error of measurement formula

st is the standard deviation of the test

rxx' is the reliability of the test

Table 1 Mean, standard deviation and standard error of measurement of Listening and Reading (2021)

Section Mean SD Alpha SEM
Listening 6.39
1.29
0.91
0.39
Academic Reading 6.26
1.17
0.91
0.35
General Training Reading 6.33
1.46
0.92
0.41

The SEM should be interpreted in terms of the final band scores reported for Listening and Reading sections (which are reported in half-bands).

Reliability of Writing and Speaking sections

For more information about the assessment criteria used for rating Writing and Speaking performance, read about the IELTS scoring in detail. Benchmarked example writing performances and CD-based speaking performances at different levels can be found, along with examiner comments, in the IELTS official practice materials. In addition, you can order the “IELTS Scores Explained” DVD which provides information specifically tailored to organisations wanting a detailed description of IELTS scores. This information helps in setting appropriate standards of English proficiency.

Reliability of rating is assured through the face-to-face training and certification of examiners and all must undergo a retraining and recertification process every two years. A Professional Support Network (PSN) manages and standardises the examiner cadre, including face-to-face examiner monitoring as well as distance monitoring (using recordings of the Speaking tests). A ‘jagged profile’ system maintains a further check on the global reliability of IELTS performance assessment. Routine targeted double marking identifies the level of divergence (i.e. jagged profile) between Writing and/or Speaking scores and Reading and Listening scores. This process allows for the identification of possible misclassified test takers. The jagged profile system is also combined with ‘Targeted sample monitoring’ to further identify possible faulty ratings by examiners. Selected centres worldwide are required to provide a sample of examiners' marked tapes and scripts. Tapes and scripts are then second-marked by a team of IELTS Principal Examiners and assistant Principal Examiners. Principal Examiners monitor for quality of both test conduct and rating, and feedback is returned to each test centre. The outcomes that emerge from these reliability measures feed back into examiner retraining and continually build on quality management and assurance systems for IELTS.

Generalisability studies based on examiner certification data show coefficients of 0.830.86 for Speaking and 0.810.89 for Writing.

The IELTS test contains four sections upon which an overall band score is awarded. Thus an estimate of composite reliability offers a useful measure for overall test reliability. Composite reliability estimates have been calculated following Feldt & Brennan (1989). To generate an appropriately cautious estimate, alpha values were used for the objectively marked papers, and G-coefficients for the single rater condition on subjectively marked scores. The composite reliability estimate for both the Academic and General Training tests was 0.96 and produced a composite SEM of 0.23. 

References
Feldt, LS & Brennan, RL (1989) Reliability. In RL Linn (Ed), Educational measurement (3rd ed, 105-146). New York: Macmillan
Shaw, SD (2004) IELTS writing: revising assessment criteria and scales (Phase 3). Research Notes 16, 3-7
Taylor, L & Jones, N (2001) Revising the IELTS Speaking Test. Research Notes 4, 9-12