Welcome to IELTS

Each year, multiple versions of each of the six IELTS modules (Listening, Academic Reading, General Training Reading, Academic Writing, General Training Writing, and Speaking) are released for use by centres testing IELTS candidates. Reliability estimates for the objectively and subjectively scored modules used in 2008 are reported below.

Reliability of objectively-scored modules (Reading and Listening)

The reliability of Listening and Reading tests is reported using Cronbach's alpha, a reliability estimate which measures the internal consistency of the 40-item test. The following Listening and Reading material released in 2008 had sufficient candidate responses to estimate and report meaningful reliability values as follows:

The figures reported for Listening and Reading modules indicate the expected levels of reliability for tests containing 40 items. On the basis of these reliability figures, an estimate of the standard error of measurement (SEM) may be calculated for these modules using the following formula:

S_t is the standard deviation of the test

r_xx' is the reliability of the test

Table 1 Mean, standard deviation and standard error of measurement of Listening and Reading

The SEM should be interpreted in terms of the final band scores reported for Listening and Reading modules (which are reported in half-bands).

Reliability of subjectively-scored modules (Writing and Speaking)

The reliability of the Writing and Speaking modules cannot be reported in the same manner as for Reading/Listening because they are not item-based; candidates' writing and speaking performances are rated by trained and standardised examiners according to detailed descriptive criteria and rating scales. The assessment criteria used for rating Writing and Speaking performance are described in the IELTS Handbook 2007. Benchmarked example writing performances and CD-based speaking performances at different levels can be found, along with examiner comments, in the IELTS official practice materials which can be ordered from the IELTS website. User-oriented band descriptors describing levels of Writing and Speaking performance are also available on the website. In addition, a new DVD “IELTS Scores Explained” provides information specifically tailored to organizations wanting a detailed description of IELTS scores. This information helps in setting appropriate standards of English proficiency. Click here for more information.

Reliability of rating is assured through the face-to-face training and certification of examiners and all must undergo a retraining and recertification process every two years. A Professional Support Network (PSN) manages and standardizes the examiner cadre, including face to face examiner monitoring as well as distance monitoring (using recordings of the Speaking tests). A ‘jagged profile’ system maintains a further check on the global reliability of IELTS performance assessment. Routine targeted double marking identifies the level of divergence (i.e., jagged profile) between Writing and/or Speaking scores and Reading and Listening scores. This process allows for the identification of possible misclassified candidates. The jagged profile system is also combined with ‘Targeted sample monitoring’ to further identify possible faulty ratings by examiners. Selected centres worldwide are required to provide a sample of examiners' marked tapes and scripts. Tapes and scripts are then second-marked by a team of IELTS Principal Examiners and assistant Principal Examiners. Principal Examiners monitor for quality of both test conduct and rating, and feedback is returned to each test centre. The outcomes that emerge from these reliability measures feed back into examiner retraining and continually build on quality management and assurance systems for IELTS.

Experimental generalisability studies were also carried out as part of the IELTS Speaking Revision Project (1998-2001) and the IELTS Writing Revision Project (2001-2005). The study conducted for the Speaking Revision produced an inter-rater correlation of 0.77, and a g-coefficient of 0.86 for the operational single-rater condition (see article in Research Notes 4); the Writing Revision study produced an inter-rater correlation of 0.77 and g-coefficients of 0.85-0.93 for the operational single-rater condition (see article in Research Notes 16). 

The IELTS exam contains four components upon which an overall band score is awarded. Thus an estimate of composite reliability offers a useful measure for overall test reliability. Approaches to estimating the reliability of a composite test are discussed in Feldt & Brennan (1989: 117) and Crocker & Algina (1986: 119-121).The method used here is taken from Feldt & Brennan (1989).

Composite reliability estimates were carried out from the period 1st January to 20 December, 2004. To generate an appropriately cautious estimate, minimum alpha values were used for the objectively marked papers; and g-coefficients for the single rater condition on subjectively marked scores. The composite reliability estimate for the Academic module was 0.95 and produced a composite SEM of 0.21. This finding shows a 95% probability for a candidate’s true score to fall within less than half a band (0.41) of the observed score. For General Training the composite reliability was 0.95 with a SEM of 0.23. If average; rather than minimum values; are used for the objective paper alphas, the reliability for both Academic and GT versions improves slightly to 0.96.