Volume 3, 2000, Paper 3
Paper 3: An investigation of the rating process in the IELTS
oral Interview
Annie Brown
Holistic assessments of oral language proficiency are often made
in relation to performance in conversational language proficiency
interviews, one such example of which is the IELTS Oral
Interview. This study seeks to explore the rating practices
of trained and accredited IELTS raters when judging candidates'
performance in IELTS interviews. In particular, it aims to
address questions such as:
* How do raters cope with the task of having
to base an assessment of ability on a single performance?
* What is the relationship of linguistic and
non-linguistic aspects of the performance?
* How is the interlocutor's performance dealt
with in the assessment of the candidate's ability?
* Do raters focus on criteria other than those
specifically mentioned in the descriptors?
* How salient are the stated criteria?
* Does the same performance elicit judgements of
the same kind from different raters?
This study adds to a small but growing body of qualitative
research into the judgements made in assessments of second language
speaking proficiency. Using data (taped IELTS interviews)
collected in an earlier study (Brown and Hill, 1998), eight IELTS
raters each rated four interviews selected from a set of eight
using the IELTS band scales. For each interview they provided
a verbal protocol where they first summarised the reasons for the
score they had awarded and then reviewed the tape in order to
identify those features of the rating procedure which influenced
their scoring. This methodology is known as stimulated verbal
recall (di Pardo, 1994). In these, the raters were asked to
talk about the judging process and to identify the salient
decision-making points of the interview.
The raters were all accredited and practicing IELTS
interviewers. The candidates were all overseas students drawn
from a pre-university (Foundation) course. At the time of the
interviews they were preparing to take IELTS prior to submitting
applications for tertiary study in Australia.
The protocols were transcribed and coded. Findings are
discussed and implications are drawn regarding the validity of this
test format.
