Volume 1, 1998, Paper 1
Paper 1: Interviewer style and candidate performance in the
IELTS oral interview
Annie Brown and Kathryn Hill
Recent research into the validity of oral language interviews
has extended the focus beyond that of statistical analysis to
investigations of the structure of the interview discourse itself,
and to the language produced by both candidate and
interviewer. Research has indicated that, despite training,
interviewer behaviour varies considerably in terms of the amount of
support they give candidates, the amount of rapport raters consider
them to have established with candidates and the extent to which
they follow the instructions in terms of the type of discourse
elicited from candidates. While several writers allude to the
potential of such variable interviewer behaviour to affect the
validity of tests, studies have not yet empirically investigated
the relationship between interviewer behaviour and candidate
performance.
The study aims firstly to investigate the extent to which
differential behaviour by IELTS interviewers affects the scores
awarded to candidates and to identify interviewers who consistently
present a difficult or easy challenge to candidates. The
second part of the study involves a discourse analysis of the
contributions of 'difficult' and 'easy' interviewers, and aims to
identify aspects of interviewer behaviour which contribute to the
challenge they present.
The study is based on interviews undertaken with 32 candidates,
each of whom was interviewed twice by two different interviewers.
Six interviewers took part in the study. The interviewers were
audio-taped and multiple-rated.
The test data were analysed using the multifaceted Rach analysis
program FACETS (Linacre, 1989) in order to identify cases where
candidates perform differentially in the two interviews, as well as
identifying interviewers who consistently elicit poorer or better
performance. A total of 10 interviews from the two most
difficult and two easiest interviewers were transcribed and
analysed.
It was found that the easier interviewers tended to shift topic
more frequently and asked simpler questions, spending longer in
Phase 2 of the interview. The more difficult interviewers
tended to use a broader range of interactional behaviours, such as
interruption and disagreement as well as asking more challenging
questions.
While the intent in the development of the IELTS interview has
not been to standardise interviewer behaviour to the extent that
all candidates receive exactly the same prompts, there must be some
concern to ensure that all candidates are treated equally in terms
of the challenge presented by the interviewer. By making
explicit those features of interviewer behaviour which have the
potential to affect the quality of the candidates' performance,
this study is of relevance to the training of raters in terms of
increasing their understanding of the effect of their performance
on that of the candidate and in ensuring the comparability of the
challenge presented to different candidates
