Explorations into speaking assessments which maximise the benefit of technology to connect test-takers and examiners remotely and which preserve the interactional construct of the test are relatively rare. Such innovative uses of technology could contribute to the fair and equitable use of assessments in many contexts and need to be supported by a sound validity argument. To address these gaps and opportunities, an IELTS Speaking Test administered via video-call technology was developed, trialled, and validated in a four-phase research and development project.
An effort to strengthen parts of a validity argument has guided an iterative process of test development and validation, which included 595 test-takers and 32 examiners from seven global locations participating in a series of mixed methods studies. Each validation phase contributed to updating a validity argument, primarily in terms of the evaluation and explanation inferences, for the Video Call Speaking (VCS) test.
Phase 4, featured in this current report, examined some administration questions raised in the previous phase, such as time allocated in each part of the test and changes in the interlocutor frame, as well as test-taker and examiner perceptions of the VCS test. The average time taken for completion of each test task was recorded for 375 test-takers to investigate how adequate the existing timing is in the VCS mode. Ten examiners, who administered the test in this phase, were asked to respond to a questionnaire and participate in semi-structured focus groups to share their perceptions of the VCS test. Test-takers were also surveyed via a questionnaire, and additionally some of them provided more in-depth perceptions of the test during focus groups.
On the whole, the existing timing for each part was found to be adequate. Examiners perceived using the revised interlocutor frame as straightforward; however, several minor additional changes were suggested. They also perceived test-takers to be comfortable and not intimidated by the video-call mode, they found the overall test delivery quite comfortable, and overall, they perceived their rating experience as positive. A small majority of test-takers agreed that the VCS test allowed them to show their full English ability, and their perceptions about the quality of the sound were generally positive.
The report ends with a summary of the validity evidence gathered throughout the four-phase test development process, contextualised within a validity argument framework.