This study investigates whether there are meaningful differences in candidates’ internal cognitive processing and their test scores where an English for Academic Purposes (EAP) writing test is presented in two modes – pencil-and-paper and computer. Analysis of the test data of 262 candidates provides strong evidence to support the claim that there were no significant differences between the scores awarded by two independent examiners for candidates’ performances on the tests taken under two conditions. Analysis of the computer familiarity and anxiety questionnaire data shows that these students in general are familiar with computer usage and their overall reactions towards working with a computer are positive. Candidates’ reactions to the computer were also examined with respect to their fair mean average scores in the tests and it was found that the effect of computer familiarity on their performances in the two modes was negligible for most of the items in the questionnaire.
However, differences of up to half a band were detected in two areas: accessibility of public computers and frequency of word processing activity. Analysis of candidates’ responses to a theory- based questionnaire survey, which inquired about the internal processing undertaken during the two task performances, indicated a similar pattern between the cognitive processes involved in writing on a computer and writing with paper-and-pencil.
Overall results of the study are, in general, encouraging for the computerisation of the writing test, though further replication or expansion of this study using a larger and more balanced population is necessary. A more in-depth investigation of the data will then throw new light on the establishment of equivalence in EAP writing assessment.