Abstract
Yu, Rea-Dickins and Kiely (2011) used concurrent thinking-aloud as the main research instrument to examine test-takers’ cognitive processes of completing IELTS Academic Writing Task 1 (AWT1). In the current follow-up study, we employed an eye-tracking system (Tobii X2-60) with retrospective stimulated individual interviews and focus-group discussions as the major data
collection tools to examine:
Twenty-seven prospective IELTS test-takers from a large Chinese university volunteered to complete three AWT1 tasks of different types of graphs which were randomly assigned to them out of four tasks. The participants’ eye movements when taking the AWT1 tasks were recorded. Immediately after the participants had completed the three AWT1 tasks, we conducted retrospective stimulated recall interviews with each individual participant, with episodes of the recorded eye movement videos replayed as stimuli for discussions. The interviews were simultaneously video recorded via Tobii Studio 3.2.1 (Enterprise version). After completing all the retrospective stimulated interviews, we conducted six student-led focus-group discussions which were audio recorded. In total, the final dataset includes 27 hours of eye movement videos, 11 hours of retrospective stimulated recall interviews, 6 hours of focus-group discussions, and 81 writings produced at eye-tracking experiments. In addition, prior to the eye-tracking experiments, we collected the baseline data on all participants’ graphicacy, computer familiarity, and English writing abilities under normal examination condition.
The quantitative eye-movement data showed that less than 10% of time was spent on reading task instructions, 20% on reading graphs and 70% focusing on writing. This is clear evidence that IELTS AWT1 is fundamentally a writing task. The qualitative analysis of the visualisations of eye-movement data demonstrated the dynamics and uniqueness of each participant’s eye-movements. Graph features were found to have exerted significant impacts on the aggregated metrics of eye-movement (total fixation duration and total visit duration), but such impacts were not noticeable in the two metrics of single fixations (first fixation duration, and fixation duration). Bar graph, line graph and pie chart were considered much easier than statistical tables due to the nature of the graphs, as well as the amount of information contained in the different types of graphs. The cognitive naturalness and perceptual properties of graphs influenced the participants’ engagement with, preference towards and judgement about the difficulty level of different types of graphs. Graph familiarity was found to have weak and short-lived impacts on the participants’ test-taking cognitive processes. Similarly, the correlations between English writing ability and the eye-movement metrics were also weak and fuzzy. In the participants’ view, it was the rigid overall structure of IELTS AWT1 writing and the predictable nature of graphs and the associated cognitive conventions of graph comprehension and presentation that can make AWT1 tasks highly coachable and mouldable and consequently a weaker relationship between English writing ability and test-taking process and performance. The findings to the four research questions present some glimpses into the complex nature of the IELTS AWT1 tasks, and the dynamic interplays between test-taker characteristics (e.g., graph familiarity, English writing ability) and task features (e.g., different types of graphs, amount of information contained in a graph, and the relationships between task instructions, graphs and the textbox as the three major components of a task). A number of suggestions are made to conduct further quantitative and qualitative analysis of the eye-movement data to explore the dynamics and the idiosyncratic nature of each participant’s eye-movements.
