Harnessing the potential of trace data and linguistic analysis to predict learner performance in a multi‐text writing task

• DOI: 10.1111/jcal.12769
• Publication Date: 2022-12-14T02:55:47Z
• Authors: Mladen Raković; Sehrish Iqbal; Tongguang Li; Yizhou Fan; Shaveen Singh; Surya Surendrannair; Jonathan Kilgour; Joep van der Graaf; Lyn Lim; Inge Molenaar; Maria Bannert; Johanna Moore; Dragan Gašević
• Keywords: TRACE (psycholinguistics)

Abstract Background Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and learning processes enacted while developing product. However, formative multiple has almost exclusively focused essay rarely included SRL processes. Objectives We explored viability using process features develop machine classifiers identify low‐ high‐performing essays a multi‐text task. Methods examined submissions 163 graduate students working an authentic assignment. utilised learners' trace data obtain state‐of‐the‐art natural language processing methods for our classifiers. Results Conclusions Of four popular this study, Random Forest achieved best performance (accuracy = 0.80 recall 0.77). The analysis important identified classification model revealed one (coverage reading topics) three (elaboration/organisation, re‐reading planning) as predictors quality. Major Takeaways classifier can used part future automated evaluation system will support at scale assessment different courses. Based performance, guidance tailored outset task help them do well