In second language (L2) writing instruction, providing real-time personalized feedback at scale within traditional classroom settings poses significant challenges for teachers. Technology tools such as the established Automated Writing Evaluation (AWE) and the emerging Large Language Models (LLMs), with their respective strengths, have emerged as viable solutions. To investigate leaners’ experiences with these two distinct computer-generated feedback types, this study examined the cognitive load and perceptions of English as a Foreign Language (EFL) students when using AWE and LLMs for writing feedback. A between-group experiment with 76 Chinese university students revealed that both tools induced low to moderate cognitive load, indicating favourable technology acceptance of both tools. Students’ perception ratings for LLMs were slightly higher than those for AWE across all three dimensions. Qualitative data also indicated a strong student preference for LLMs due to their detailed, adaptive, and interactive feedback, though some still favoured AWE for its simplicity. The findings supporting the previous research results that LLMs exhibit significant potential in providing multi-dimensional feedback, AWE remains relevant for certain learners. The study highlights the need for tool selection based on learner proficiency and the potential benefits of integrating Generative AI with traditional AWE methods in L2 writing instruction.

Ruyin Zheng. AWE and LLMs in L2 Writing Feedback: An Exploration of EFL Learners' Experiences. Academic Journal of Humanities & Social Sciences (2025), Vol. 8, Issue 7: 60-69. https://doi.org/10.25236/AJHSS.2025.080708.

[1] Hyland K. Teaching and researching writing[M]. Routledge, 2015.

[2] Vygotsky L S. Mind in society: The development of higher psychological processes[M]. Harvard university press, 1978.

[3] Lee I. Utility of focused/comprehensive written corrective feedback research for authentic L2 writing classrooms[J]. Journal of Second Language Writing, 2020, 49: 100734.

[4] Attali Y. Exploring the feedback and revision features of Criterion[J]. Journal of Second Language Writing, 2004, 14(3): 1-20.

[5] Bai L, Hu G. In the face of fallible AWE feedback: How do students respond?[J]. Educational Psychology, 2017, 37(1): 67-81.

[6] Barrot J S. Using automated written corrective feedback in the writing classrooms: Effects on L2 writing accuracy[J]. Computer Assisted Language Learning, 2023, 36(4): 584-607.

[7] Chen M, Cui Y. The effects of AWE and peer feedback on cohesion and coherence in continuation writing[J]. Journal of Second Language Writing, 2022, 57: 100915.

[8] Zhang Z V, Hyland K. Fostering student engagement with feedback: An integrated approach[J]. Assessing Writing, 2022, 51: 100586.

[9] Kohnke L, Moorhouse B L, Zou D. ChatGPT for language teaching and learning[J]. Relc Journal, 2023, 54(2): 537-550.

[10] Gillani N, Eynon R, Chiabaut C, et al. Unpacking the “Black Box” of AI in education[J]. Educational Technology & Society, 2023, 26(1): 99-111.

[11] Roumeliotis K I, Tselikas N D. Chatgpt and open-ai models: A preliminary review[J]. Future Internet, 2023, 15(6): 192.

[12] Han J, Yoo H, Myung J, et al. LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interaction[J]. arXiv preprint arXiv:2310.05191, 2023.

[13] Banihashem S K, Kerman N T, Noroozi O, et al. Feedback sources in essay writing: peer-generated or AI-generated feedback?[J]. International Journal of Educational Technology in Higher Education, 2024, 21(1): 23.

[14] Steiss J, Tate T, Graham S, et al. Comparing the quality of human and ChatGPT feedback of students’ writing[J]. Learning and Instruction, 2024, 91: 101894.

[15] Guo K, Wang D. To resist it or to embrace it? Examining ChatGPT’s potential to support teacher feedback in EFL writing[J]. Education and Information Technologies, 2024, 29(7): 8435-8463.

[16] Ranalli J. L2 student engagement with automated feedback on writing: Potential for learning and issues of trust[J]. Journal of Second Language Writing, 2021, 52: 100816.

[17] Meyer J, Jansen T, Schiller R, et al. Using LLMs to bring evidence-based feedback into the classroom: AI-generated feedback increases secondary students’ text revision, motivation, and positive emotions[J]. Computers and Education: Artificial Intelligence, 2024, 6: 100199.

[18] Koltovskaia S. Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study[J]. Assessing Writing, 2020, 44: 100450.

[19] Zhang Z V. Engaging with automated writing evaluation (AWE) feedback on L2 writing: Student perceptions and revisions[J]. Assessing Writing, 2020, 43: 100439.

[20] Guo K. EvaluMate: Using AI to support students’ feedback provision in peer assessment for writing[J]. Assessing Writing, 2024, 61: 100864.

[21] Su Y, Lin Y, Lai C. Collaborating with ChatGPT in argumentative writing classrooms[J]. Assessing Writing, 2023, 57: 100752.

[22] Koltovskaia S, Rahmati P, Saeli H. Graduate students’ use of ChatGPT for academic text revision: Behavioral, cognitive, and affective engagement[J]. Journal of Second Language Writing, 2024, 65: 101130.

[23] Sweller J. Cognitive load theory, learning difficulty, and instructional design[J]. Learning and instruction, 1994, 4(4): 295-312.

[24] Hoch E, Sidi Y, Ackerman R, et al. Comparing mental effort, difficulty, and confidence appraisals in problem-solving: A metacognitive perspective[J]. Educational Psychology Review, 2023, 35(2): 61.

[25] Hattie J, Timperley H. The power of feedback[J]. Review of educational research, 2007, 77(1): 81-112.

[26] Winstone N E, Nash R A, Rowntree J, et al. ‘It'd be useful, but I wouldn't use it’: barriers to university students’ feedback seeking and recipience[J]. Studies in Higher Education, 2017, 42(11): 2026-2041.

[27] Hwang G J, Yang L H, Wang S Y. A concept map-embedded educational computer game for improving students' learning performance in natural science courses[J]. Computers & Education, 2013, 69: 121-130.

[28] Paas F G W C. Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive-load approach [J]. Journal of educational psychology, 1992, 84(4): 429.

[29] Sweller J, Van Merrienboer J J G, Paas F G W C. Cognitive architecture and instructional design[J]. Educational psychology review, 1998, 10: 251-296.

[30] Strijbos J W, Pat-El R, Narciss S. Structural validity and invariance of the feedback perceptions questionnaire [J]. Studies in Educational Evaluation, 2021, 68: 100980.

[31] Hancock P A, Billings D R, Schaefer K E. Can you trust your robot?[J]. Ergonomics in Design, 2011, 19(3): 24-29.

[32] Braun V, Clarke V. Using thematic analysis in psychology[J]. Qualitative research in psychology, 2006, 3(2): 77-101.