The Impact of AI-Assisted Argument Generation on Argumentation Depth
Journal: Journal of Higher Education Research DOI: 10.32629/jher.v7i3.5302
Abstract
This study employs a comparative experiment to systematically evaluate the impact of AI-assisted argument generation on argumentation depth in IELTS writing. Addressing two major shortcomings in previous research: 1. Experiments lacking a post-test with AI support withdrawn, failing to verify students' independent argumentation ability; 2. Subjective experience reports failing to establish a statistical association between argument hierarchical complexity and writing scores. Our experiment recruited 37 participants, divided into an experimental group (EG, using the DeepSeek Argument Tree module) and a control group (CG, traditional group discussion), for a 6-week IELTS writing training (pre-test, 4 weeks of AI-assisted writing, post-test with AI disabled). The core findings are as follows: 1. Successful ability transfer: The EG group scored significantly higher in the post-test (t=3.01, df=35, p=0.005, Cohen’s d=0.99). 2. Depth determines score: For each additional level in the argument hierarchy (e.g., adding a rebuttal point), the EG group's writing score increased by an average of 0.5 points. 3. Enhanced critical thinking: The EG group used 63% more data citations and demonstrated higher usage of rebuttal structures. This proves the effectiveness of AI assistance while highlighting the need to avoid over-reliance.
Keywords
AIGC; IELTS writing; argumentation depth
Full Text
PDF - Viewed/Downloaded: 0 TimesReferences
[1]Zhu, X., & Liu, H. (2024). Construction and practice of an AI writing process path based on the genre-based approach. Distance Education Journal, 42(1), 89-97.
[2]Fathi, J., & Rahimi, M. (2024). The role of AI writing mediators in enhancing cognitive and linguistic performance. Computers & Education, 210, 105000.
[3]Reynolds, B., Smith, T., & Zhang, K. (2023). The impact of AI-assisted writing on task response and coherence in IELTS Task 2: A controlled experiment. Language Learning & Technology, 27(2), 78-95.
[4]Javahery, A. (2025). Discourse analysis of ChatGPT-generated argumentative essays: A case of logical coherence at the expense of critical depth. Journal of Educational Technology & Society, 28(1), 45-59.
[5]Yin, L., & Jiahao, C. (2024). Modeling human-AI cognitive synergy: A six-step collaborative writing framework. International Journal of Artificial Intelligence in Education, 34(3), 321-345.
[6]Li, Y. (2022). Research on argument structure complexity in argumentative essays based on AI argument generators. Computer-Assisted Foreign Language Education, (4), 45-51.
[7]Wang, L. (2020). The "copy-paste" phenomenon in AI-assisted writing and its inhibition of critical thinking. China Educational Technology & Equipment, (15), 23-28.
[8]Gao, Y. (2022). An empirical study on the impact of AI writing tools on argumentation depth of students with different critical thinking levels. Modern Educational Technology, 32(5), 68-75.
[9]Zhang, W. (2017). SPSS Statistical Analysis Basic Tutorial(3rd ed.). Higher Education Press.
[10]Levene, H. (1960). Robust tests for equality of variances. In I. Olkin, S. G. Ghurye, W. Hoeffding, W. G. Madow, & H. B. Mann (Eds.), Contributions to probability and statistics: Essays in honor of Harold Hotelling (pp. 278-292).
[11]Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
[12]Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3/4), 591-611.
[2]Fathi, J., & Rahimi, M. (2024). The role of AI writing mediators in enhancing cognitive and linguistic performance. Computers & Education, 210, 105000.
[3]Reynolds, B., Smith, T., & Zhang, K. (2023). The impact of AI-assisted writing on task response and coherence in IELTS Task 2: A controlled experiment. Language Learning & Technology, 27(2), 78-95.
[4]Javahery, A. (2025). Discourse analysis of ChatGPT-generated argumentative essays: A case of logical coherence at the expense of critical depth. Journal of Educational Technology & Society, 28(1), 45-59.
[5]Yin, L., & Jiahao, C. (2024). Modeling human-AI cognitive synergy: A six-step collaborative writing framework. International Journal of Artificial Intelligence in Education, 34(3), 321-345.
[6]Li, Y. (2022). Research on argument structure complexity in argumentative essays based on AI argument generators. Computer-Assisted Foreign Language Education, (4), 45-51.
[7]Wang, L. (2020). The "copy-paste" phenomenon in AI-assisted writing and its inhibition of critical thinking. China Educational Technology & Equipment, (15), 23-28.
[8]Gao, Y. (2022). An empirical study on the impact of AI writing tools on argumentation depth of students with different critical thinking levels. Modern Educational Technology, 32(5), 68-75.
[9]Zhang, W. (2017). SPSS Statistical Analysis Basic Tutorial(3rd ed.). Higher Education Press.
[10]Levene, H. (1960). Robust tests for equality of variances. In I. Olkin, S. G. Ghurye, W. Hoeffding, W. G. Madow, & H. B. Mann (Eds.), Contributions to probability and statistics: Essays in honor of Harold Hotelling (pp. 278-292).
[11]Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
[12]Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3/4), 591-611.
Copyright © 2026 Jiaqi Liu, Xin Yang, Jialing Zhang
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License