University students' cyclical self-assessment process mediated by artificial intelligence in academic writing

Authors

  • Yulia Tria Hapsari Universitas Sebelas Maret
  • Nur Arifah Drajati Universitas Sebelas Maret
  • Endang Setyaningsih Universitas Sebelas Maret

DOI:

https://doi.org/10.26418/jeltim.v5i1.59858

Keywords:

academic writing, artificial intelligence, self-assessment, university students

Abstract

This study was designed to investigate the cyclical self-assessment process in the Academic Writing class mediated by Artificial Intelligence (AI). This narrative inquiry elicited data from three university students having different writing proficiency levels: high, middle, and low levels at one university in Indonesia. The data was collected through reflective notes and interviews. The data was then analysed using thematic analysis. The study revealed two significant findings. First, the three university students with different writing proficiency levels engaged in different stages of cyclical self-assessment caused by two main factors, namely learning motivation and level of trust in AI. The more motivated university student was to learn, the more likely they were to seek external feedback actively. Additionally, their level of trust in Automated Written Corrective Feedback (AWCF) and Automated Writing Evaluation (AWE) engaged them in an evaluation and revision process that engaged them in a cyclical self-assessment process that would improve their final results. Second, Artificial Intelligence (AI) could facilitate an effective cyclical self-assessment process with various features. The implication of the study is discussed.

References

Adachi, C., Tai, J. H. M., & Dawson, P. (2018). Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education. Assessment & Evaluation in Higher Education, 43(2), 294–306. https://doi.org/10.1080/02602938.2017.1339775

Adrefiza, A., & Fortunasari, F. (2020). Written corrective feedback on students’ thesis writing: an analysis of student-supervisory interactions. Journal of English Language Teaching Innovations and Materials (Jeltim), 2(1), 14–24. http://dx.doi.org/10.26418/jeltim.v2i1.37317

Barkhuizen, G. (2014). Revisiting narrative frames: An instrument for investigating language teaching and learning. System, 47, 12–27. https://doi.org/10.1016/j.system.2014.09.014

Barkhuizen, G., Benson, P., & Chik, A. (2014). narrative inquiry in language teaching and research. Routledge.

Barrot, J. S. (2021). Using automated written corrective feedback in the writing classrooms: Effects on L2 writing accuracy. Computer Assisted Language Learning, https://doi.org/10.1080/09588221.2021.1936071

Bourke, R. (2018). Self-assessment to incite learning in higher education: developing ontological awareness. Assessment & Evaluation in Higher Education, 43(5), 827–839. https://doi.org/10.1080/02602938.2017.1411881

Bram, B., & Angelina, P. (2022). Indonesian Tertiary Education Students’ Academic Writing Setbacks and Solutions. International Journal of Language Education, 6(3), 267–280. https://doi.org/10.26858/ijole.v6i3.22043

Brown, T. L. & Harris, L. R. (2013). Student self-assessment. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment. SAGE.

Chen, S. Y., & Tseng, Y. F. (2021). The impacts of scaffolding e-assessment English learning: a cognitive style perspective. Computer Assisted Language Learning, 34(8), 1105-1127. https://doi.org/10.1080/09588221.2019.1661853

Dunlosky, J., & Rawson, K. A. (2012). Overconfidence Produces Underachievement: Inaccurate Self Evaluations Undermine Students’ Learning and Retention. Learning & Instruction, 22(4), 271–280. https://doi.org/10.1016/j.learninstruc.2011.08.003

Ebadi, S., & Rahimi, M. (2019). Mediating EFL learners’ academic writing skills in online dynamic assessment using Google Docs. Computer Assisted Language Learning, 32(5-6), 527–555. https://doi.org/10.1080/09588221.2018.1527362

Hyland, K. (2013). Writing in the university: Education, knowledge, and reputation. Language Teaching, 46(1), 1–18. https://doi.org/10.1017/S0261444811000036

Jiang, L., & Yu, S. (2022). Appropriating automated feedback in L2 writing: experiences of Chinese EFL student writers. Computer Assisted Language Learning, 35(7), 1329–1353. https://doi.org/10.1080/09588221.2020.1799824

Jusslind, S., & Widlund, A. (2021). Academic writing workshop-ing to support students writing bachelor’s and master’s theses: a more-than-human approach. Teaching in Higher Education, https://doi.org/10.1080/13562517.2021.1973409

Lessard, S., Caine, V., & Clandinin, D. J. (2018). Exploring neglected narratives: understanding vulnerability in narrative inquiry. Irish Educational Studies, 37(2), 191–204. https://doi.org/10.1080/03323315.2018.1465835

Li, Z., Link, S., Ma, H., Yang, H., & Hegelheimer, V. (2014). The role of automated writing evaluation holistic scores in the ESL classroom. System, 44, 66-78. https://doi.org/10.1016/j.system.2014.02.007

Micán, A. D., & Medina, L. C. (2017). Boosting vocabulary learning through self-assessment in an English language teaching context. Assessment & Evaluation in Higher Education, 42(3), 398–414. https://doi.org/10.1080/02602938.2015.1118433

Nazari, N., Shabbir, M. S., & Setiawan, R. (2021). Application of Artificial Intelligence powered digital writing assistant in higher education: Randomized controlled trial. Heliyon, 7(5), E07014. https://doi.org/10.1016/j.heliyon.2021.e07014

Nieminen, J. H., Asikainen, H., & Rämö, J. (2021). Promoting deep approach to learning and self-efficacy by changing the purpose of self-assessment: A comparison of summative and formative models. Studies in Higher Education, 46(7), 1296–1311. https://doi.org/10.1080/03075079.2019.1688282

Nieminen, J. H., & Tuohilampi, L. (2020). ‘Finally studying for myself’ – examining student agency in summative and formative self-assessment models. Assessment & Evaluation in Higher Education, 45(7), 1031–1045. https://doi.org/10.1080/02602938.2020.1720595

Panadero, E., Brown, G. T. L., & Strijbos, J. W. (2016). The future of student self-assessment: A review of known unknowns and potential directions. Educational Psychology Review, 28, 803–830. https://doi.org/10.1007/s10648-015-9350-2

Panadero, E., Fernández-Ruiz, J., & Sánchez-Iglesias, I. (2020).

Secondary education students’ self-assessment: the effects of feedback, subject matter, year level, and gender. Assessment in Education: Principles, Policy & Practice, 27(6), 607-634. https://doi.org/10.1080/0969594X.2020.1835823

Park, J. (2018). Effectiveness of teacher and peer feedback: through the lens of Korean tertiary writing classroom. The Journal of Asia TEFL, 15(2), 429–444. http://dx.doi.org/10.18823/asiatefl.2018.15.2.11.429

Shang, H. F. (2022). Exploring online peer feedback and automated corrective feedback on EFL writing performance. Interactive Learning Environments, 30(1), 4–16. https://doi.org/10.1080/10494820.2019.1629601.

Shermis, M. D., Burstein, J., & Bursky, S. A. (2013). Introduction to automated essay evaluation. In M. D. Shermis & J. Burstein (Eds.), Handbook of automated essay evaluation (pp. 23–37). Routledge.

Teng, L. S. (2022). Explicit strategy-based instruction in L2 writing contexts: A perspective of self-regulated learning and formative assessment. Assessing Writing, 53, Article 100645. https://doi.org/10.1016/j.asw.2022.100645

Vajjala, S. (2018). Machine learning in applied linguistics. In C. A. Chapelle (Ed.). The encyclopedia of applied linguistics. Wiley.

Wilson, J., & Roscoe, R. D. (2020). Automated writing evaluation and feedback: multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87–125. https://doi.org/10.1177/0735633119830764.

Yan, Z. (2020). Self-assessment in the process of self-regulated learning and its relationship with academic achievement. Assessment & Evaluation in Higher Education, 45(2), 224–238. https://doi.org/10.1080/02602938.2019.1629390

Yan, Z., & Brown, G. T. L. (2017). A cyclical self-assessment process: towards a model of how students engage in self-assessment. Assessment & Evaluation in Higher Education, 42(8), 1247–1262. https://doi.org/10.1080/02602938.2016.1260091

Yan, Z., & Carless, D. (2022). Self-assessment is about more than self: the enabling role of feedback literacy. Assessment & Evaluation in Higher Education. 47(7), 1116–1128. https://doi.org/10.1080/02602938.2021.2001431

Yan, Z., Chiu, M.M., & Ko, P.Y. (2020). Effects of self-assessment diaries on academic achievement, self-regulation, and motivation. Assessment in Education: Principles, Policy & Practice, 5, 562–583. https://doi.org/10.1080/0969594X.2020.1827221

Zhang, Z. (2020). Engaging with automated writing evaluation (AWE) feedback on L2 writing: Student perceptions and revisions. Assessing Writing, 43, Article 100439. https://doi.org/10.1016/j.asw.2019.100439

Zong, Z., Schunn, C. D., & Wang, Y. (2021). Learning to improve the quality peer feedback through experience with peer feedback. Assessment & Evaluation in Higher Education, 46(6), 973–992. https://doi.org/10.1080/02602938.2020.1833179

Downloads

Published

2023-04-20

Issue

Vol. 5 No. 1 (2023): April 2023

Section

Articles

License

Authors retain copyright and grant the journal the right to publish the work, which is simultaneously made available under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence: https://creativecommons.org/licenses/by/4.0/