Evaluating a multimodal digital book's impact on ESP students' learning outcomes
DOI:
https://doi.org/10.26418/jeltim.v6i2.82230Keywords:
business English, ESP, Kirkpatrick evaluation model, multimodal digital book, technology-enhanced language learningAbstract
The use of digital books in EFL setting has become widespread, eliciting responses. However, the evaluation on the multimodal digital book using a Kirkpatrick model of evaluation is limited. This study focused on the evaluation of the use of multimodal digital book in English for Business Course toward the students"™ English competence and behavioral outcomes. 60 students were involved in this study. The integration of Kirkpatrick evaluation model and mixed-method approach used as the research design. Moreover, interviews were used to acquire qualitative data, while surveys, pre- and post-test results, and performance ratings were used to collect quantitative data. The result showed overwhelming feedback from the students which showed over 85% of the scores on every aspect. Pre-and post-test result then showed a significant result with a large effect size (Cohen's d = 1.917). Furthermore, based on six practical activities and a final project result, the students"™ business English skills were steadily improved. Based on the UTAUT model framed, the students also showed the digital book"™s high levels of acceptability and perceived use. The findings also showed the multimodal digital book challenges conventional methods of teaching ESP.
References
Al-Abdullah, S. (2022). ESP textbook evaluation: The case of Kuwaiti students of business administration. English Language Teaching, 15(10). https://doi.org/10.5539/elt.v15n10p75
Alsalamah, A., & Callinan, C. (2021). Adaptation of Kirkpatrick’s four-level model of training criteria to evaluate training programmes for head teachers. Education Sciences, 11(3), 116. https://doi.org/10.3390/educsci11030116h
Amirinejad, M., & Rahimi, M. (2023). Integrating digital storytelling into STEAM teaching: Examining young language learners’ development of self-regulation and English literacy. International Journal of Technology in Education, 6(4), 720–735. https://doi.org/10.46328/ijte.551
Anthony, L. (2018). Introducing English for specific purposes. Routledge.
Apriliyanti, D. L. (2020). Enhancing teachers’ competencies through professional development program: Challenges and benefactions. Acuity: Journal of English Language Pedagogy, Literature and Culture, 5(1), 28–38. https://doi.org/10.35974/acuity.v5i1.2042
Basturkmen, H. (2021). Is ESP a materials and teaching-led movement? Language Teaching, 54(4), 491–501. https://doi.org/https://doi.org/10.1017/S0261444820000300
Bates, C. C., Klein, A., Schubert, B., McGee, L., Anderson, N., Dorn, L., McClure, E., & Ross, R. H. (2017). Eâ€books and eâ€book apps: Considerations for beginning readers. The Reading Teacher, 70(4), 401–411. https://doi.org/10.1002/trtr.1543
Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806
Chen, G.-D., Chang, C.-K., Wang, C.-Y., & Jian, X.-L. (2018). Development and evaluation of a novel e-book interface for scaffolding thinking context to learn from writing examples. Interactive Learning Environments, 26(7), 970–988. https://doi.org/10.1080/10494820.2018.1427113
Creswell, J. W. (2022). A concise introduction to mixed methods research. Sage Publications.
del Carmen Boloña, M., & Allen, C. (2022). Learners’ perceptions of using Moodle books in online ESP courses. Intelligent CALL, Granular Systems and Learner Data: Short Papers from EUROCALL 2022, 30–35. https://doi.org/10.14705/rpnet.2022.61.1430
Fan, H. (2020). The Implementation and challenges of using role-play to improve Chinese university students’ English communicative competence. Exchanges the Interdisciplinary Research Journal. https://doi.org/10.31273/eirj.v8i1.592
GarcÃa-Pinar, A. (2022). Exploring verbal and non-verbal expressions of ESP undergraduates’ own voices and identities. International Journal of English Studies, 22(2), 155–175. https://doi.org/10.6018/ijes.508651
Grieve, A., Ta, B. T., & Ross, B. (2022). Placement educators’ perspectives of international social work students’ sociopragmatic communication skills. Advances in Health Sciences Education. https://doi.org/10.1007/s10459-022-10155-1
Guha, S. (2021). Creating audio books for children with visual impairment: The collaborative approach leading to virtual learning. International Journal of Technology in Teaching and Learning, 16(1), 37–48. https://doi.org/10.37120/ijttl.2020.16.1.04
Hafner, C. A. (2015). Remix Culture and English Language Teaching: The Expression of Learner Voice in Digital Multimodal Compositions. Tesol Quarterly, 49(3), 486–509. https://doi.org/10.1002/tesq.238
Heckadon, P., & Tuzlukova, V. (2020). Skill-Based ESP classroom: Teaching for preserving and enhancing Oman’s comparative advantages in an era of economic and digital transformation. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3798110
Heller, M. L., & Cassady, J. C. (2016). Predicting community college and university student success. Journal of College Student Retention Research Theory & Practice, 18(4), 431–456. https://doi.org/10.1177/1521025115611130
Hellwig, A. F. J., Jones, P., Matruglio, E., & Georgiou, H. (2022). Multimodality and English for special purposes: Signification and transduction in architecture and civil engineering models. Frontiers in Communication, 7. https://doi.org/10.3389/fcomm.2022.901719
Hsieh, Y., & Huang, S. (2020). Using an E-book in the secondary English classroom: Effects on EFL reading and listening. Education and Information Technologies, 25(2), 1285–1301. https://doi.org/10.1007/s10639-019-10036-y
Indriani, S., & Suteja, H. (2023). Fostering reading interest through digital storytelling for young learners in the early childhood. Journal of Education and Learning (EduLearn), 17(2), 301–306. https://doi.org/10.11591/edulearn.v17i2.18372
Iswati, L., & Triastuti, A. (2021). Voicing the challenges of ESP teaching: Lessons from ESP in non-English departments. Studies in English Language and Education, 8(1), 276–293. https://doi.org/10.24815/siele.v8i1.17301
Kabaran, G. G., & Uşun, S. (2021). Evaluation of the professional development program in digital material design according to the Kirkpatrick’s model. International Journal of Curriculum and Instructional Studies (IJOCIS), 11(1), 65–88. https://doi.org/10.31704/ijocis.2021.004
Kersten, S. (2017). Becoming nonfiction authors: Engaging in science inquiry. The Reading Teacher, 71(1), 33–41. https://doi.org/10.1002/trtr.1577
Kholis, A., & Azmi, U. (2023). A need analysis on developing English interactive multimodal e-book oriented to 21st century skills. Elsya Journal of English Language Studies, 5(1), 85–106. https://doi.org/10.31849/elsya.v5i1.11804
Kirkpatrick, D. (1996). Great ideas revisited. Training & Development, 50(1), 54–60. https://link.gale.com/apps/doc/A18063280/AONE?u=anon~2eb918b8&sid=googleScholar&xid=1090fcaa
Korat, O., & Falk, Y. (2019). Ten years after: Revisiting the question of e-book quality as early language and literacy support. Journal of Early Childhood Literacy, 19(2), 206–223. https://doi.org/10.1177/1468798417712105
Kristianti, T., & YP, H. S. (2023). The effectiveness of project-based learning (PBL) for engineering students in ESP class. Journal of Education Learning and Innovation (Elia), 3(1), 146–154. https://doi.org/10.46229/elia.v3i1.619
Kucirkova, N. (2017). An integrative framework for studying, designing and conceptualising interactivity in children’s digital books. British Educational Research Journal, 43(6), 1168–1185. https://doi.org/10.1002/berj.3317
Kustini, S., Suherdi, D., & Musthafa, B. (2020). Beyond traditional literacies: A multimodal-based instruction to fostering student digital literacy learning. Jurnal Pendidikan Bahasa Dan Sastra, 20(1), 37–47. https://doi.org/10.17509/bs_jpbsp.v20i1.25969
Laksana, I. P. Y., Paramita, I. G. A. P. D., Yuliantini, N. N., Sukerti, G. N. A., & Suciani, N. K. (2024). Engaging vocational college students’ learning through e-portfolio based digital module. Yavana Bhasha: Journal of English Language Education, 7(1), 21–32. https://ojs.uhnsugriwa.ac.id/index.php/YB/article/view/3430
Mune, C., & Agee, A. (2016). Are e-books for everyone? An evaluation of academic e-book platforms’ accessibility features. Journal of Electronic Resources Librarianship, 28(3), 172–182. https://doi.org/10.1080/1941126X.2016.1200927
Nazim, M., & Hazarika, Z. (2017). Efficacy of ESP in EFL context: A case study of Saudi Arabia. Arab World English Journal, 8(1), 145–164. https://doi.org/10.24093/awej/vol8no1.11
Ozbay, I., & Ugurelli, Y. O. (2023). Changing children’s literature in the digital age: Digital books. International Journal of Education and Literacy Studies, 11(1), 68–85. https://doi.org/10.7575/aiac.ijels.v.11n.1p.68
Park, J., & Lee, J. (2021). Effects of e-books and printed books on EFL learners’ reading comprehension and grammatical knowledge. English Teaching, 76(3), 35–61. https://doi.org/10.15858/engtea.76.3.202109.35
Reio Jr, T. G., Rocco, T. S., Smith, D. H., & Chang, E. (2017). A critique of Kirkpatrick’s evaluation model. New Horizons in Adult Education and Human Resource Development, 29(2), 35–53. https://doi.org/10.1002/nha3.20
Salehi, H., Khadivar, Z., Babaee, R., & Singh, H. K. A. J. (2015). An evaluating study on ESP medical textbook: Instructors and learners’ needs analysis. English Language Teaching, 8(8), 142–151. https://doi.org/10.5539/elt.v8n8p142
Santos, L. M. D. (2020). The motivations of learning English for specific purposes (ESP) courses: A study of English for nursing purposes. Universal Journal of Educational Research, 8(12A), 7809–7817. https://doi.org/10.13189/ujer.2020.082569
Solikhah, I. (2020). Evaluating EAP textbooks for Indonesian university students using impressionistic and in-depth assessment. Indonesian Journal of English Language Teaching and Applied Linguistics, 4(2), 395–411. https://eric.ed.gov/?id=EJ1291658
Susanti, A., Presdyasmara, C., Dewi, F. C., & Wardani, Y. (2021). Developing students’ English skill through digital video as multimodal for young learners in online learning. https://doi.org/10.2991/assehr.k.211223.040
Topuzov, O., Malykhin, O., & Aristova, N. (2020). The right mix of approaches in the English language teaching: Achieving desired learning outcomes in the ESP classroom. Society Integration Education Proceedings of the International Scientific Conference, 5, 558. https://doi.org/10.17770/sie2020vol5.4818
Venkatesh, V., Thong, J. Y. L., & Xu, X. (2016). Unified theory of acceptance and use of technology: A synthesis and the road ahead. Journal of the Association for Information Systems, 17(5), 328–376. https://ssrn.com/abstract=2800121
Widodo, H. P. (2016). Teaching English for specific purposes (ESP): English for vocational purposes (EVP) BT - English Language Teaching Today: Linking theory and practice (W. A. Renandya & H. P. Widodo (eds.); pp. 277–291). Springer International Publishing. https://doi.org/10.1007/978-3-319-38834-2_19
Yau, H. K., & Ho, C. S. J. (2023). The evaluation of Hong Kong students’ perception of e-books in higher education. TOJET: The Turkish Online Journal of Educational Technology, 22(3), 125–130. http://www.tojet.net/volumes/v22i3.pdf#page=135
Yılmaz, Y., & Üstündağ, M. T. (2023). E-Learning performance assessment model proposal for e-learning academies. International Journal of Curriculum and Instructional Studies (IJOCIS), 13(2), 404–423. https://eric.ed.gov/?id=EJ1413288
Yuniarti, Y., Yulian, R., Yuniarti, Y. (2022). Digital story telling based on multimodal elements on EFL learners’ speaking performance. JPI (Jurnal Pendidikan Indonesia), 11(2), 308–316. https://doi.org/10.23887/jpiundiksha.v11i2.40217
Zhang, R. (2021). Blended course evaluation in the context of English for specific purposes: Accountability and development. Sage Open, 11(4), 215824402110545. https://doi.org/10.1177/21582440211054502
Downloads
Additional Files
Published
Issue
Section
License
Authors retain copyright and grant the journal the right to publish the work, which is simultaneously made available under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence: https://creativecommons.org/licenses/by/4.0/
