Measuring what matters: A design-based development of English literacy assessment integrating climate literacy for EFL learners

Authors

  • Endah Purwandani Magister Pendidikan Bahasa, Universitas Sriwijaya
  • Sary Silvhiany Universitas Sriwijaya
  • Rita Inderawati Magister Pendidikan Bahasa Universitas Sriwijaya

DOI:

https://doi.org/10.26418/jeltim.v7i2.96878

Keywords:

AKM, climate change, education, ecoliteracy, English literacy

Abstract

Integrating climate literacy into education is increasingly crucial to prepare learners as responsible global citizens capable of addressing complex environmental challenges and promoting sustainable development. In the context of English Language Teaching (ELT), embedding climate-related content can serve a dual purpose. It enhances linguistic competence while cultivating ecological awareness, critical thinking, and social responsibility. However, there remains a lack of assessment instruments specifically designed to evaluate both English literacy and climate literacy simultaneously in EFL contexts. Therefore, this study aimed to develop a valid and reliable PISA-based English literacy assessment that integrates climate literacy for EFL learners. Employing a Design-Based Research (DBR) framework with Tessmer’s formative evaluation model, the study followed five iterative stages: self-evaluation, expert review, one-to-one evaluation, small-group testing, and field testing. The participants consisted of 422 eighth-grade students and six experts in English education and environmental literacy. Data analysis combined quantitative and qualitative approaches. Quantitative data were examined using Aiken’s V, item discrimination and difficulty indices, and Cronbach’s Alpha to determine validity and reliability. While qualitative insights were drawn from interviews and classroom observations. The findings revealed that 40 items were valid, with high reliability (α = 0.838) and balanced item difficulty. The instrument demonstrated strong content validity (Aiken’s V = 0.90–1.00) and effectively measured English literacy alongside ecoliteracy domains, encompassing knowledge (head), attitude (heart), and action (hand). The results underscore the potential of this dual-purpose assessment model to align language education with Indonesia’s curriculum transformation and the broader goals of global sustainability education worldwide. How to cite this paper: Purwandani, E., Shilviany, S., & Inderawati, R. (2025).Measuring What Matters: A Design-Based Development of English Literacy for EFL Learners. Journal of English Language Teaching Innovations and Materials (Jeltim), 7(2), 163-181. http://dx.doi.org/10.26418/jeltim.v7i2.96878

Author Biography

Sary Silvhiany, Universitas Sriwijaya

Dr. Sary Silvhiany is a lecturer at English Education, Sriwijaya University. She earned her PhD from Indiana University Bloomington, USA and her MA from the University of Arizona, USA and M.Pd from Sriwijaya University. Her research focuses on literacy as social and critical practices, multilingual inquiry pedagogy, & Teacher Education.

References

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813

Bernadtua Simanjuntak, M., Sutrisno, S., Kurnia Putrawan, B., Steven Lumingkewas, M., & Hutabarat, C. (2023). Enhancing Environmental Literacy through English Language Education in Maritime Institutions. BIO Web of Conferences, 79. https://doi.org/10.1051/bioconf/20237911001

Bulan, A., Ramadhan, R. A., & Wahyuni, N. (2024). Need Analysis for Developing English Teaching Materials Using the Eco-ELT Approach to Enhance Environmental Literacy Among High School Students in Dompu Regency. Jurnal Pendidikan Dan Pembelajaran Indonesia (JPPI), 4(1), 72–80. https://doi.org/10.53299/jppi.v4i1.417

Capra, F. (1996). The Web of Life: A New Understanding of Living Systems. In Anchor Book (pp. 1–347). https://www.researchgate.net/publication/31665646_The_web_of_life_a_new_scientific_understanding_of_living_systems_F_Capra

Cates, K. A. (2002). Teaching for a Better World: Global Issues and Language Education. Human Rights Education in Asian Schools, 5, 41–52.

Hong, Y., & Zhang, J. (2023). An Attempt of Construction of English Language Intelligent Flipped Classroom Teaching Mode from an Ecological Perspective. Asia-Pacific Journal of Humanities and Social Sciences, 3(1), 195–208. https://doi.org/10.53789/j.1653-0465.2023.0301.022

Incesu, O., & Yas, M. A. (2024). The relationship between nursing students’ environmental literacy and awareness of Global Climate Change. Public Health Nursing, 41(1), 67–76. https://doi.org/10.1111/phn.13255

Jasmine, M. P., Silvhiany, S., & Inderawati, R. (2025). Exploring Young Learners’ Needs in Multimodal Text Design for Integrating Sustainable Education into ELT. Journal of Languages and Language Teaching, 13(2), 785. https://doi.org/10.33394/jollt.v13i2.13775

Kaya, Volcan Hasan, Eister, D. (2018). German Students’ Environmental Literacy in Science Education Based on PISA Data. Science Education International, 29(2), 75–87. https://doi.org/10.33828/sei.v29.i2.2

Kemendikbudristek. (2022). Peraturan Menteri Pendidikan,Kebudayaan,Riset, dan Teknologi Republik Indonesia Nomor 9 Tahun 2022 tentang Evaluasi Sistem Pendidikan Oleh Pemerintah Pusat dan Pemerintah Daerah Terhadap Pendidikan ANak Usia Dini, Pendidikan Dasar dan Pendidikan Menengah (Issue 308).

Kemendikbudristek, P. A. P. B. (2023). Framework Asesmen Kompetensi Minimum (AKM).

Masitoh, M., Silvhiany, S., & Inderawati, R. (2025). Integrating Climate Change Education into Secondary School English Lessons: A Project-Based. 12(1), 84–102. https://doi.org/10.33394/jo-elt.v12i1.15272

McBride, B. B., Brewer, C. A., Berkowitz, A. R., & Borrie, W. T. (2013). Environmental literacy, ecological literacy, ecoliteracy: What do we mean and how did we get here? Ecosphere, 4(5). https://doi.org/10.1890/ES13-00075.1

Micalay-Hurtado, M. A., & Poole, R. (2022). Eco-critical language awareness for English language teaching (ELT): Promoting justice, wellbeing, and sustainability in the classroom. Journal of World Languages, 8(2), 371–390. https://doi.org/10.1515/jwl-2022-0023

Mujib, M., Mardiyah, M., & Suhermen, S. (2023). Development and psychometric properties of a questionnaire on environmental literacy in Indonesian secondary school. Revista Fuentes, 25(3), 267–282. https://doi.org/10.12795/revistafuentes.2023.22642

Nurhaliza, S., Muhaimin, A., Jasmine, M. P., & Silvhiany, S. (2024). Exploring the integration of climate change education: Teachers’ perspectives from p5 projects and english lessons in secondary school. The Journal of English Literacy Education, 11(1), 62–73. https://jele.ejournal.unsri.ac.id/index.php/jele/article/view/35/21

Nurhaliza, S., Silvhiany, S., & Inderawati, R. (2024). Integrating climate change education in English lessons and P5 projects in Junior High School. Journal of Languages and Language Teaching, 12(4), 1926-1938. DOI: https://doi.org/10.33394/jollt.v12i4.12622

OECD. (2023). PISA 2022 Assessment and Analytical Framework: Science, Reading, Mathematics and Global Competence.

Putri, C., Silvhiany, S., & Inderawati, R. (2024). Empowering Young Learners: Integrating Climate Change Education With Bilingual Picture Books in Elt. English Review: Journal of English Education, 12(2), 601–616. https://doi.org/10.25134/erjee.v12i2.9889

Rosli, M. S., Saleh, N. S., Alshammari, S. H., Ibrahim, M. M., Atan, A. S., & Atan, N. A. (2021). Improving Questionnaire Reliability using Construct Reliability for Researches in Educational Technology. International Journal of Interactive Mobile Technologies, 15(4), 109–116. https://doi.org/10.3991/IJIM.V15I04.20199

Saiful, J. A., & Setyorini, A. (2022). Ecocriticism Course: Development of English Pre-service Teachers’ Pedagogical Content Knowledge of Sustainability. Journal of Teacher Education for Sustainability, 24(2), 5–18. https://doi.org/10.2478/jtes-2022-0013

Salsabila, Z., Sholeh, M., Sulaeman, N. F., Junus, M., & Nuryadin, A. (2024). Assessing Environmental Literacy: Comparative Analysis of Students in Environment-Focused and General Junior High Schools in Balikpapan. Kasuari: Physics Education Journal (KPEJ), 7(1), 23–30. https://doi.org/10.37891/kpej.v7i1.505

Silvhiany, S., Kurniawan, D., & Safrina, S. (2023). Climate change awareness in ELT: Ethnography in connected learning and ecojustice pedagogy. Journal of English Language Teaching Innovations and Materials (Jeltim), 5(2), 91. https://doi.org/10.26418/jeltim.v5i2.63548

Tessmer, M. (1993). Planning and Conducting Formative Evaluations. Kogan Page Limited.

Tian, H., & Chen, S. (2023). Structural Analysis of Environmental Literacy of Urban Residents in China—Based on the Questionnaire Survey of Qingdao Residents. Sustainability (Switzerland), 15(6). https://doi.org/10.3390/su15065552

Ünal Gezer, M. (2022). Turkey’s Primary English-as-a-foreign Language (EFL) Coursebooks: An Investigation of Emergent Literacy Tasks. Cukurova University Faculty of Education Journal, 51(1), 220–236. https://doi.org/10.14812/cuefd.982901

UNESCO. (2020). Education for sustainable development: A roadmap. https://doi.org/10.1111/j.2048-416x.2009.tb00140.x

Veisi, H., Lacy, M., Mafakheri, S., & Razaghi, F. (2019). Assessing environmental literacy of university students: A case study of Shahid Beheshti University in Iran. Applied Environmental Education and Communication, 18(1), 25–42. https://doi.org/10.1080/1533015X.2018.1431163

Wang, X., Jiang, L., Fang, F., & Elyas, T. (2021). Toward Critical Intercultural Literacy Enhancement of University Students in China From the Perspective of English as a Lingua Franca. SAGE Open, 11(2). https://doi.org/10.1177/21582440211027544

Widyaningrum, L., Rizal, D., & Prayogo, A. (2022). Multimodal Project-Based Learning in listening and speaking activities: Building environmental care? EduLite: Journal of English Education, Literature and Culture, 7(2), 209. https://doi.org/10.30659/e.7.2.209-223

Xie, J., Yang, G., Wang, G., & Xia, W. (2021). How do network embeddedness and environmental awareness affect farmers’ participation in improving rural human settlements? Land, 10(10). https://doi.org/10.3390/land10101095

Downloads

Published

2025-11-01

Issue

Vol. 7 No. 2 (2025): October 2025

Section

Articles

License

Authors retain copyright and grant the journal the right to publish the work, which is simultaneously made available under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence: https://creativecommons.org/licenses/by/4.0/