TRANSFIGURATIONS OF THE COMPUTERIZED MATHEMATICS FINAL EXAM: HOW WAS THE RESULTS OF COMPUTERIZED ADAPTIVE TEST-BASED ASSESSMENT?
DOI:
https://doi.org/10.26418/jpmipa.v15i2.69485Keywords:
Keywords, computerized adaptive test, final exam, mathematics, rasch model.Abstract
Abstract
Computerized Adaptive Test (CAT) allows the use of targeted tests, that is, each test taker obtains items that match his or her ability level. The purpose of this research is to find out the final semester exam with the CAT model. The research methodology is descriptive quantitative. Purposive sampling technique was used to take a sample of 73 students. The results of this study indicate that the CAT used in this semester final exam based on Moodle can select test items given to test takers adaptively according to the user's ability level. The very high ability category was 42 students, the high ability category was 11 students, the medium ability category was 9 students, the low ability category was 4 students, and the very low ability category was 7 students. Overall, the ability of learners is in the high category.
References
Abidin, A. Z., Istiyono, E., Fadilah, N., & Dwandaru, W. S. B. (2019). A computerized adaptive test for measuring the physics critical thinking skills. International Journal of Evaluation and Research in Education, 8(3), 376–383. https://doi.org/10.11591/ijere.v8i3.19642
As’ari, A. R., Mahmudi, A., & Nuerlaelah, E. (2017). Our prospective mathematic teachers are not critical thinkers yet. Journal on Mathematics Education, 8(2), 145–156. https://doi.org/10.22342/jme.8.2.3961.145-156
Atikah, A., Sudiyatno, S., Rahim, A., & Marlina, M. (2022). Assessing the item of final assessment mathematics test of junior high school using Rasch model. Jurnal Elemen, 8(1), 117–130. https://doi.org/10.29408/jel.v8i1.4482
Aybek, E. C., & Demirtasli, R. N. (2017). Computerized adaptive test (CAT) applications and item response theory models for polytomous items. International Journal of Research in Education and Science, 3(2), 475–487. https://doi.org/10.21890/ijres.327907
Cetin, Y., Mirasyedioglu, S., & Cakiroglu, E. (2019). An inquiry into the underlying reasons for the impact of technology enhanced problem-based learning activities on students’ attitudes and achievement. Eurasian Journal of Educational Research, 2019(79), 191–208. https://doi.org/10.14689/ejer.2019.79.9
Chan, S. W., Looi, C. K., & Sumintono, B. (2021). Assessing computational thinking abilities among Singapore secondary students: a Rasch model measurement analysis. Journal of Computers in Education, 8(2), 213–236. https://doi.org/10.1007/s40692-020-00177-2
Choi, Y., & McClenen, C. (2020). Development of adaptive formative assessment system using computerized adaptive testing and dynamic bayesian networks. Applied Sciences (Switzerland), 10(22), 1–17. https://doi.org/10.3390/app10228196
Clark, L. A., & Watson, D. (2019). Constructing Validity: New Developments in Creating Objective Measuring Instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/https://dx.doi.org/10.1037/pas0000626
Delgado-Gómez, D., Laria, J. C., & Ruiz-Hernández, D. (2019). Computerized adaptive test and decision trees: A unifying approach. Expert Systems with Applications, 117, 358–366. https://doi.org/10.1016/j.eswa.2018.09.052
Diego, L. A. B. (2017). Friends with Benefits: Causes and Effects of Learners’ Cheating Practices During Examination. IAFOR Journal of Education, 5(2), 121–138. https://doi.org/10.22492/ije.5.2.06
Dillon, E. W., & Smith, J. A. (2017). Determinants of the match between student ability and college quality. Journal of Labor Economics, 35(1), 45–66. https://doi.org/10.1086/687523
Ernest, P. (2018). The Ethics of Mathematics: Is Mathematics Harmful? Springer International Publishing. https://doi.org/10.1007/978-3-319-77760-3_12
Fadzil, H. M. (2018). Designing infographics for the educational technology course: Perspectives of preservice science teachers. Journal of Baltic Science Education, 17(1), 8–18.
Febliza, A., & Okatariani, O. (2020). The Development of Online Learning Media by Using Moodle for General Chemistry Subject. Journal of Educational Science and Technology (EST), 6(1), 40–47. https://doi.org/10.26858/est.v6i1.12339
FM, Lord. (1980). Applications of item response theory to practical problems. Lawrence Erlbaum Associates.
Frey, A., Spoden, C., & Born, S. (2020). Construction of Psychometrically Sound Written University Exams. Psychological Test and Assessment Modeling, 6(4), 415–525.
Furner, J. (2017). Teachers and Counselors: Building Math Confidence in Schools. European Journal of STEM Education, 2(2), 1–10. https://doi.org/10.20897/ejsteme.201703
Futri, V. I., Rosnawati, R., Rahim, A., & Marlina, M. (2022). Rasch Model Study on Mathematics Examination Test Using Item Response Theory Approach. International Journal on Emerging Mathematics Education, 6(1), 29. https://doi.org/10.12928/ijeme.v6i1.21761
Gibbons, R. D., & deGruy, F. V. (2019). Without Wasting a Word: Extreme Improvements in Efficiency and Accuracy Using Computerized Adaptive Testing for Mental Health Disorders (CAT-MH). Current Psychiatry Reports, 21(8), 1–9. https://doi.org/10.1007/s11920-019-1053-9
Gresham, G. (2018). Preservice to inservice: does mathematics anxiety change with teaching experience? Journal of Teacher Education, 69(1), 90–107. https://doi.org/10.1177/0022487117702580
H, W. (1990). Computerized adaptive testing: a primer. LEA.
H, W., & NJ, D. (2000). Computerized Adaptive Testing: A Primer. NJ: Lawrence Erlbaum.
Hadi, S. (2013). Pengembangan Computerized Adaptive Test Berbasis Web. Aswaja Pressindo.
Hadi, S., Haryanto, H., AM, M. A., Marlina, M., & Rahim, A. (2022). Developing Classroom Assessment Tool using Learning Management System-based Computerized Adaptive Test in Vocational High Schools. Journal of Education Research and Evaluation, 6(1), 143–155. https://doi.org/10.23887/jere.v6i1.35630
Hambleton, R. K. Swaminathan, H., & Rogers, H. J. (1985). Item Response Theory: Principles and Application. Kluwer Inc.
Hambleton, R. K., Swaminathan, H., & Rogers, D. J. (1991). Fundamentals of Item Response Theory Library of Congress Cataloging-in-Publication Data.
Haryanto. (2011). Pengembangan Computerized Adaptive Testing (CAT) dengan Algoritma Logika Fuzzy. Jurnal Penelitian Dan Evaluasi Pendidikan, 15(1), 47–70. https://doi.org/https://doi.org/10.21831/pep.v15i1.1087
Hendriana, H., Prahmana, R. C. I., & Hidayat, W. (2019). The innovation of learning trajectory on multiplication operations for rural area students in Indonesia. Journal on Mathematics Education, 10(3), 397–408. https://doi.org/10.22342/jme.10.3.9257.397-408
Istiyono, E., Dwandaru, W. S. B., & Faizah, R. (2018). Mapping of physics problem-solving skills of senior high school students using PhysProSS-CAT. Research and Evaluation in Education, 4(2), 144–154. https://doi.org/10.21831/reid.v4i2.22218
Istiyono, E., Dwandaru, W. S. B., Setiawan, R., & Megawati, I. (2019). Developing of Computerized Adaptive Testing to Measure Physics Higher Order Thinking Skills of Senior High School Students and its Feasibility of Use. European Journal of Educational Research, 9(1), 91–101. https://doi.org/https://doi.org/10.12973/eu-jer.9.1.91
Jamiludin, Darnawati, & Uke, W. A. S. (2017). Students’ Perception Towards National Examination 2017: Computer-Based Test or Paper-Based Test. Mediterranean Journal of Social Sciences, 8(4), 139–144. https://doi.org/10.5901/mjss.2017.v8n4s1p139
Kenedi, ary K., Helsa, Y., Ariani, Y., Ainil, M., & Hendri, S. (2019). Mathematical Connection of Elementary School Students to Solve Mathematical Problems. Journal on Mathematics Education, 10(1), 69–79.
Kezer, F. (2021). The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing. International Journal of Assessment Tools in Education, 8(1), 145–155. https://doi.org/10.21449/ijate.735155
Kusuma, A. P., Waluya, S. B., Rochmad, R., & Mariani, S. (2021). Algebra Problem Solving Ability Based on Solo Taxonomy Assessed From Cognitive Style. Jurnal Pendidikan Matematika Dan IPA, 12(2), 148. https://doi.org/10.26418/jpmipa.v12i2.44911
Ling, G., Attali, Y., Finn, B., & Stone, E. A. (2017). Is a Computerized Adaptive Test More Motivating Than a Fixed-Item Test? Applied Psychological Measurement, 41(7), 495–511. https://doi.org/10.1177/0146621617707556
Magis, D., & Barrada, J. R. (2017). Computerized adaptive testing with R: Recent updates of the package catR. Journal of Statistical Software, 76(1), 1–19. https://doi.org/10.18637/jss.v076.c01
Magis, D., Yan, D., & Von Davier, A. A. (2017). Computerized adaptive and multistage testing with R: Using packages catR and mstR. Springer.
Mahanal, S., Zubaidah, S., Sumiati, I. D., Sari, T. M., & Ismirawati, N. (2019). RICOSRE: A learning model to develop critical thinking skills for students with different academic abilities. International Journal of Instruction, 12(2), 417–434. https://doi.org/10.29333/iji.2019.12227a
Makransky, G., Lilleholt, L., & Aaby, A. (2017). Development and validation of the Multimodal Presence Scale for virtual reality environments: A confirmatory factor analysis and item response theory approach. Computers in Human Behavior, 72, 276–285. https://doi.org/10.1016/j.chb.2017.02.066
Martin, A. J., & Lazendic, G. (2018). Computer-adaptive testing: Implications for students’ achievement, motivation, engagement, and subjective test experience. Journal of Educational Psychology, 110(1), 27–45. https://doi.org/https://doi.org/10.1037/edu0000205
MartÃnez-Sierra, G., & GarcÃa-González, M. del S. (2017). Students’ Emotions in the High School Mathematical Class: Appraisals in Terms of a Structure of Goals. International Journal of Science and Mathematics Education, 15(2), 349–369. https://doi.org/10.1007/s10763-015-9698-2
Mazana, M. Y., Montero, C. S., & Casmir, R. O. (2018). Investigating Students’ Attitude towards Learning Mathematics. International Electronic Journal of Mathematics Education, 14(1). https://doi.org/10.29333/iejme/3997
Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers and Education, 71, 56–76. https://doi.org/10.1016/j.compedu.2013.09.011
Noperta, N., & Sari, M. (2023). The Influence of Peer Tutoring-Based Humanistic Mathematics Learning on the Motivation of Learning Mathematics of High School Students. Jurnal Pendidikan Matematika Dan IPA, 14(1), 134–146. https://doi.org/http://dx.doi.org/10.26418/jpmipa.v14i1.53507
Plajner, M. (2016). Probabilistic Models for Computerized Adaptive Testing. Czech Technical University in Prague. http://arxiv.org/abs/1703.09794
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 1–14. https://doi.org/10.1103/PhysRevPhysEducRes.15.020111
Plummer, O. R., Abboud, J. A., Bell, J. E., Murthi, A. M., Romeo, A. A., Singh, P., & Zmistowski, B. M. (2019). A concise shoulder outcome measure: application of computerized adaptive testing to the American Shoulder and Elbow Surgeons Shoulder Assessment. Journal of Shoulder and Elbow Surgery, 28(7), 1–8. https://doi.org/10.1016/j.jse.2018.11.068
Pramono, A. J. B., & Retnawati, H. (2020). Implementation of cat in indonesia school: Current challenges and strategies. Universal Journal of Educational Research, 8(11), 5599–5609. https://doi.org/10.13189/ujer.2020.081164
Radović, S., Marić, M., & Passey, D. (2019). Technology enhancing mathematics learning behaviours: Shifting learning goals from “producing the right answer†to “understanding how to address current and future mathematical challenges.†Education and Information Technologies, 24(1), 103–126. https://doi.org/10.1007/s10639-018-9763-x
Rahim, A., Hamdi, S., & Arcana, I. N. (2020). Developing Bilingual Learning Multimedia in Integral Application Learning Material For Vocational School. Al-Jabar: Jurnal Pendidikan Matematika, 11(2), 201–210. https://doi.org/10.24042/ajpm.v11i2.6816
Rahim, A., & Haryanto, H. (2021). Implementation of Item Response Theory (IRT) Rasch Model in Quality Analysis of Final Exam Tests in Mathematics. Journal of Educational Research and Evaluation, 10(2), 57–65. https://doi.org/10.15294/jere.v10i2.51802
Ramirez, G., Shaw, S. T., & Maloney, E. A. (2018). Math Anxiety: Past Research, Promising Interventions, and a New Interpretation Framework. Educational Psychologist, 53(3), 145–164. https://doi.org/10.1080/00461520.2018.1447384
Rashid, A. H. A., Shukor, N. A., Tasir, Z., & Na, K. S. (2021). Teachers’ perceptions and readiness toward the implementation of virtual learning environment. International Journal of Evaluation and Research in Education, 10(1), 209–214. https://doi.org/10.11591/ijere.v10i1.21014
Rizbudiani, A. D., Jaedun, A., Rahim, A., & Nurrahman, A. (2021). Rasch model item response theory (IRT) to analyze the quality of mathematics final semester exam test on system of linear equations in two variables (SLETV). Al-Jabar : Jurnal Pendidikan Matematika, 12(2), 399–412. https://doi.org/10.24042/ajpm.v12i2.9939
Sahin, A., Hurtado Grooscors, H. A., & Góngora-Cortés, J. J. (2018). Review of FastTest: A Platform for Adaptive Testing. Measurement: Interdisciplinary Research and Perspectives, 16(4), 256–263. https://doi.org/10.1080/15366367.2018.1492867
Stiggins, R., & Chappuis, J. (2012). Introduction to student invoved assessment for learning (6th ed.). Addison Wesley.
Stufflebeam, D. L., & Shinkfield, A. J. (2012). Systematic evaluation: A self-instructional guide to theory and practice (G. F. Mada). Nijhoff Publishing.
Sufanti, M., & Santosa, C. A. H. F. (2021). The Influence of Stad Cooperative Strategies (Teaching AIDS and Multimedia Power Points) and Learning Style on Mathematics Learning Outcomes. Jurnal Pendidikan Matematika Dan IPA, 12(1), 40. https://doi.org/10.26418/jpmipa.v12i1.43180
Sumintono, B. (2014). Model Rasch untuk Penelitian Sosial Kuantitatif. ITS Surabaya, November 201, 1–9. http://deceng3.wordpress.com
Sumintono, B., & Widhiarso, W. (2015). Aplikasi Model Rasch untuk Penelitian Ilmu-ilmu Sosial. Trim Komunikata.
Thompson, G. (2017). Computer adaptive testing, big data and algorithmic approaches to education. British Journal of Sociology of Education, 38(6), 827–840. https://doi.org/10.1080/01425692.2016.1158640
Tigchelaar, M., Bowles, R. P., Winke, P., & Gass, S. (2017). Assessing the Validity of ACTFL Can-Do Statements for Spoken Proficiency: A Rasch Analysis. Foreign Language Annals, 50(3), 1–17. https://doi.org/10.1111/flan.12286
Toroujeni, S. M. H. (2021). Computerized testing in reading comprehension skill: investigating score interchangeability, item review, age and gender stereotypes, ICT literacy and computer attitudes. In Education and Information Technologies. Springer US. https://doi.org/10.1007/s10639-021-10584-2
Wang, S., Fellouris, G., & Chang, H. H. (2019). Statistical Foundations for Computerized Adaptive Testing with Response Revision. Psychometrika, 84(2), 375–394. https://doi.org/10.1007/s11336-019-09662-9
Wang, M. Te, Ye, F., & Degol, J. L. (2017). Who Chooses STEM Careers? Using A Relative
Downloads
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This work is licensed under a Creative Commons Attribution 4.0 International License
