The Differences between the Adaptive Computerized and Linear Testings in Terms of Estimating Ability According to the Item Response Theory among the Faculty of Education Students, Al-Azhar University

Document Type : Original Article

Author

قسم علم النفس التعليمي والإحصاء التربوي بکلية التربية بنين، جامعة الأزهر بالقاهرة.

Abstract

This research aimed to compare the computerized adaptive and linear testings concerning estimating ability by the two methods of maximum liklihood and bayes and by using the two rules of terminating the test with a specified number of items and with minimum standard error. The number of the participants in the research were (100) students of the third year at the College of Education for Boys, Al-Azhar University in Cairo. The researcher used a question bank in some chapters of the educational psychology course (1) (Learning Psychology). The researcher used paired samples  t- test to verify the research hypotheses. The researcher relied on the following statistical programs SPSS 25, jMetrik 4.1.1, FastTest 3.75.39. The results of the research revealed that there were no statistically significant differences between the two means of assessing ability of computerized linear testing and computerized adaptive testing concerning using the maximum likelihood method or bayes to estimate ability and the termination test rule with a specified number of items or minimum standard error. The results also revealed that there were statistically significant differences between the two means of standard error of assessing ability of computerized linear testing and computerized adaptive testing using the maximum likelihood method or bayes to estimate ability and the termination test rule with a specified number of items or minimum standard error in favor of the computerized adaptive testing.                                                                                             

Keywords

References

Al-Hossini, Hisham. (2019, November 6-7). Mechanisms of activating study of evaluating public exams in the Arab world and developing it. The Eleventh Conference of Arab Ministers of Education in Bahrain "Educational policies and their role in achieving the fourth goal of sustainable development goals 2030". League of Arab States, Arab Organization for Education, Culture and Science, Manama, Kingdom of Bahrain.
Al-Hossini, Hisham, & Al- Adroos, Mervet. (2014, February 6). Using computerized adaptive testing to measure and diagnose learners' mental abilities. Working paper presented to e-learning conference in pre-university education institutions, National Center for Examinations and Educational Evaluation, Cairo, Egypt.
 
 
Al-Saadany, Mohammed.(2019). Electronic test patterns (adaptive, middle, Linear) and their effect on  interaction with test anxiety level (abnormal - pathological) on development of achievement and reduce anxiety among educational technology students. Journal of Educational Technology, Egyptian Association for Educational Technology, 29 (8), 11-94.
Al-Subaihi, Ali, & Al-Ahmadi, Sharaf. (2011). Computerized adaptive testing as an application  of developing evaluation methods  and electronic tests. Journal of  Human and Educational Studies, Faculty of Education, Damanhour University, 3(1), 83-126. 
Al-Kramenh, Mohammed. (2015). Constructing A computerized adaptive testing of logical mathematial intelligence for gifted students using item response theory [Doctoral dissertation, Jordanian University, Jordan[. Dar Al-Mandumah Database, Dissertations.
El-Kayal, Mokhtar, Nour Al-Deen, Amin, & Al-Dhanhani, Maysoon. (2017). Psychometric properties of  a computerized adaptive testing for diagnosing mathematics learning disabilities. Journal of Psychological Counseling, Ain Shams University, 50 (2), 213-254.
Danaa, Zeenat. (2012). Building a computerized adaptive testing in a course of individual differences psychology according to the hierarchical strategy (multi-level) in item response theory at Taibah University. Journal of Education World, The Arab Foundation for Scientific Consultation and Human Resource Development, 37 (13), 153-186.
Soliman, Amin. (2009). Measurment and evaluation in human since (basics, tools, application). Dar Al-Kitab Al-Hadith.
Allam, Salah Al-Deen. (2005). Unidimensional and multidimensional item response models and their applications in psychological and educational measurement. Dar Al-Fikr Al-Arabi.
Allam, Salah Al-Deen. (2006). Educational and psychological measurement and evaluation: its basics, applications and contemporary trends. Dar Al-Fikr Al-Arabi.
Omara, Ehab, & Kazem, Ali.(2020). A Feasibility study for developing a computerized adaptive Form of  colored sequential matrices test for Omani’s children using the item response theory. International Journal for Educational Researches, United Arab Emirates University, 44(3), 142-181.
Odeh, Ahmed, & Obaidat, Omar.(2013). The Effectiveness of computerized adaptive testing in estimating mental ability using Raven’s matrices. Journal of Educational Sciences Studies, Jordanian University, 40 (2), 1602-1621.
Mohammed, Mohammed.(2017). Computrized adaptive testing: Develping item banking. Journal of Educational Creations, Association of Arab’s Educators, (3), 11-20.
Morad, Salah, & Soliman, Amin.(2005).  Tests and measures in psychological and educational sciences steps of preparing it and their characteristics (2 nd ed). Dar Al-Kitab Al-Hadith.
Mussallum, Fuad.(2011). A Proposed style for selecting items and estimating ability in computer adaptive testing [Doctoral dissertation, University of Yarmouk, Jordan[. Dar Al-Mandumah Database, Dissertations.
Nour Al-Deen, Amin.(2006). The Effect of behavior for revision and answer-changing item on estimating ability and sufficiency of measurment in computerized adaptive testing. Egyptian Association for Psychological Studies, 53(16), 75-132.
Nour Al-Deen, Amin.(2019). The Effectiveness of computerized adaptive measurment in measuring the achievement of university students. Saudi Journal of Psychological Sciences, King Saud University, Saudi Association for Educational and Psychological Sciences, (64), 29-47.
 ثالثًا: المراجع الأجنبية: 
Aybek, E., & Gulleroglu, H.(2021). Attitudes toward pirated content: A scale development study based on graded response model. Eurasian Journal of Educational Research,91, 127-144.                                 
Babcock, B., & Weiss, D. (2012). Termination criteria in computerized adaptive tests: do variable-length CATs provide efficient and effective measurement?Journal of Computerized Adaptive Testing, 1(1), 1-18.
Bjorner, J., Chang, C., Thissen, D., & Reeve, B.(2007). Developing tailored instruments: item banking and computerized adaptive assessment. Quality of Life Research, 16, 95-108.
Boyd, A.(2003).  Strategies for controlling testlet exposure rates in computerized adaptive testing systems [Doctoral dissertation, University of Texas at Austin, United States of America[. ProQuest Dissertations and Theses Global.
Cella, D., Gershon, R., Lai, J., & Choi, S.(2007). The future of outcomes measurment: item banking, tailored short-forms, and computerizd adaptive assessment. Quality of Life Research, 16, 133-141.
Chang, C .(2005). Developing tailored instruments: item banking and computerized adaptive assessment. Evaluation and Program Planning Journal ,24, 215-251.
Cheng, I., & Basu, A.(2009). Interactive graphis for computer adaptive testing. Computer Graphics Forum, 28(8), 2033–2045.
Choi, S., Grady, M., & Dodd, B.(2011). A new stopping rule for computerized adaptive testing. Educational and Psychological Measurment, 71(1), 37-53.
Cisar, S., Radosav, D., Markoski, B., Pinter, R., & Cisar, P.(2010). Computer adaptive testing of student knowledge. Acta Polytechnical Hungarica, 7(4), 139-152.
Davey, T.(2011). A Guide to computer adaptive testing systems. the Council of Chief State School Officers.
Davey, T., & Pitoniak, M.(2006). Designing computerized adaptive tests. In S. Dowing & T. Haladyna (Eds.), Handbook of test development (543-574). Lawrence Erlbaum Associates.
 Domino, G., & Domino, M.(2006). Psychological testing: An introduction (2 nd ed). Cambridge University Press. 
Embretson, S., & Reise, S. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates.
Erdem-Kara,B. (2019). Computer adaptive testing simulations in R. International Journal of Assessment Tools in Education, 6(5), 44-56.
Finkelman, M.(2004). Statistical isssues in computerized adaptive testing [Doctoral dissertation, University of Stanford, California, United States of America[.ProQuest Dissertations and Theses Global.
Finkelman, M., Weiss, D., & Kim-Kang, G.(2010). Item selection and hypothesis testing for the adaptive measurment of change. Applied Psychological Measurement, 34(4), 238–254.
Hogenboom, S., Hermans, F., & Van der Mass, H.(2021). Computerized adaptive assessment of understanding of programming concepts in primary school children. Computer Scince Education, https://doi.org/10.1080/08993408.2021.191446.
Hol, A. Vorst, H., & Mellenberg, G.(2008). Computerized adaptive testing of personality traits. Journal of Psychology, 216(1),            12-21.
Istiyono, E., Brams, W., Setiawan, R., & Megawati, I.(2020).  Developing of computerized adaptive testing to measure physics higher order thinking skills of senior high school students and its feasibility of use. European Journal of Educational Research, 9(1),91-101.
Jacobusse, G., & Van Buuren, S.(2007). Computerized adaptive testing for measuring development of young children. Statistics in Medicine, 26, 2629-2638.
Jian-quan, T., Dan-min, M., Xia, Z., & Jing-jing, G. (2007). An Introduction to the computerized adaptive testing. Us- China Education Review, 4(1), 72-81.
Kalender, I., & Berberoglu, G.(2017). Can computerized adaptive testing work in students’ admission to higher education programs in Turkey? Educational Sciences: Theory & Practice, 17(2),573-596.
Leroux, A., & Dodd, B. (2014). A comparison of stopping rules for computerized adaptive screening measures using the rating scale model. Journal of Applied Measurement, 15, 213-226.
Leroux, A. & Dodd, B. (2016). A Comparison of exposure control procedures in CATs using the GPC model. Journal of Experimental Education, 84(4), 666-685.
Linacre, J.(2000). Computer adaptive testing: a methodology whose time has com. Komesa Press.
Magis, D., Yan, D., & Von Davier, A.(2017). Computerized adaptive testing and multistage testing with R using packages catR and mstR. Springer International Publishing.
Martin,A.,& Lazendic, G.(2018). Computer-adaptive testing: Implications for students’ achievement, motivation, engagement, and subjective test experience. Journal of Educational Psychology, 110(1), 27-45.
Mizumoto, A., Sasao, Y., & Webb, S.(2019). Developing and evaluating a computerized adaptive testing version of the word part levels test. Language Testing, 36(1), 101-123.
Ozyurt, H., & Ozyurt, O. (2015). Ability level estimation of students on probability Unit Via computerized adaptive testing.  Eurasian Journal of Educational Research, (58), 27-44.
Ozyurt, H., Ozyurt, O., Baki, A., & Guven B.(2012). An Application of individualized assessment in educational hypermedia: design of computerized adaptive testing system and its integration into UZWEBMAT. Procedia- Social and Behavioral Science, 46, 3191-3196.
Ramadan, A., & Aleksandrovna, E.(2018, November 27-29). Computerized adaptive testing. Conference paper: Information Technology in Education, Saratov National Research State University, Russia.
Rezaie, M., & Golshan, M.(2015). Computer adaptive test (CAT): Advantages and limitations. International Journal of Educational Investigations, 2(5), 128-137.
Runder, L.(1998). An on- line, interactive, computer adaptive testing tutorial. ERIC Clearinghouse on Assessment and Evaluation.
Runder, L.(2010). Implementing the graduate management admission test computerized adaptive test. In W. Van der Linden & C., Glas (Eds.), Elements of adaptive testing (151-166). Springer.
 
 
Sahin, A., & Weiss, D.(2015). Effects of calibration sample size and item  bank size on ability estimation in computerized adaptive testing. Educational Sciences: Theory & Practice, 15(6), 1585-1595.
Samsudin, M., SomChut, T., & Ismail, M.(2019). Evaluating computerized adaptive testing efficiency in measurement student’s performance in science timss.  Indonesian Journal of Science Education, 8(4), 547-560.
Stafford, R., Runyon, C., Casabianca, J., & Dodd, B. (2019). Comparing computer adaptive testing stopping rules under the generalized partial-credit model. Behavior Research Methods, 51(3), 1305–1320.
Su, Y.(2018). A Comparison of constrained item selection method in multidimensional computerized adaptive testing. Applied Psychological Measurment, 40(5), 346-360.
Thompson, N., & Weiss, D.(2011). A Framework for the development of computerized adaptive tests. Practical Assessments, Research & Evaluation, 16(1), 1-9.
Tian X., & Dai , B.(2020). Developing a computerized adaptive test to assess stress in Chinese college students. Frontiers in Psychology, 11(7), 1-9.
Triantafillou, E., Georgiadou, E., & Economides, A.(2008). The Design and evaluation of a Computerized  adaptive test on mobile devices. Computers & Education, 50(4),1319-1330.
Van der Linden, W., & Pashley, P.(2010). Item selection and ability estimation in adaptive testing. In W. Van der Linden & C., Glas (Eds.), Elements of adaptive testing (3-30). Springer.
Veldkamp, B.(2016). On the issue of itme selection in computerized adaptive testing with response times. Journal of Educational Measurement, 53(2), 212-228.
Wang, C., Weiss, D., & Shang, Z.(2019). Variable-length stopping rules for multidimensional computerized adaptive testing. Psychometrika, 84(3), 749-771.
Weiss, D.(2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development, 37(2), 70-84.       
Weiss, D.(2011). Better data from better measurements using computerized adaptive testing. Journal of Methods and Measurement in the Social Sciences, 2(1), 1-27. 
Yasuda, J., Mae, N., Hull, M., & Taniguchi, M.(2021).  Optimizing the length of computerized adaptive testing for the force concept inventory. The American Physical Society, Physical Review Physics Education Research, 17, 1-15.
 

Files

Share

How to cite

Statistics