Differential Item Functioning of Sternberg Triarchic Abilities Test  in light of Item Response Theory  and Multi-group Confirmatory Factor Analysis among University Students

Gad Allah, Ashraf Saad Mohammed

doi:10.21608/jsrep.2024.342631

Differential Item Functioning of Sternberg Triarchic Abilities Test in light of Item Response Theory and Multi-group Confirmatory Factor Analysis among University Students

Document Type : Original Article

Author

Ashraf Saad Mohammed Gad Allah

Lecturer of Educational Psychology and Statistics, Faculty of Education for Boys in Cairo, Al-Azhar University.

10.21608/jsrep.2024.342631

Abstract

This research aimed at investigating the Differential Item Functioning (DIF) of Sternberg Triarchic Abilities Test (STAT) in light of Item Response Theory (IRT) according to the variables (gender- specialization). It also aimed at investigating the Factorial structure of STAT according to the variables (gender- specialization). For instruments, the researcher translated and used STAT (prepared by Sternberg, 1993). Participants were (2907) male and female university students enrolled in faculties of education, Al-Azhar University. They were distributed according to gender: males (1870) and females (1037), and according to specialization: scientific (1229) and literary (1678). Using statistical programs represented in (jMetriK- BILOG-MG- SPSS –AMOS) and statistical methods represented in Likelihood Ratio Test and Multi-group Confirmatory Factor Analysis (MGCFA), the results revealed that there were (8) items in STAT showed differential functioning according to the gender: two items in favor of females, and six items in favor of males. Also, there were (6) items in STAT showed differential functioning according to the specialization in favor of the scientific specialization. In addition, the results revealed that the factorial invariance (configural, metric, and scalar) was achieved in the structure of STAT according to both gender and specialization, but the strict invariance was not achieved. The results were discussed and interpreted based on the theoretical framework and previous research results.

Keywords

References

Ajmi, M., Mustakim, S., Roslan, S. & Almehrizi, R. (2023). Detecting the Differential Item Functioning of Numerical Ability Test in the Gulf Multiple Mental Abilities Scale by Mental-Haenszel and Likelihood Ratio Test. International Journal of Academic Research in Progressive Education and Development, 12(2), 2400–2415.

Ali, S. & Ali, H. (2022). Reflective thinking and successful intelligence and
their relationships to self-Efficacy among kindergarten teachers. Journal of Positive School Psychology, 6(6), 4164-4179.

Almajeed, N. (2019). Factor structure of Successful intelligence test among preparatory school students. International Journal of Research in Social Sciences and Humanities, 1(9), 169-179.

American Educational Research Association (AERA), American Psychological Association (APA) & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington: American Educational Research Association.

Ayala, R. (2009). The Theory and Practice of Item Response Theory. New York: The Guilford Press.

Bock, R. & Gibbons, R. (2021). Item response theory. USA: John Wiley & Sons, Inc.

Boone, W., Staver, J. & Yale, M. (2014). Rasch Analysis in the
Human Sciences. New York: Springer Science Business Media.

Bosman, M. (2023). A Comparison of the efficacies of differential item functioning detection methods. International Journal of Assessment Tools in Education, 10 (1), 145–159.

Brody, N. (2003). Construct validation of the Sternberg Triarchic Abilities Test
Comment and reanalysis. Intelligence, 31, 319–329.

Brown, T. (2006). Confirmatory factor analysis for applied research. New York: The Guilford Press.

Chooi, W., Long, H. & Thampson, L. (2014). The Sternberg Triarchic Abilities Test (Level-H) is Measure of g. Journal of Intelligence. 2, 56-67.

Demars, C. (2010). Item response theory. New York: Oxford University Press, Inc.

Elosua, P. & Hambleton, R. (2018). Psychological and educational test score comparability across groups in the presence of item bias. Journal of Psychology and Education, 13(1), 23-32.

Elst, W., Ouwehand, C., Rijan, P., Lee, N., Boxtel, M., & Jolles, J. (2013). The shorting raven standard progressive matrices: Item response theory- based Psychometric analysis and normative data. Assessment, 20(1), 48- 59.

Embretson, S., & Reise, S. (2000). Item Response theory for Psychologists. New Jersey: Lawrence Erlbaum Associates.

Flowers, c., Raju, N. & Oshima, T. (2002, April 2-4). A Comparison of Measurement Equivalence Methods Based on Confirmatory Factor Analysis and Item Response Theory. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New Orleans, L.A., 1-28.

French, B., Hand, B., Therrien, W. & Vazquez, J. (2012). Detection of Sex
Differential Item Functioning in the Cornell Critical Thinking Test.
European Journal of Psychological Assessment, 28 (3), 201-207.

Gnambs, T. & Batinic, B. (2011). Evaluation of measurement precision with
Rasch-type models: The case of the short generalized opinion
leadership scale. Personality and Individual Differences, 50, 53-
58.

Gyamfi, A. (2023). Differential Item Functioning of Performance-Based Assessment in Mathematics for Senior High Schools. Journal of Evaluation and Education, 5(1), 1-17.

Holmefur, M., Sundberg, K., Wettergren, L. & Langius-Eklof, A. (2015).
Measurement properties of the 13-item sense of coherence scale
using Rasch analysis. Quality of Life Research, 24(6), 1455-1463.

Jiang. N. (2022). Investigating performance of model fit indices in multiple-group confirmatory factor analysis: Complications with or analysis: complications with ordinal data. (Unpublished doctoral dissertation), University of South Carolina

Karami, H. (2012). An Introduction to Differential Item Functioning. The International Journal of Educational and Psychological Assessment, 11(2), 59- 76.

Kim, J. & Oshima, T. (2012). Effect of Multiple Testing Adjustment in
Differential Item Functioning Detection. Educational and Psychological
Measurement, 73 (3), 458-470.

Kim, S., Kim, S. & Kamphaus, R. (2010). Is aggression the same for boys and girls? Assessing measurement invariance with confirmatory factor analysis and item response theory. School Psychology Quarterly. 25(1), 45-61.

KoKe, L. & Vernon, P. (2003). The Sternberg Triarchic Abilities Test (STAT) as a measure of academic achievement and general intelligence. Personality and Individual Differences, 35(8), 1803–1807.

Lai, J., Teresi, J. & Gershon, R. (2005). Procedures for the Analysis of
Differential Item Functioning (DIF) For Small Sample Sizes. Evaluation & the Health Professions, 28 (3), 283-294.

Langer, M. (2008). A reexamination of Lord's Wald test for differential item functioning using item response theory and modern error estimation. (Unpublished doctoral dissertation), University of North Carolina.

Lee, H. & Geisinger, K. (2016). The Matching Criterion Purification for Differential Item Functioning Analyses in a Large-Scale Assessment.
Educational and Psychological Measurement, 76(1), 141-163.

Li, X. & Wang, W. (2015). Assessment of differential item functioning under
cognitive diagnosis models: The DINA model example. Journal of Educational Measurement, 52(1), 28-54.

Linden, W. (2016). Unidimensional logistic response models. In W. Linden (Ed.), Handbook of item response theory, Volume One: Models (13-30). New York: Taylor& Francis Group.

Liu, R. & Bradley, K. (2021). Differential item functioning among English language learners on a Large-scale mathematics assessment. Frontiers in Psychology, 12, 1- 12.

Macdonald, P. (2002). Computer Adaptive Test for Measuring
Personality Factors Using Item Response Theory. (Unpublished doctoral dissertation), the University Of Western Ontario.

Magno, C. (2009). Demonstrating the Difference between Classical Test Theory and Item Response Theory Using Derived Test Data, the International Journal of Educational and Psychological Assessment, 1(1), 1-11.

Meriac, J., Poling, T, & Woehr, D. (2009) . Are there gender differences in work ethic? An examination of the measurement equivalence of the multidimensional work ethic profile. Personality and individual differences, 47(3), 209-213.

Mitana, J., Muwagga,A. & Ssempala,C. (2018). Assessment for successful ıntellıgence: a paradigm shift in classroom practice. International Journal of Educational Research Review, 4(1), 106-115.

Momani, R., Gharaibeh, S. (2017). Investigating the construct validity of Sternberg’s triarchic abilities test level-H (Arabic version). International Journal of Advanced and Applied Sciences, 4(11) 2017, 28-34.

Nguyen, P., Nguyen, C., Dang, C. & Duong, H. (2022). Proposing a framework to assess the intellectual development and competence of Vietnamese students based on Sternberg's Triarchic theory of Intelligence. Journal of Language and Linguistic Studies, 18(2), 685-700.

Qiu, C., Hatton, R., & Hou, M., (2020). Variations in Raven Progressive matrices scores among Chinese children and adolescents. Personality and Individual Differences, 164, 11-64.

Reise, S. & Waller, N.(2003(. How many IRT parameters does it take to model psychopathology items?. Psychological Methods, 8(2), 164- 184.

Robert, C., Lee, W. & Chan, K. (2006). An empirical analysis of measurement equivalence with the INDCOL measure of individualism and collectivism: implication for valid cross- cultural inference. Personnel Psychology, 59, 65-99.

Rutkowski, L., Gonzalez, E. & von Davier, M. (2014). Assessment design for international large-scale assessments. In L. Rutkwoski, M. von Davier, & D. Rutkowski (Eds.), Handbook of international large-scale assessment (pp. 75-95). Boca Raton, FL: CRC Press.

Sak, U. (2009). Test of the three-mathematical minds (M3) for the
identification of mathematically gifted students. Roeper Review, 31, 53-67.

Salami, M., Farzad, V. & Sarami, G. (2014).A Study of factor structure, reliability, and validity of the Successful Intelligence Scale. Quarterly of Educational Measurement, 4 (15), 1- 16.

Saw, K. & Han, B. (2021). Effectiveness of successful intelligence training program: A meta-analysis. PsyCh Journal, 10, 323–339.

Şekercioglu, G. (2018). Measurement invariance: Concept and implementation. International Online Journal of Education and Teaching (IOJET), 5(3). 609-634.

Steinmetz, H., Schmidt, P., Booh, A., Wieczorek, S. & Schwartz, S. (2009). Testing measurement invariance using multi-group CFA: Differences between educational groups in human values measurement. Quality& Quantity, 43(4), 599-616.

Stemler, S., Grigorenko, L., Jarvin, L. & Sternberg, R. (2006). Using the theory of successful intelligence as a basis for augmenting AP exams in Psychology and Statistics. Contemporary Educational Psychology, 31, 344-376.

Sternberg, R. (1993). Sternberg Triarchic Abilities Test (STAT). Office of Educational Research and Improvement, U. S. Department of Education: The Psychological Corporation.

Sternberg, R. (2000). The theory of successful intelligence. Gifted
Education International, 15 (1), 4- 21.

Sternberg, R. (2002). Beyond g: The theory of successful intelligence. In R. J. Sternberg, & E. L. Grigorenko (Eds.), the general factor of intelligence: How general is it? Mahwah, NJ: Lawrence Erlbaum Associates.

Sternberg, R. (2005). WICS: A model of positive educational leadership comprising wisdom, intelligence, and creativity synthesized. Educational Psychology Review, 17(3), 191- 262.

Sternberg, R., Castejon, J., Prieto, M., Hautamaki, J. & Grigorenko, E. (2001). Confirmatory Factor Analysis of the Sternberg Triarchic Abilities Test in Three International Samples An Empirical Test of the Triarchic Theory of Intelligence. European Journal of Psychological Assessment, 17(1), 1–16.

Sternberg, R., & Grigorinko, E. (2002). The Theory of Successful Intelligence as a Basis for Gifted Education. Gifted Child Quarterly, 46 (4), 265-277.

Sternberg, R., & Grigorenko, E. (2007). Teaching for successful
intelligence (2nd ed.). Thousand Oaks, California: Corwin press.

Stoneberg, B. (2004). A study of gender-based and ethnic-based differential item functioning (DIF) in the spring 2003 Idaho standards achievement tests applying test. (Unpublished doctoral dissertation), University of Maryland.

Thissen, D. (2001). IRTLRDIF v.2.0b: Software for the computation of statistics involved in item response theory likelihood-ratio tests for differential item functioning. Chapel Hill: L. L. Thurstone Psychometric Laboratory, University of North Carolina.

Walker, C. (2011). What’s the DIF? Why Differential Item Functioning
Analyses Are an Important Part of Instrument Development and Validation. Journal of Psycho educational Assessment, 29(4), 364–376.

Wang, J. & Wang, X. (2012). Structural equation modeling: Applications using Mplus. United Kingdom: John Wiley & Sons Ltd.

Wang, M. & Russell, S. (2005). Measurement equivalence of the job descriptive index across Chinese and American workers: Results from confirmatory factor analysis and item response theory. Educational and Psychological Measurement, 65 (4), 709-732.

Wicherts, J. ( 2007). Group differences in intelligence test performance. (Unpublished doctoral dissertation), University of Amsterdam.

Woods, C. (2009). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33(1), 42-57.

Zumbo, B. & Gelin, M. (2005). A Matter of Test Bias in Educational Policy Research: Bringing the Context into Picture by Investigating Sociological/ Community Moderated (or Mediated) Test and Item Bias. Journal of Educational Research & Policy Studies, 5(1), 1-23.