Employing Learning Analytics in Online Learning Environments between 2018-2022: A Systematic Review

Document Type : Original Article

Authors

1 1PhD researcher in the Department of Educational Technologies, King Abdulaziz University

2 2Professor of Educational Communication and Educational Technology at King Abdulaziz University

Abstract

The aim of the current study was to explore the practical use of learning analytics in online learning environments from 2018 to 2022. This study employed a systematic review approach to scrutinize previous studies that incorporated learning analytics in online learning environments within the specified time frame. A total of 86 studies were selected for review, which included four in Arabic and 82 in English. The study found that Learning Management Systems (LMS) were the most prevalent platforms for utilizing learning analytics in both Arabic and English studies. Performance data emerged as the most commonly used type of data in Arabic studies. In English-language studies learner interaction data had the highest percentage. The objectives of using learning analytics showed variation in Arabic studies, while monitoring and analysis were the primary objectives in English studies. Data mining was the preferred methods of data analysis in Arabic studies, with data mining being the most utilized method in English studies. The study revealed that both researcher and learners were equally represented as stakeholders in Arabic studies. However, in English studies, teachers were more prominently represented. Based on the findings of the study, several recommendations were proposed. These include the design of intelligent learning environments that leverage learning analytics to personalize and adapt the learning process. It is also recommended to adopt a modern approach that places the learner at the core of the educational process by employing learning analytics (dashboards) in personalized learning settings.

Keywords


Al-Asr, S. A. A. (2021). Using Learning Analytics Technology to Predict the Effectiveness of Web-Based Electronic Discussions and Its Impact on Improving Overall Performance, Developing Metacognitive Skills, and Satisfaction with Learning for Postgraduate Students. Educational Technology - Egyptian Association for Educational Technology, 31(6), 184-193.
 
 
Al-Rehaili, T. B. A. (2022). The Effectiveness of Designing a Virtual Learning Environment Based on Learning Analytics in Developing Self-Regulated Learning Behaviors and Its Relationship to Academic Performance. Journal of Hafar Al-Batin University for Educational and Psychological Sciences, (4), 11-58.
Abdel Rahman, I. A. S. M. A., & Al-Mahdi, M. M. G. (2019). Levels of Support in an Intelligent Learning Environment Based on Educational Analytics and Its Impact on Developing Scientific Research Proposal Writing Skills and Satisfaction with Learning among Students. Educational Technology - Egyptian Association for Educational Technology, 29(6), 4-113.
Al-Alwan, J. (2020). Contemporary Administrative Trends in Human Resource Development: A Methodological Review of Relevant Literature. Journal of Humanities and Social Sciences - Imam Muhammad ibn Saud Islamic University, (46), 67-119.
Mohamed, I. Z. M. (2020). Developing an Adaptive Web Environment According to the Herrmann Model and Learning Analytics: Its Impact on Developing Augmented Reality Application Production Skills and Deep Learning Among Educational Technology Students at the College of Quality Education. Educational Technology - Arab Association for Educational Technology, (43), 1-144.
Mousa, M. A. F. (2020). Monitoring the Reality of Research on the Development of Intelligent Learning Environments Enhanced with Learning Analytics and Recommendations for Future Research. Educational Technology - Egyptian Association for Educational Technology, 30(8), 3-20.                        
ثالثًا: المراجع باللغة الإنجليزية
‏Akçapınar, G., Altun, A., & Aşkar, P. (2019). Using learning analytics to develop early-warning system for at-risk students. International Journal of Educational Technology in Higher Education, 16(1), 1-20.‏
 
AlJarrah, A., Thomas, M. K., & Shehab, M. (2018). Investigating temporal access in a flipped classroom: procrastination persists. International Journal of Educational Technology in Higher Education, 15(1), 1-18.‏
Almeda, M. V., Zuech, J., Utz, C., Higgins, G., Reynolds, R., & Baker, R. S. (2018). Comparing the Factors That Predict Completion and Grades Among For-Credit and Open/MOOC Students in Online Learning. Online Learning, 22(1), 1-18
Alwafi, E. M. (2022). Designing an Online Discussion Strategy with Learning Analytics Feedback on the Level of Cognitive Presence and Student Interaction in an Online Learning Community. Online Learning, 26(1), 80-92.‏
Aristizábal, J. A. (2018). Using learning analytics to improve students’ reading skills: A Case study in an American international school with EAL students. Gist: Education and Learning Research Journal, (17), 193-214.‏
Bayazit, A., Apaydin, N., & Gonullu, I. (2022). Predicting at-risk students in an online flipped anatomy course using learning analytics. Education Sciences, 12(9), 581.‏
Bernard, R. M., Borokhovski, E., & Tamim, R. M. (2014). Detecting bias in meta-analyses of distance education research: big pictures we can rely on. Distance Education35(3), 271-293.‏
Boroujeni, M. S., & Dillenbourg, P. (2019). Discovery and temporal analysis of MOOC study patterns. Journal of Learning Analytics, 6(1), 16-33.‏
Brennan, A., Sharma, A., & Munguia, P. (2019). Diversity of online behaviours associated with physical attendance in lectures. Journal of Learning Analytics, 6(1), 34-53.‏
Çakiroğlu, Ü., & Kahyar, S. (2022). Modelling online community constructs through interaction data: A learning analytics based Approach. Education and Information Technologies, 27(6), 8311-8328.‏
Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2013). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5-6), 318-331.
Chen, W. (2019). Knowledge-aware learning analytics for smart learning .Procedia Computer Science, 159, 1957–1965.
Chen, L., Yoshimatsu, N., Goda, Y., Okubo, F., Taniguchi, Y., Oi, M., ... & Yamada, M. (2019). Direction of collaborative problem solving-based STEM learning by learning analytics approach. Research and Practice in Technology Enhanced Learning, 14, 1-28.‏
Chen, F., & Cui, Y. (2020). Utilizing Student Time Series Behaviour in Learning Management Systems for Early Prediction of Course Performance. Journal of Learning Analytics, 7(2), 1-17.‏
Chen, Z., Xu, M., Garrido, G., & Guthrie, M. W. (2020). Relationship between students’ online learning behavior and course performance: What contextual information matters?. Physical Review Physics Education Research, 16(1), 010138.‏
Clark, R. C., & Mayer, R. E. (2023). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. john Wiley & sons.‏
Conijn, R., Van den Beemt, A., & Cuijpers, P. (2018). Predicting student performance in a blended MOOC. Journal of Computer Assisted Learning, 34(5), 615-628.‏
Davies, R., Allen, G., Albrecht, C., Bakir, N., & Ball, N. (2021). Using educational data mining to identify and analyze student learning strategies in an online flipped classroom. Education Sciences, 11(11), 668.
Dickler, R. (2021). Using Innovative Methods to Explore the Potential of an Alerting Dashboard for Science Inquiry. Journal of learning analytics, 8(2).‏
Dooley, L., & Makasis, N. (2020). Understanding student behavior in a flipped classroom: Interpreting learning analytics data in the veterinary pre-clinical sciences. Education sciences, 10(10), 260.‏
Eryilmaz, E., Thoms, B., Ahmed, Z., & Lee, K. H. (2020). Effects of recommendations on message quality and community formation in online conversations. Education and information technologies, 26, 49-68.‏
Ferri, F., Grifoni, P., & Guzzo, T. (2020). Online learning and emergency remote teaching: Opportunities and challenges in emergency situations. Societies, 10(4), 86.‏
Gallego-Romero, J. M., Alario-Hoyos, C., Estévez-Ayres, I., & Delgado Kloos, C. (2020). Analyzing learners’ engagement and behavior in MOOCs on programming with the Codeboard IDE. Educational Technology Research and Development, 68, 2505-2528.
‏ Garcia, V., Conesa, J., & Perez-Navarro, A. (2022). Videos with hands: An analysis of usage and interactions of undergraduate science students for acquiring physics knowledge. Journal of Science Education and Technology, 31(4), 442-460.‏
Gardner, C., Jones, A., & Jefferis, H. (2020). Analytics for tracking student engagement. Journal of Interactive Media in Education, 2020(1).‏
Gelan, A., Fastré, G., Verjans, M., Martin, N., Janssenswillen, G., Creemers, M., ... & Thomas, M. (2018). Affordances and limitations of learning analytics for computer-assisted language learning: A case study of the VITAL project. Computer Assisted Language Learning, 31(3), 294-319.‏
Goundar, S., Deb, A., Lal, G., & Naseem, M. (2022). Using online student interactions to predict performance in a first-year computing science course. Technology, Pedagogy and Education, 31(4), 451-469.‏
Han, F., Pardo, A., & Ellis, R. A. (2020). Students' self‐report and observed learning orientations in blended university course design: How are they related to each other and to academic performance?. Journal of Computer Assisted Learning, 36(6), 969-980.‏
Han, F., & Ellis, R. A. (2022). The relations between self-reported perceptions of learning environment, observational learning strategies, and academic outcome. Journal of Computing in Higher Education, 35(1), 111-125.‏
Harindranathan, P., & Folkestad, J. (2019). Learning Analytics to Inform the Learning Design: Supporting Instructors' Inquiry into Student Learning in Unsupervised Technology-Enhanced Platforms. Online Learning, 23(3), 34-55.‏
Hernández-Lara, A. B., Perera-Lluna, A., & Serradell-López, E. (2021). Game learning analytics of instant messaging and online discussion forums in higher education. Education+ Training, 63(9), 1288-1308.‏
Holmes, W., Nguyen, Q., Zhang, J., Mavrikis, M., & Rienties, B. (2019). Learning analytics for learning design in online distance learning. Distance Education, 40(3), 309-329.‏
Howard, E., Meehan, M., & Parnell, A. (2018). Contrasting prediction methods for early warning systems at undergraduate level. The Internet and Higher Education, 37, 66-75.‏
Huang, C., Han, Z., Li, M., Wang, X., & Zhao, W. (2021). Sentiment evolution with interaction levels in blended learning environments: Using learning analytics and epistemic network analysis. Australasian Journal of Educational Technology, 37(2), 81-95.‏
Hsu, T. C., Abelson, H., Patton, E., Chen, S. C., & Chang, H. N. (2021). Self-efficacy and behavior patterns of learners using a real-time collaboration system developed for group programming. International Journal of Computer-Supported Collaborative Learning, 1-24.
Husni, N. H. A., Jumaat, N. F., & Tasir, Z. (2022). Investigating Student's Cognitive Engagement, Motivation and Cognitive Retention in Learning Management System. International Journal of Emerging Technologies in Learning, 17(9).‏
Ifenthaler, D., Gibson, D., & Dobozy, E. (2018). Informing learning design through analytics: Applying network graph analysis. Australasian Journal of Educational Technology, 34(2).‏
Ifenthaler, D., Schumacher, C., & Kuzilek, J. (2022). Investigating students' use of self‐assessments in higher education using learning analytics. Journal of Computer Assisted Learning, 39(1), 255-268.‏
Jaramillo-Morillo, D., Ruipérez-Valiente, J., Sarasty, M. F., & Ramírez-Gonzalez, G. (2020). Identifying and characterizing students suspected of academic dishonesty in SPOCs for credit through learning analytics. International Journal of Educational Technology in Higher Education, 17(1), 1-18.
Kaliisa, R., Kluge, A., & Mørch, A. I. (2020). Combining Checkpoint and Process Learning Analytics to Support Learning Design Decisions in Blended Learning Environments. Journal of Learning Analytics, 7(3), 33-47.‏
‏Kew, S. N., & Tasir, Z. (2021). Analysing Students' Cognitive Engagement in E-Learning Discussion Forums through Content Analysis. Knowledge Management & E-Learning, 13(1), 39-57.‏
Kew, S. N., & Tasir, Z. (2021). Learning analytics in online learning environment: a systematic review on the focuses and the types of student-related analytics data. Technology, Knowledge and Learning, 1-23.‏
Kinshuk, Chen, N. S., Cheng, I. L., & Chew, S. W. (2016). Evolution is not enough: revolutionizing current learning environments to smart learning environments. International Journal of Artificial Intelligence in Education, 26, 561–581.
‏Knoop-van Campen, C. A., Wise, A., & Molenaar, I. (2021). The equalizing effect of teacher dashboards on feedback in K-12 classrooms. Interactive Learning Environments, 31(6), 3447-3463.‏
Koh, E., Jonathan, C., & Tan, J. P. L. (2019). Exploring conditions for enhancing critical thinking in networked learning: Findings from a secondary school learning analytics environment. Education Sciences, 9(4), 287.‏
Krieter, P. (2022). Are You Still There? An Exploratory Case Study on Estimating Students’ LMS Online Time by Combining Log Files and Screen Recordings. IEEE Transactions on Learning Technologies, 15(1), 55-63.
Lahza, H., Khosravi, H., & Demartini, G. (2022). Analytics of learning tactics and strategies in an online learnersourcing environment. Journal of Computer Assisted Learning, 39(1), 94-112.‏
Lawrence, J., Brown, A., Redmond, P., Maloney, S., Basson, M., Galligan, L., & Turner, J. (2021). Does course specific nudging enhance student engagement, experience and success?: A data-driven longitudinal tale. Student Success, 12(2), 28-37.‏
Lewis, S., Heath, G., Lim, L., & Roberts, R. (2021). " I'm not a number, I'm someone to them": Supporting commencing university students' through technology-mediated personalised communication. Student Success, 12(1), 24-34.‏
Li, S., Du, J., & Sun, J. (2022). Unfolding the learning behaviour patterns of MOOC learners with different levels of achievement. International Journal of Educational Technology in Higher Education, 19(1), 22.‏
Lim, L. A., Gentili, S., Pardo, A., Kovanović, V., Whitelock-Wainwright, A., Gašević, D., & Dawson, S. (2021). What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. Learning and Instruction, 72, 101202.‏
Romera, D. D. M., de la Serna, M. C., & de Montiano, G. P. (2021). The Influence of Students' Gender on the Use of Virtual Campuses. A Case Study.Pixel-Bit. Revista de Medios y Educación60, 169-210.‏.‏
Matcha, W., Gasevic, D., Uzir, N. A. A., Jovanovic, J., Pardo, A., Lim, L., ... & Tsai, Y. S. (2020). Analytics of Learning Strategies: Role of Course Design and Delivery Modality. Journal of Learning Analytics, 7(2), 45-71.‏
de Melo-Minardi, R. C., de Melo, E. C., & Bastos, L. L. (2022, May). OnlineBioinfo: Leveraging the Teaching of Programming Skills to Life Science Students Through Learning Analytics. In Frontiers in Education (Vol. 7, p. 727019).
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Altman, D., Antes, G., Atkins, D., Barbour, V., Barrowman, N., & Berlin, J. A. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. J. Chin. Integr. Med, 7, 889–896.
Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). e-Learning, online learning, and distance learning environments: Are they the same?. The Internet and higher education, 14(2), 129-135.
Moore, R. L., Oliver, K. M., & Wang, C. (2019). Setting the pace: Examining cognitive processing in MOOC discussion forums with automatic text analysis. Interactive Learning Environments, 27(5-6), 655-669.‏
Musabirov, I., Pozdniakov, S., & Tenisheva, K. (2019). Predictors of academic achievement in blended learning: The case of data science minor. International Journal of Emerging Technologies in Learning (Online), 14(5), 64.‏
Muslim, A., Chatti, M. A., & Guesmi, M. (2020). Open learning analytics: a systematic literature review and future perspectives. Artificial Intelligence Supported Educational Technologies, 3-29.‏
Nguyen, Q., Huptych, M., & Rienties, B. (2018). Using temporal analytics to detect inconsistencies between learning design and student behaviours. Journal of Learning Analytics, 5(3), 120-135.‏
Nguyen, Q., Rienties, B., & Richardson, J. T. (2019). Learning analytics to uncover inequality in behavioural engagement and academic attainment in a distance learning setting. Assessment & Evaluation in Higher Education, 45(4), 594-606.‏
Oliveira, E., de Barba, P. G., & Corrin, L. (2021). Enabling adaptive, personalised and context-aware interaction in a smart learning environment: Piloting the iCollab system. Australasian Journal of Educational Technology, 37(2), 1-23.‏
Ouyang, F., & Dai, X. (2022). Using a three-layered social-cognitive network analysis framework for understanding online collaborative discussions. Australasian Journal of Educational Technology, 38(1), 164-181.‏
Padilla Rodriguez, B. C., Armellini, A., & Rodriguez Nieto, M. C. (2020). Learner engagement, retention and success: why size matters in massive open online courses (MOOCs). Open Learning: The Journal of Open, Distance and e-Learning, 35(1), 46-62.‏
Papamitsiou, Z., & Economides, A. A. (2021). The impact of on‐demand metacognitive help on effortful behaviour: A longitudinal study using task‐related visual analytics. Journal of Computer Assisted Learning, 37(1), 109-126.‏
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128-138.‏
Pérez Sánchez, C. J., Calle-Alonso, F., & Vega-Rodriguez, M. A. (2022). Learning analytics to predict students’ performance: A case study of a neurodidactics-based collaborative learning platform. Education and Information Technologies, 27(9), 12913-12938.‏
Prat, A., & Code, W. J. (2020). WeBWorK log files as a rich source of data on student homework behaviours. International Journal of Mathematical Education in Science and Technology, 52(10), 1540-1556.‏
Rienties, B., Lewis, T., McFarlane, R., Nguyen, Q., & Toetenel, L. (2018). Analytics in online and offline language learning environments: the role of learning design to understand student online engagement. Computer Assisted Language Learning, 31(3), 273-293.‏
Safsouf, Y., Mansouri, K., & Poirier, F. (2021). TaBAT: design and experimentation of a learning analysis dashboard for teachers and learners. Journal of Information Technology Education, 20, 331-350.‏
Saqr, M., Fors, U., & Tedre, M. (2018). How the study of online collaborative learning can guide teachers and predict students’ performance in a medical course. BMC medical education, 18, 1-14.‏
Saqr, M., & Alamro, A. (2019). The role of social network analysis as a learning analytics tool in online problem based learning. BMC medical education, 19, 1-11.‏
Saqr, M., Nouri, J., Vartiainen, H., & Malmberg, J. (2020). What makes an online problem-based group successful? A learning analytics study using social network analysis. BMC medical education, 20, 1-11.‏
Saqr, M., Peeters, W., & Viberg, O. (2021). The relational, co-temporal, contemporaneous, and longitudinal dynamics of self-regulation for academic writing. Research and Practice in Technology Enhanced Learning16, 1-22.‏
Saqr, M., Tuominen, V., Valtonen, T., Sointu, E., Väisänen, S., & Hirsto, L. (2022, May). Teachers’ Learning Profiles in Learning Programming: The Big Picture!. In Frontiers in Education (Vol. 7, p. 840178).‏
Shimada, A., Konomi, S. I., & Ogata, H. (2018). Real-time learning analytics system for improvement of on-site lectures. Interactive Technology and Smart Education, 15(4), 314-331.‏
Siddaway, A. (2014). What is a systematic literature review and how do I do one. University of Stirling, 1(1), 1-13.‏
Sidi, Y., Blau, I., & Shamir‐Inbal, T. (2022). Mapping active and collaborative learning in higher education through annotations in hyper‐video by learning analytics. Journal of Computer Assisted Learning, 38(6), 1752-1764.‏
Siemens, G., & Gasevic, D. (2012). Guest editorial-learning and knowledge analytics. Journal of Educational Technology & Society, 15(3), 1-2.
Singh, V., & Thurman, A. (2019). How many ways can we define online learning? A systematic literature review of definitions of online learning (1988-2018). American Journal of Distance Education33(4), 289-306.‏
Smith, P. (2019). Engaging online students through peer-comparison progress dashboards. Journal of Applied Research in Higher Education, 12(1), 38-56.‏
Sokout, H., Usagawa, T., & Mukhtar, S. (2020). Learning Analytics: Analyzing Various Aspects of Learners’ Performance in Blended Courses. The Case of Kabul Polytechnic University, Afghanistan. International Journal of Emerging Technologies in Learning (iJET), 15(12), 168-190.‏
Summers, R. J., Higson, H. E., & Moores, E. (2020). Measures of engagement in the first three weeks of higher education predict subsequent activity and attainment in first year undergraduate students: a UK case study. Assessment & Evaluation in Higher Education, 46(5), 821-836.‏
Summers, R., Higson, H., & Moores, E. (2021). The impact of disadvantage on higher education engagement during different delivery modes: a pre-versus peri-pandemic comparison of learning analytics data. Assessment & Evaluation in Higher Education, 48(1), 56-66.‏
Ulfa, S., & Fatawi, I. (2021). Predicting factors that influence students’ learning outcomes using learning analytics in online learning environment. International Journal of Emerging Technologies in Learning (iJET), 16(1), 4-17.‏
Valle, N., Antonenko, P., Valle, D., Sommer, M., Huggins-Manley, A. C., Dawson, K., ... & Baiser, B. (2021). Predict or describe? How learning analytics dashboard design influences motivation and statistics anxiety in an online statistics course. Educational Technology Research and Development, 69(3), 1405-1431.‏
Van Laer, S., & Elen, J. (2020). Adults’ self-regulatory behaviour profiles in blended learning environments and their implications for design. Technology, Knowledge and Learning, 25(3), 509-539.‏
Veerasamy, A. K., Laakso, M. J., & D’Souza, D. (2022). Formative assessment tasks as indicators of student engagement for predicting at-risk students in programming courses. Informatics in Education, 21(2), 375-393.‏
Wakelam, E., Jefferies, A., Davey, N., & Sun, Y. (2020). The potential for student performance prediction in small cohorts with minimal available attributes. British Journal of Educational Technology, 51(2), 347-370.‏
Walsh, J. N., & Rísquez, A. (2020). Using cluster analysis to explore the engagement with a flipped classroom of native and non-native English-speaking management students. The International Journal of Management Education, 18(2), 100381.‏
Walsh, J. N., O'Brien, M. P., & Costin, Y. (2021). Investigating student engagement with intentional content: An exploratory study of instructional videos. The International Journal of Management Education, 19(2), 100505.‏
Wells, J., Spence, A., & McKenzie, S. (2021). Student Participation in Computing Studies to Understand Engagement and Grade Outcome. Journal of Information Technology Education, 20.‏
Wong, J., Khalil, M., Baars, M., de Koning, B. B., & Paas, F. (2019). Exploring sequences of learner activities in relation to self-regulated learning in a massive open online course. Computers & Education, 140, 103595.‏
Wong, B. T. M., Li, K. C., & Cheung, S. K. (2022). An analysis of learning analytics in personalised learning. Journal of Computing in Higher Education, 1-20.
Yang, Y., Zhu, G., Sun, D., & Chan, C. K. (2022). Collaborative analytics-supported reflective assessment for scaffolding pre-service teachers’ collaborative inquiry and knowledge building. International Journal of Computer-Supported Collaborative Learning, 17(2), 249-292.‏
Yürüm, O. R., Taşkaya-Temizel, T., & Yıldırım, S. (2022). The use of video clickstream data to predict university students’ test performance: A comprehensive educational data mining approach. Education and Information Technologies28(5), 5209-5240.‏
Zhang, J., Huang, Y., & Gao, M. (2022). Video Features, Engagement, and Patterns of Collective Attention Allocation: An Open Flow Network Perspective. Journal of Learning Analytics, 9(1), 32-52.‏