Volume 7 • Issue 1 • PP: 20–29 • 2024
Exploring Predictive Models for Students’ Performance in Exams: A Comparative Analysis of Regression Algorithms
Open Access & Copyright
© 2024 The Author(s). Published by ASPG. This article is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
View PDF Abstract
Student-centered analysis of academic performance is also the most important aspect in improving education by being able to determine what measures work best, individualized learning approaches, and intervention programs. In this study, we performed a detailed analysis based on the “Students Performance in Exams” dataset and different regression methods to estimate students’ grades. We sought to assess the functioning of numerous metrics and determine an optimal model for this task. Our descriptive analysis identified meaningful trends within this dataset, as it includes central factors like gender, race/ethnic diversity-based status of a student, and parental education level based on which the children are catered to by informing them about important lunches and test preparation courses alongside scores in “Math,” “Readings,” “Writing,” etc. We used a wide range of regression models: XGBoost, CatBoost, GradientBoostingRegressor, etc. Metrics like Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Average Marginal Loss were used to assess each model rigorously. Importantly, the XGBoost model gave out an MSE value of 0.028, which was the best among all values obtained from various other models. The superiority of the XGBoost model is supported by the excellent performance that was reported across many metrics. This work can be important for informing educational practitioners and policymakers regarding the best possible accurate and realistic model that would predict the students’ outcome results. Educational data analytics incorporating the XGBoost model can be used for the customization of interventions and mapping resource allocation while promoting a results-oriented approach based on data in education. This study is a step towards the accumulation of knowledge on educational data analytics. It can serve as a background for further research aimed at improving predictive models regarding student performance.
Keywords
References
[1] E. B. Costa, B. Fonseca, M. A. Santana, F. F. de Araújo, and J. Rego, “Evaluating the effectiveness of educa-tional data mining techniques for early prediction of students’ academic failure in introductory programming courses,” Computers in Human Behavior, vol. 73, pp. 247–256, 2017.
[2] Q. Liu, Z. Huang, Y. Yin, E. Chen, H. Xiong, Y. Su, and G. Hu, “EKT: Exercise-aware knowledge tracing for student performance prediction,” IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 1, pp. 100–115, 2021.
[3] O. El Aissaoui, Y. El Alami El Madani, L. Oughdir, A. Dakkak, and Y. El Allioui, “A multiple linear regression-based approach to predict student perfor-mance,” in Advanced Intelligent Systems for Sustainable Development (AI2SD’2019), edited by M. Ezziyyani, Springer International Publishing, pp. 9–23, 2020.
[4] T. O. Hodson, “Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not,” Geoscientific Model Development, vol. 15, no. 14, pp. 5481–5487, 2022.
[5] H. Waheed, S.-U. Hassan, N. R. Aljohani, J. Hardman, S. Alelyani, and R. Nawaz, “Predicting academic per-formance of students from VLE big data using deep learning models,” Computers in Human Behavior, vol. 104, p. 106189, 2020.
[6] N. Tomasevic, N. Gvozdenovic, and S. Vranes, “An overview and comparison of supervised data mining techniques for student exam performance prediction,” Computers & Education, vol. 143, p. 103676, 2020.
[7] B. K. Francis and S. S. Babu, “Predicting academic performance of students using a hybrid data mining approach,” Journal of Medical Systems, vol. 43, no. 6, p. 162, 2019.
[8] X. Xu, J. Wang, H. Peng, and R. Wu, “Prediction of academic performance associated with internet usage behaviors using machine learning algorithms,” Comput-ers in Human Behavior, vol. 98, pp. 166–173, 2019.
[9] V. L. Miguéis, A. Freitas, P. J. V. Garcia, and A. Silva, “Early segmentation of students according to their aca-demic performance: A predictive modelling approach,” Decision Support Systems, vol. 115, pp. 36–51, 2018.
[10] S. A. Alwarthan, N. Aslam, and I. U. Khan, “Predicting student academic performance at higher education using data mining: A systematic review,” Applied Computa-tional Intelligence and Soft Computing, vol. 2022, no. 1, p. 8924028, 2022.
[11] S. Helal, J. Li, L. Liu, E. Ebrahimie, S. Dawson, D. J. Murray, and Q. Long, “Predicting academic performance by considering student heterogeneity,” Knowledge-Based Systems, vol. 161, pp. 134–146, 2018.
[12] F. Grivokostopoulou, I. Perikos, and I. Hatzilygeroudis, “An educational system for learning search algorithms and automatically assessing student performance,” Inter-national Journal of Artificial Intelligence in Education, vol. 27, no. 1, pp. 207–240, 2017.
[13] S. Alturki, L. Cohausz, and H. Stuckenschmidt, “Pre-dicting master’s students’ academic performance: An empirical study in Germany,” Smart Learning Environ-ments, vol. 9, p. 38, 2022.
[14] S. Stieglitz, M. Mirbabaie, B. Ross, and C. Neuberger, “Social media analytics–challenges in topic discovery, data collection, and data preparation,” International Journal of Information Management, vol. 39, pp. 156–168, 2018.
[15] F. Raposo, R. Borja, and C. Ibelli-Bianco, “Predictive regression models for biochemical methane potential tests of biomass samples: Pitfalls and challenges of laboratory measurements,” Renewable and Sustainable Energy Reviews, vol. 127, p. 109890, 2020.
[16] Y. Qiu, J. Zhou, M. Khandelwal, H. Yang, P. Yang, and C. Li, “Performance evaluation of hybrid WOA-XGBoost, GWO-XGBoost and BO-XGBoost models to predict blast-induced ground vibration,” Engineering with Computers, vol. 38, no. 5, pp. 4145–4162, 2022.
[17] G. Huang, L. Wu, X. Ma, W. Zhang, J. Fan, X. Yu, W. Zeng, and H. Zhou, “Evaluation of CatBoost method for prediction of reference evapotranspiration in humid regions,” Journal of Hydrology, vol. 574, pp. 1029–1041, 2019.
[18] A. Keprate and R. M. C. Ratnayake, “Using gradient boosting regressor to predict stress intensity factor of a crack propagating in small bore piping,” in 2017 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), pp. 1331–1336, 2017.
[19] M. Xie and Z. Tian, “A review on pipeline integrity man-agement utilizing in-line inspection data,” Engineering Failure Analysis, vol. 92, pp. 222–239, 2018.
[20] Y. Song, J. Liang, J. Lu, and X. Zhao, “An efficient instance selection algorithm for k nearest neighbor re-gression,” Neurocomputing, vol. 251, pp. 26–34, 2017.
[21] M. Reza and M. A. Haque, “Photometric redshift estima-tion using ExtraTreesRegressor: Galaxies and quasars from low to very high redshifts,” Astrophysics and Space Science, vol. 365, no. 3, p. 50, 2020.
[22] A. F. Schmidt and C. Finan, “Linear regression and the normality assumption,” Journal of Clinical Epidemiol-ogy, vol. 98, pp. 146–151, 2018.
[23] F. Zhang and L. J. O’Donnell, “Chapter 7–support vector regression,” in Machine Learning, edited by A. Mechelli and S. Vieira, Academic Press, pp. 123–140, 2020.
[24] B. T. Pham, M. D. Nguyen, K.-T. T. Bui, I. Prakash, K. Chapi, and D. T. Bui, “A novel artificial intelligence approach based on multi-layer perceptron neural net-work and biogeography-based optimization for predict-ing coefficient of consolidation of soil,” CATENA, vol. 173, pp. 302–311, 2019.
[25] R. Couronné, P. Probst, and A.-L. Boulesteix, “Random forest versus logistic regression: A large-scale bench-mark experiment,” BMC Bioinformatics, vol. 19, no. 1, p. 270, 2018.
[26] E. Pekel, “Estimation of soil moisture using decision tree regression,” Theoretical and Applied Climatology, vol. 139, no. 3, pp. 1111–1119, 2020.
[27] V. Plevris, G. Solorzano, N. P. Bakas, and M. E. A. Ben Seghier, “Investigation of performance metrics in regression analysis and machine learning-based pre-diction models,” in Proceedings of the 8th European Congress on Computational Methods in Applied Sci-ences and Engineering (ECCOMAS 2022), pp. 1–25, 2022.
Cite This Article
Choose your preferred format
Publisher's Note
The statements, opinions, and data presented in this article are solely those of the author(s) and do not necessarily represent those of ASPG, the journal, or its editors. ASPG and the editors disclaim responsibility for any harm arising from the use of any ideas, methods, instructions, or products described in this article, to the fullest extent permitted by applicable law.