aspg-18-2281-1792025628 20261015005348 American Scientific Publishing Group admin@americaspg.com American Scientific Publishing Group Journal of Intelligent Systems and Internet of Things JISIoT 2769-786X 2690-6791 2024 11 1 Firuz Kamalov Department of Electrical Engineering, Canadian University Dubai, Dubai, UAE Said Elnaffar Department of Computer Science, Canadian University Dubai, Dubai, UAE Aswani Cherukuri School of Information Systems, Vellore Institute of Technology, India Annapurna Jonnalagadda School of Computer Science and Engineering, Vellore Institute of Technology, India Feature selection is an important preprocessing step in many data science and machine learning applications. Although there exist several sophisticated feature selection algorithms, their benefits are sometimes overshadowed by their complexity and slow execution. Therefore, in many cases, a more simple algorithm may be better suited. In this paper, we demonstrate that a rudimentary forward selection algorithm can achieve optimal performance with a low time complexity. Our study is based on an extensive empirical evaluation of the forward feature selection algorithm in the context of linear regression. Concretely, we compare the forward selection algorithm against the gold standard exhaustive search algorithm based on several datasets. The results show that the forward selection algorithm achieves high performance with relatively fast execution. Given the simplicity, accuracy, and speed of the forward feature selection algorithm, we recommend it as a primary feature selection method for most regression applications. Our results are particularly pertinent in the case of big data and real-time analysis. 2024 44 54 2281 https://creativecommons.org/licenses/by/4.0/ 10.54216/JISIoT.110105 https://americaspg.com/journal/18/article/2281 https://americaspg.com/storage/91702156648.pdf