Journal of Intelligent Systems and Internet of Things JISIoT 2690-6791 2769-786X 10.54216/JISIoT https://www.americaspg.com/journals/show/2281 2019 2019 Department of Electrical Engineering, Canadian University Dubai, Dubai, UAE Firuz Kamalov Department of Computer Science, Canadian University Dubai, Dubai, UAE Said Elnaffar School of Information Systems, Vellore Institute of Technology, India Aswani Cherukuri School of Computer Science and Engineering, Vellore Institute of Technology, India Annapurna Jonnalagadda Feature selection is an important preprocessing step in many data science and machine learning applications. Although there exist several sophisticated feature selection algorithms, their benefits are sometimes overshadowed by their complexity and slow execution. Therefore, in many cases, a more simple algorithm may be better suited. In this paper, we demonstrate that a rudimentary forward selection algorithm can achieve optimal performance with a low time complexity. Our study is based on an extensive empirical evaluation of the forward feature selection algorithm in the context of linear regression. Concretely, we compare the forward selection algorithm against the gold standard exhaustive search algorithm based on several datasets. The results show that the forward selection algorithm achieves high performance with relatively fast execution. Given the simplicity, accuracy, and speed of the forward feature selection algorithm, we recommend it as a primary feature selection method for most regression applications. Our results are particularly pertinent in the case of big data and real-time analysis. 2024 2024 44 54 10.54216/JISIoT.110105 https://www.americaspg.com/articleinfo/18/show/2281