International Journal of Advances in Applied Computational Intelligence IJAACI 2833-5600 10.54216/IJAACI https://www.americaspg.com/journals/show/2438 2022 2022 Tashkent State Universtiy of Economics, Tashkent, Uzbekistan Ahmed Aziz Tashkent State Universtiy of Economics, Tashkent, Uzbekistan Sanjar Mirzaliev Tashkent State Universtiy of Economics, Tashkent, Uzbekistan Yuldashev Maqsudjon This research is about the increasing cybersecurity challenges posed by modern malware threats and argues for an improved approach through optimized machine learning algorithms. We apply a Tree-structured Parzen Estimator (TPE) for hyperparameter tuning, focusing on the optimization of tree-based models such as Random Forest and Gradient Boosting. Our methodology includes careful correlation analysis, variable distribution examination, and feature importance assessment to make our models more robust and transparent. We present comprehensive visualizations that demonstrate the results of our optimized approach, which show improved accuracy, precision, and recall in malware detection. Our findings highlight the significance of feature engineering and model tuning, revealing subtle patterns indicative of malicious behavior. The findings indicate that our model provides a method that not only improves detection capabilities but also emphasizes the need for continuous improvement and innovation in addressing the ever-changing nature of malware threats. 2023 2023 26 32 10.54216/IJAACI.040203 https://www.americaspg.com/articleinfo/31/show/2438