Journal of Cybersecurity and Information Management JCIM 2690-6775 2769-7851 10.54216/JCIM https://www.americaspg.com/journals/show/3127 2019 2019 Department of ECE, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Andhra Pradesh, India; Department of ECE, Sir C R Reddy College of Engineering, Eluru, India Nallamothu Nallamothu Department of ECE, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Andhra Pradesh, India Penke Satyanarayana Channel estimation poses critical challenges in millimeter-wave (mmWave) massive Multiple Input, Multiple Output (MIMO) communication models, particularly when dealing with a substantial number of antennas. Deep learning techniques have shown remarkable advancements in improving channel estimation accuracy and minimizing computational difficulty in 5G as well as the future generation of communications. The main intention of the suggested method is to use an optimal hybrid deep learning strategy to create a better channel estimation model. The proposed method, referred to as optimized D-LSTM, combines the power of a deep neural network (DNN) and long short-term memory (LSTM), and the optimization process involves the integration of the Reptile Search Algorithm (RSA) to enhance the performance of  deep learning model. The suggested hybrid deep learning method considers the correlation between the measurement matrix and the signal vectors that were received as input to predict the amplitude of the beam space channel. The newly proposed estimation model demonstrates remarkable superiority over traditional models in both Normalized Mean-Squared Error (NMSE) reduction and enhanced spectral efficiency. The spectral efficiency of the designed RSA-D-LSTM is 68.62%, 62.26%, 30.3%, and 19.77% higher than DOA, DHOA, HHO, and RSA. Therefore, the suggested system provides better channel estimation to improve its efficiency. 2025 2025 211 224 10.54216/JCIM.150116 https://www.americaspg.com/articleinfo/2/show/3127