2690-6791ISSN (Online)
2769-786XISSN (Print)
Volume 17 , Issue 2 , PP: 50-63, 2025 | Cite this article as | XML | Html | PDF | Full Length Article
Salam Al-augby 1 * , Zahraa Ch. Oleiwi 2 , Hasanen Alyasiri 3 , Fahad Ghalib Abdulkadhim 4
Doi: https://doi.org/10.54216/JISIoT.170205
One of the major concerns when transitioning emails is the potential influx of unsolicited and unwanted spam emails. These unwanted emails can clog inboxes, causing recipients to overlook important messages and opportunities. To ensure security and avoid the destructive and dangerous effect of these spam emails, machine learning and deep learning methods have been conducted to design spam detection models. In this work, a combination of embedding models and multi-layer artificial neural networks as deep learning classification models is utilized in order to introduce an approach to spam detection. The proposed classifier leverages the Bidirectional Encoder Representations from Transformers (BERT) model for word embedding, applied to the Enron-Spam dataset, offering a noteworthy technique for considerable spam detection. Experimental results demonstrate that the proposed spam detection model achieved a 99% recall rate for detecting spam emails. Notably, this model is a step forward in generality and improving the efficiency of spam detection. It presents a good attempt at presenting a solution for detecting spam emails and fake text within communication environments.
Spam email , BERT model , Embedding models , Deep learning
[1] N. A. Saeejil et al., "Exploring Algorithmic Paradigms in Message Classification: Insights from the Enron E-mail Dataset," in International Conference on Advances in Information Communication Technology & Computing, 2024, pp. 27-40.
[2] C. N. Mohammed and A. M. Ahmed, "A semantic-based model with a hybrid feature engineering process for accurate spam detection," Journal of Electrical Systems and Information Technology, vol. 11, no. 1, p. 26, 2024.
[3] A. Hussain, A. Khatoon, A. Aslam, and M. A. Khosa, "A Comparative Performance Analysis of Machine Learning Models for Intrusion Detection Classification," Journal of Cybersecurity, vol. 6, 2024.
[4] D. Gupta, S. Dubey, and M. Mallik, "Foretelling the compressive strength of concrete using twin support vector regression," International Journal of Information Technology, pp. 1-18, 2024.
[5] S. Balamurugan, E. Gurumoorthi, P. Devi, and R. Maruthamuthu, "Impact of nutrients in food quality and safety by machine learning classifier using internet of things," International Journal of Information Technology, pp. 1-10, 2024.
[6] R. Cho, M. Zaman, K. T. Cho, and J. Hwang, "Investigating brain activity patterns during learning tasks through EEG and machine learning analysis," International Journal of Information Technology, pp. 1-8, 2024.
[7] S. Tared, L. Khaouane, S. Hanini, A. Khaouane, and M. Roubehie Fissa, "Enhancing lung cancer prediction through crow search, artificial bee colony algorithms, and support vector machine," International Journal of Information Technology, pp. 1-11, 2024.
[8] V. Chirchi, E. Chirchi, and K. E. Chirchi, "Pattern matching for the iris biometric recognition system uses KNN and fuzzy logic classifier techniques," International Journal of Information Technology, vol. 16, no. 5, pp. 2937-2944, 2024.
[9] D. Jayabalan and S. Elango, "ICE-VDOP: an integrated clustering and ensemble machine learning methods for an enhanced vector-borne disease outbreak prediction using climatic variables," International Journal of Information Technology, vol. 16, no. 4, pp. 2077-2088, 2024.
[10] S. Mondal, S. Ghosh, and A. Nag, "Brain stroke prediction model based on boosting and stacking ensemble approach," International Journal of Information Technology, vol. 16, no. 1, pp. 437-446, 2024.
[11] A. Qazi, N. Hasan, R. Mao, M. E. M. Abo, S. K. Dey, and G. Hardaker, "Machine Learning-Based Opinion Spam Detection: A Systematic Literature Review," IEEE Access, 2024.
[12] S. Xiao, R. Hao, G. Cheng, X. Xu, and T. Li, "EC-BERT: A BERT Language Model with Error Correction for Mandarin Chinese Speech Recognition," Journal of Shanghai Jiaotong University (Science), pp. 1-7, 2024.
[13] A. M. M. Al Zoubi, "Spam Reviews Detection Models in Multilingual Contexts applying Sentiment Analysis, Metaheuristics, and Advanced Word Embedding," 2024.
[14] A. Singla, "Roberta and BERT: Revolutionizing Mental Healthcare through Natural Language," Shodh Sagar Journal of Artificial Intelligence and Machine Learning, vol. 1, no. 1, pp. 10-27, 2024.
[15] M. A. Uddin, M. N. Islam, L. Maglaras, H. Janicke, and I. H. Sarker, "ExplainableDetector: Exploring Transformer-based Language Modeling Approach for SMS Spam Detection with Explainability Analysis," arXiv preprint arXiv:2405.08026, 2024.
[16] K. S. Reddy and E. S. Reddy, "An Efficient Methodology to Detect Spam In Social Networking Sites," International Journal of Computer Science and Information Security (IJCSIS), vol. 15, no. 7, 2017.
[17] N. Ali, A. Fatima, H. Shahzadi, A. Ullah, and K. Polat, "Feature extraction aligned email classification based on imperative sentence selection through deep learning," Journal of Artificial Intelligence and Systems, vol. 3, no. 1, pp. 93-114, 2021.
[18] V. S. Tida and S. Hsu, "Universal spam detection using transfer learning of BERT model," arXiv preprint arXiv:2202.03480, 2022.
[19] O. Agboola, "Spam Detection Using Machine Learning and Deep Learning," Louisiana State University and Agricultural & Mechanical College, 2022.
[20] Y. Guo, Z. Mustafaoglu, and D. Koundal, "Spam detection using bidirectional transformers and machine learning classifier algorithms," Journal of Computational and Cognitive Engineering, vol. 2, no. 1, pp. 5-9, 2023.
[21] M. K. Islam, M. A. Al Amin, M. R. Islam, M. N. I. Mahbub, M. I. H. Showrov, and C. Kaushal, "Spam-detection with comparative analysis and spamming words extractions," in 2021 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), 2021, pp. 1-9.
[22] V. Metsis, I. Androutsopoulos, and G. Paliouras, "Spam filtering with naive bayes-which naive bayes?," in CEAS, 2006, vol. 17, Mountain View, CA, pp. 28-69.
[23] A. P. Bhopale and A. Tiwari, "An Application of Transfer Learning: Fine-Tuning BERT for Spam Email Classification," in Machine Learning and Big Data Analytics (Proceedings of International Conference on Machine Learning and Big Data Analytics (ICMLBDA) 2021), 2022, pp. 67-77.
[24] J. D. M.-W. C. Kenton and L. K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," in Proceedings of naacL-HLT, 2019, vol. 1, Minneapolis, Minnesota, p. 2.
[25] P. Tang and Y. Guan, "Log anomaly detection based on BERT," Signal, Image and Video Processing, pp. 1-11, 2024.
[26] F. Souza, R. Nogueira, and R. Lotufo, "BERT models for Brazilian Portuguese: Pretraining, evaluation and tokenization analysis," Applied Soft Computing, vol. 149, p. 110901, 2023.
[27] A. Vaswani, "Attention is all you need," in Advances in Neural Information Processing Systems, 2017.
[28] X. Luo, H. Ding, M. Tang, P. Gandhi, Z. Zhang, and Z. He, "Attention mechanism with BERT for content annotation and categorization of pregnancy-related questions on a community Q&A site," in 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2020, pp. 1077-1081.
[29] S. Lu, M. Wang, S. Liang, J. Lin, and Z. Wang, "Hardware accelerator for multi-head attention and position-wise feed-forward in the transformer," in 2020 IEEE 33rd International System-on-Chip Conference (SOCC), 2020, pp. 84-89.
[30] O. Galal, A. H. Abdel-Gawad, and M. Farouk, "Rethinking of BERT sentence embedding for text classification," Neural Computing and Applications, pp. 1-14, 2024.
[31] P. P. S. Bedi, M. Bala, and K. Sharma, "MLM: Masked Language Modeling Using Deep Learning for Efficient Summarization of Unstructured Data," in International Conference on Data Analytics & Management, 2023, pp. 339-347.
[32] S. Al-augby and K. Nermend, "Using Rule Text Mining Based Algorithm to Support the Stock Market Investment Decision," Transformations in Business & Economics, vol. 14, 2015.
[33] S. Kumar, J. R. Saini, and P. B. Bafna, "Identification of Malayalam Stop-Words, Stop-Stems and Stop-Lemmas Using NLP," in IOT with Smart Systems: Proceedings of ICTIS 2022, vol. 2, Springer, 2022, pp. 341-350.
[34] Z. Ch. Oleiwi, E. N. AlShemmary, and S. Al-Augby, "Developing hybrid CNN-GRU arrhythmia prediction models using fast Fourier transform on imbalanced ECG datasets," Mathematical Modelling of Engineering Problems, vol. 11, no. 2, pp. 413–429, Feb. 2024, doi:10.18280/mmep.110213.
[35] A. Ghourabi, M. A. Mahmood, and Q. M. Alzubi, "A hybrid CNN-LSTM model for SMS spam detection in arabic and english messages," Future Internet, vol. 12, no. 9, p. 156, 2020.
[36] L. GuangJun, S. Nazir, H. U. Khan, and A. U. Haq, "Spam detection approach for secure mobile message communication using machine learning algorithms," Security and Communication Networks, vol. 2020, no. 1, p. 8873639, 2020.
