2692-4048ISSN (Online)
2770-0070ISSN (Print)
Volume 17 , Issue 1 , PP: 135-145, 2025 | Cite this article as | XML | Html | PDF | Full Length Article
Esther Daniel 1 , V. Kathiresan 2 , Priyadarshini .C 3 , Golden Nancy .R 4 * , P. Sindhu 5
Doi: https://doi.org/10.54216/FPA.170110
Sign language recognition is important for enhancing message and user-friendliness for the community of deaf and hearing-impaired people. This paper proposes a Malayalam Sign Language (MSL) method using sign language that emerged from the state of Kerala. The main factor contributing to this emergence of such regional sign language is the absence of a standardized and consistent approach to the use of Indian Sign Language (ISL) in various states. This is due to the variations in signs, grammar, and syntax used in different regions. The system uses the You Only Look Once v8 (YOLOv8) algorithm-based object detection method which is based on Convolution Neural Network (CNN), a widely accepted deep learning neural network design employed mainly in computer vision. As the dataset for MSL is not publicly available, we used an MSL video from YouTube provided by the National Institute of Speech and Hearing for training a custom model. We pre-processed the video to extract the frames and annotate them with sign labels. Then, we trained the YOLOv8 algorithm on the annotated frames to detect the hand region and recognize signs in real time. The proposed approach achieved an accuracy of 97.21% calculated from the mean Average Precision value on the MSL dataset. The result achieved outperformed other existing approaches even while using less dataset count compared to others.
Malayalam sign language , Yolo, Computer vision , Deep learning , Machine learning , Convolution neural network
[1] N, Ajmi., Akhila Raj. B. R, Basina. B, Varsha. S, and Vishnu S Kumar. “Malayalam Sign Language Recognizer – IJERT.” Malayalam Sign Language Recognizer – IJERT, May 17, 2022. https://doi.org/10.17577/IJERTV11IS050083.
[2] Kunjumon, Jinsu, and Rajesh Kannan Megalingam. “Hand Gesture Recognition System For Translating Indian Sign Language Into Text And Speech.” 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT), November 2019. https://doi.org/10.1109/icssit46314.2019.8987762.
[3] Katoch, Shagun, Varsha Singh, and Uma Shanker Tiwary. “Indian Sign Language Recognition System Using SURF with SVM and CNN.” Array 14 (July 2022): 100141. https://doi.org/10.1016/j.array.2022.100141.
[4] Ultralytics. “Ultralytics | Revolutionizing the World of Vision AI,” n.d. https://ultralytics.com.
[5] Areeb, Qazi Mohammad, Maryam, Mohammad Nadeem, RoobaeaAlroobaea, and Faisal Anwer. “Helping Hearing-Impaired in Emergency Situations: A Deep Learning-Based Approach.” IEEE Access 10 (2022): 8502–17. https://doi.org/10.1109/access.2022.3142918.
[6] Daniels, Steve, NanikSuciati, and ChastineFathichah. “Indonesian Sign Language Recognition Using YOLO Method.” IOP Conference Series: Materials Science and Engineering 1077, no. 1 (February 1, 2021): 012029. https://doi.org/10.1088/1757-899x/1077/1/012029.
[7] Luthfy, Dicky, CasiSetianingsih, and Marisa W. Paryasto. "Indonesian Sign Language Classification Using You Only Look Once." eProceedings of Engineering 10, no. 1 (2023).
[8] Our Work - WFD World Federation of the Deaf, "Our Work - WFD," n.d. https://wfdeaf.org/our-work/.
[9] Stewart Kirubakaran S, V.P. Arunachalam, S. Karthik and S. Kannan (2022) Towards Developing Privacy-Preserved Data Security Approach (PP-DSA) in Cloud Computing Environment Computer Systems Science and Engineering Tech Science Press. DOI: 10.32604/csse.2023.02669044:1882-1895.
[10] National Association of the Deaf, n.d. http://nadindia.org/.
[11] Hearing Like Me. "Understanding India's Deaf Community | Hearing Like Me," August 6, 2021. https://www.hearinglikeme.com/notes-from-indias-deaf-community/.
[12] Roboflow Annotate. “Roboflow Annotate,” n.d. https://roboflow.com/annotate.
[13] Ju, Rui-Yang, and Weiming Cai. "Fracture Detection in Pediatric Wrist Trauma X-ray Images Using YOLOv8 Algorithm." arXiv preprint arXiv:2304.05071 (2023).
[14] Stewart Kirubakaran et.al A Brief Assessment on Stock Market Forecast using Long Short Term Memory (LSTM) Algorithm published in 2023 7th IEEE International Conference on Trends in Electronics and Informatics (ICOEI). DOI: 10.1109/ICOEI56765.2023.10125607.
[15] Selvakumar, S. K., Ashok, V., Selvadurai, J. J., Ebenezer, V., Nivin, A. J., & Aron, D. A. (2023). Divination of stock market exploration using long short-term memory (LSTM). Multidisciplinary Science Journal, 6(1), 2024001. https://doi.org/10.31893/multiscience.2024001.
[16] Issac, K., Daniel, E., & Varghese, B. G. (2023, May). A Deep Learning Approach for Recognizing Malayalam Sign Language Gestures. In 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 93-97). IEEE.
[17] Seetha, S., ChristlinShanuja, C., Daniel, E., Chandra, S., & Raj, S. (2022, April). Sign Language to Sentence Interpreter Using Convolutional Neural Network in Real Time. In International Conference on Computational Intelligence in Pattern Recognition (pp. 387-400). Singapore: Springer Nature Singapore
[18] Gunvantray, T. D., & Ananthan, T. (2024, March). Sign Language to Text Translation Using Convolutional Neural Network. In 2024 International Conference on Emerging Smart Computing and Informatics (ESCI) (pp. 1-5). IEEE.
[19] Konoor, A. M., & Padmavathi, S. (2022, June). A Hybrid Approach on Lexical Indian Sign Language Recognition. In International Conference on Soft Computing and Signal Processing (pp. 171-180). Singapore: Springer Nature Singapore.
[20] Gokulraj, V. A., Chirranjeavi, M., Aaruran, S., & Harikumar, M. E. (2023, October). Transhumeral Sensory System to Control Robotic Arm. In 2023 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER) (pp. 185-190). IEEE.
[21] Padmini, P., Paramasivam, C., Lal, G. J., Alharbi, S., & Bhowmick, K. (2022). A Real-Time Oral Cavity Gesture Based Words Synthesizer Using Sensors. Computers, Materials & Continua, 71(3).
[22] Jia, W., & Li, C. (2024). SLR-YOLO: An improved YOLOv8 network for real-time sign language recognition. Journal of Intelligent & Fuzzy Systems, 46(1), 1663-1680.
[23] Zhang, L. J., Fang, J. J., Liu, Y. X., Le, H. F., Rao, Z. Q., & Zhao, J. X. (2023). CR-YOLOv8: Multiscale object detection in traffic sign images. IEEE Access, 12, 219-228.
[24] Durga, S., Surya, S., & Daniel, E. (2018, May). SmartMobiCam: Towards a new paradigm for leveraging smartphone cameras and IaaS cloud for smart city video surveillance. In 2018 2nd International Conference on Trends in Electronics and Informatics (ICOEI) (pp. 1035-1038). IEEE.
[25] Lim, K. Y., Priyadarshi, A., Nadiah, N. F., Lee, J. H. J., Lau, J. X., Chew, C. K. L., ... & Wong, D. (2024, March). Towards Seamless Communication for Sign Language Support: Architecture, Algorithms, and Optimization. In International Conference on Science and Technology Education (pp. 401-410). Cham: Springer Nature Switzerland.
[26] S. Hemamalini ,V. D. Ambeth Kumar ,R. Venkatesan,S. Malathi. (2023). Relevance Mapping based CNN model with OSR-FCA Technique for Multi-label DR Classification. Journal of Fusion: Practice and Applications, 11 ( 2 ), 90-110.
[27] C. S. Manigandaa,V. D. Ambeth Kumar,G. Ragunath,R. Venkatesan,N. Senthil Kumar. (2023). De-Noising and Segmentation of Medical Images using Neutrophilic Sets. Journal of Fusion: Practice and Applications, 11 ( 2 ), 111-123.
[28] V. Sathya Preiya,V. D. Ambeth Kumar,R. Vijay,Vijay K.,N. Kirubakaran. "Blockchain-Based E-Voting System with Face Recognition." Fusion: Practice and Applications, Vol. 12, No. 1, 2023 ,PP. 53-63
