Downloads: 0 | Views: 47

Subjective Paper | Computer Science and Information Technology | India | Volume 14 Issue 1, January 2025 | Popularity: 4.3 / 10

Rate This Article!      

ISL-CNN: A CNN based Automated System for the Recognition of Indian Sign Language for Hearing-Impaired

Reshna S, Haris P A, Imthias Ahamad T P, Jayaraju M

Abstract: Sign language is a visual/ gestural language used by people with hearing disabilities. It uses specific shapes and movements of the hands, arms and fingers along with movements of the head, face and eyes. Sign Language Recognition System is an automated system that can translate sign language into spoken language or text. Indian Sign Language (ISL) uses both hands to make gestures to represent most of the signs and one hand moves faster than the other at times in dynamic hand gestures. It involves both global and local hand motions. To determine all these aspects, the position of hands and head, configuration (angles and rotations), and movement (velocities) need to be identified. In this study, we developed ISL_CNN architecture to interpret signs in ISL. We used the dataset developed by Robotics and AI Lab, IIIT, Allahabad. We implemented a modified version of VGG16 Convolutional Neural Networks (ISL _CNN) for the classification of ISL signs of the English alphabet and isolated signs of 23 words. The dataset contained images of both static and dynamic signs. In our study, we used 11 dynamic signs and 9 static signs. The accuracy obtained for the alphabet datasets was 99.81% with 0.0034 loss and that of the ISL words dataset was 99.48% with 0.021 loss. The proposed system may be improved to predict all signs in the ISL dictionary by adding new words and terms, thus making the hearing-impaired person more independent. Additionally, a text-to-speech engine can convert these predicted words into speech.

Keywords: Convolutional Neural Networks, Indian Sign Language Recognition System, ISL_CNN, Static and Dynamic Gestures, VGG16

Edition: Volume 14 Issue 1, January 2025

Pages: 1046 - 1055

DOI: https://www.doi.org/10.21275/SR25124140109

Make Sure to Disable the Pop-Up Blocker of Web Browser

Click to Cite this Article

Similar Articles