Sign Language to Text Conversion in Real Time using Transfer Learning

• DOI: 10.48550/arxiv.2211.14446
• Publication Date: 2022-10-07
• Authors: Thakar, Shubham; Shah, Samveg; Shah, Bhavya; Nimkar, Anant V.
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Machine Learning (cs.LG)

Will be published in IEEE Explore; Typos corrected<br/>The people in the world who are hearing impaired face many obstacles in communication and require an interpreter to comprehend what a person is saying. There has been constant scientific research and the existing models lack the ability to make accurate predictions. So we propose a deep learning model trained on ASL i.e. American Sign Language which will take actions in the form of ASL as input and translate it into text. To achieve the translation a Convolution Neural Network model and a transfer learning model based on the VGG16 architecture are used. There has been an improvement in accuracy from 94% of CNN to 98.7% of Transfer Learning, an improvement of 5%. An application with the deep learning model integrated has also been built.<br/>