Sign Language Recognition for Dumb & Deaf People using Python, OpenCv & Tensor Flow

Abstract

This study examines the various stages of an automatic system for the identification of sign language (SLR). A large dataset and the best algorithms must be used to train a system that can read and understand a sign. In current society, there is a lack of communication with the deaf. The use of Sign Language (SL) helped to break down this barrier as it uses visually conveyed sign patterns to communicate meaning to non-sign language users. It is also useful in communicating with individuals suffering from autism spectrum disorder (ASD). Normal people cannot understand the signs used by deaf people, as they do not know the meaning of a particular sign. The system presented is intended to address this issue. The system captures various gestures of the hand by using a webcam. Pre-processing of the image takes place, then, determination of edges occurs by using object detection. Finally, a template-matching algorithm identifies the sign and displays the text. The output is in a textual format so one can easily interpret the meaning of a particular sign. This also curtails the difficulty of communicating with the deaf. The implementation of the system is by creating libraries then using tensor flow object detection pipeline configuration for object detection and finally running the model in real-time by using OpenCV in Python in real time and obtaining an audio message of the indicated hand sign.