A Review of Real-Time Sign Language Translation Systems Based on Deep Learning Approach

Abstract

Non-verbal people use sign language (SL) as their major means of communication. This is a critical communication gap because the general population cannot understand SL. In the past, sign language recognition (SLR) systems had a high number of challenges, such as a low degree of accuracy, use of bulky external sensors, and inability to break down and decode continuous sign patterns in real time. It is time to systematically analyze the shift in the paradigm between applying the old methods with feature engineering and the new methods of deep learning to determine the most promising avenues of application in the real-time environment. The given review paper is a thorough overview of recent progress in the area of vision-based and sensor-based sign language detection translation systems. It is centered on the review of the development of approaches, performance measurement, and the use of deep learning methods. The most commonly used deep convolutional neural networks (CNNs) include Xception, Faster R-CNN, and Inception-V3, which are effective in accurate gesture recognition and classification. More advanced systems are real-time, attention-based encoder-decoder-based systems and recurrent networks to continuously translate a video or multimodal sensor stream to text/speech. Studies are commonly focused on local sign languages, such as Bangladeshi Sign Language, Indian Sign Language, and American Sign Language. This review reveals a strong move in favor of end-to-end deep learning systems that can provide high-end, low-latency translation systems to be deployed on mobile devices and in real-time. Irrespective of these achievements, future research should focus on the standardization of large-scale and continuous sign language datasets and the creation of strong and generalized models with the ability to manage various environmental conditions, as well as the dynamic complexities of signing.