Sign Language Recognition for Daily Activites Using Deep Learning

Introduction

The conversation is a key mechanism by which information sharing is being done among the people. People with disabilities need some way of communication to deal with normal and disabled people. But this is a challenging and difficult process when normal and deaf and mute people try to communicate. The best and most effective way of communication for deaf and dumb persons is sign language. Sign language communication not only benefits the communication between the deaf and dumb. But also benefits the communication between normal and deaf and dumb people. A deaf and mute person uses hand gestures for communication. Also, for normal people understanding their gestures is a complicated task. Hence, there is a need for a proper system which could resolve this problem and thereby make communication easier. Many systems have been developed for this process and still finding a complete solution is in research (Tolentino, 2019). The primary goal of this chapter is to find the gap between the current sign language identification system in the aspect of accessibility, cost and problem to apply in day-to-day activities. The first and primary challenge is collecting the real-time dataset. In the real scenario, capturing the images contains various backgrounds with noises. So, an efficient pre-processing step is required for accurate prediction. After pre-processing, the next step is segmentation which separates the background from the target signs. Multiple image processing algorithms are used to segment the image. The image after the segmentation process is modelled through machine learning algorithms and deep learning algorithms.

For disabled persons, the most popular language used is the American Sign Language (ASL). The pre-processed images are readily available as a dataset for ASL languages. But most of the existing datasets are of the alphabet. The current existing dataset is not suitable for recognizing words and day-to-day activities. Few of the datasets available in Indian sign languages are collected for words but the signs cannot be considered universal which differs with each country. The developed systems for sign language detection involve two basic approaches.

The first is contact-based i.e. using sensors and gloves hardware for gesture understanding and the second one is a non-contact vision-based system. The vision-based system can be categorized either as static or dynamic. The static system uses two-dimensional images for the analysis whereas the dynamic system uses proper cameras for capturing the real-time movements (Ismail et al, 2021). The hardware glove-based method is not comfortable for the users. But, the vision-based system is easier to implement than the hardware-based contact system and no physical contact is needed with the users. But the major issues to be addressed in vision-based are the accuracy of the detection and the recognition time. This accuracy of detection and recognition time can be improved by using proper machine/deep learning algorithms.