Real-Time Bangla Sign Words Interpretation Uttered at Medical Emergencies

Abstract

Sign language is a medium of communication using gestures which is developed for people who suffer from auditory or verbal impairment. But sign language is usually not widely familiar to common people. For this reason, a sign language interpreter is essential for translating sign word representation to regular text or voice. Though there are some research works on American Sign Language interpretation, but it is inadequate in number for other languages. This chapter illustrates real time interpretation of medical emergency relevant Bangla sign words using machine learning. Considering medical emergency situations, a real time word level interpreter is designed, and the performance is evaluated. The designed deep neural network uses Mediapipe Holistic network for feature detection from human body in transitional movement and Long short term memory (LSTM) to predict the image sequences of each sign action. The designed system provides 99% model training accuracy and 92.22% test accuracy in real time detection.

Key Terms in this Chapter

OpenCV: A computer vision library.

LSTM: This stands for long short term memory network, i.e., applied for sequential data prediction.

Sign Word: Words that are conveyed with hand or body gesture or expression. Especially applied by verbally challenged people

CNN: This stands for convolutional neural network, which is used for pattern prediction.

MediaPipe Holistic: A framework developed by Google that is made opensource for realtime detection of body gesture