Real-Time Object Detection and Audio Output System for Blind Users: Using YOLOv3 Algorithm and 360 Degree Camera Sensor

Introduction

The ability to perceive and understand our surroundings is essential for human survival, and our senses of sight and hearing play a critical role in this process. However, for people with visual impairments, the world can be a challenging and often isolating place. Blindness is a complex condition that affects people in different ways, but one of the most significant challenges faced by blind individuals is the inability to see and navigate their environment independently. In this context, technology can play a vital role in enhancing the independence and quality of life for blind individuals.

Real-time object detection and audio output systems have emerged as a promising technology for aiding the mobility of visually impaired individuals. These systems use camera sensor to detect objects in real-time, which are then translated into audio signals that can be interpreted by the user. This allows blind individuals to navigate their environment safely and independently, improving their quality of life and increasing their sense of self-reliance.

Figure 1.

Illustration of the overall system architecture

Overall, the real-time object detection and audio output system developed in this project has the potential to significantly improve the quality of life for blind individuals by enabling them to navigate their environment safely and independently. The use of YOLOv3 algorithm and 360-degree camera sensor offers a powerful and effective solution for real-time object detection.

One of the most popular and effective object detection algorithms is YOLO (You Only Look Once), which has been widely used for real-time object detection in various applications. In this project, we aim to develop a real-time object detection and audio output system for blind users using the YOLOv3 algorithm and a 360-degree camera sensor. The system will be able to detect and identify objects in the user's environment and provide audio output to the user.

The use of a 360-degree camera sensor in the system allows for a comprehensive view of the user's environment, enabling the detection of objects from all angles. This feature is particularly useful in environments with obstacles or complex layouts, where traditional cameras may not provide sufficient coverage.

Background

Object detection has been an active research topic in the field of computer vision for several years. One of the most popular and successful object detection frameworks is YOLOv3 (You Only Look Once), which is known for its speed and accuracy. YOLOv3 uses a deep convolutional neural network (CNN) to detect objects in an image, and it is able to perform this task in real-time on a variety of hardware platforms.

Several recent studies have explored the use of object detection to assist visually impaired individuals. For example, the paper by Sai Nikhil Alisetti et al. proposed a predictive object tracking system based on deep CNNs for guiding and navigation of blind people (Alisetti et al., 2019). Similarly, the work by Maid et al. focused on developing an object detection system for blind users (Maid et al., 2021). Another study by Sagana et al. presented an object recognition system for visually impaired people (Sagana et al., 2019).

In addition to object detection, the use of 360 cameras has become increasingly popular in recent years. These cameras are capable of capturing a 360-degree view of the environment, which makes them ideal for applications such as virtual reality and augmented reality. The paper by W. Yang et al. proposed an object detection system for equirectangular panorama images captured using a 360-degree camera (Yang et al., 2018).