Augmented Reality-Based 3D Human Hands Tracking From Monocular True Images Using Convolutional Neural Network

Augmented Reality-Based 3D Human Hands Tracking From Monocular True Images Using Convolutional Neural Network

A. F. M. Saifuddin Saif, Zainal Rasyid Mahayuddin
Copyright: © 2023 |Pages: 9
DOI: 10.4018/978-1-6684-5849-5.ch008
OnDemand:
(Individual Chapters)
Available
$37.50
Current Special Offers
No Current Special Offers
TOTAL SAVINGS: $37.50

Abstract

Precise modeling of hand tracking from monocular camera calibration parameters using semantic cues is an active area of research for the researchers due to lack of accuracy and computational overheads. In this context, deep learning-based framework (i.e., convolutional neural network-based human hands tracking in the current camera frame) has become an active research problem. In addition, tracking based on monocular camera needs to be addressed due to updated technology such as Unity3D engine and other related augmented reality plugins. This research aims to track human hands in continuous frame by using the tracked points to draw 3D model of the hands as an overlay. In the proposed methodology, Unity3D environment was used for localizing hand object in augmented reality (AR). Later, convolutional neural network was used to detect hand palm and hand keypoints based on cropped region of interest (ROI). The proposed method achieved accuracy rate of 99.2% where single monocular true images were used for tracking. Experimental validation shows the efficiency of the proposed methodology.
Chapter Preview
Top

2. Previous Study

Development of own libraries for augmented reality, placement of virtual 3D objects in order to check the suitability in the virtual environment turned lots of active research interest in the area of augmented reality in the broader aspects of computer vision and deep learning especially convolutional neural network (CNN) (Tanzi et al., 2021; Su et al., 2021; Perdpunya et al., 2021; Lan, 2021; Saif and Mahayuddin, 2020). Earlier of this research trends, recognition of hands and tracking them were done by offline image processing causes huge computational overheads. After that, human hands tracking in real time was progressed using web camera and personal computer. Later, convolutional neural network, features tracking and modeling the network made the overall process easier for human hand tracking. At the current progress of research progress, human hands tracking via mobile phone camera as well as recognizing the pose of the hands in the current camera frame becomes active research problem. This research aims to track human hands pose in continuous frame by using the tracked points to draw a 3D model of the hands as an overlay in the original tracked imaged. In addition, this research also used the points to generate colliders and effectors to interact with virtual objects. Area of previous research methods is shown in Fig.1.

Complete Chapter List

Search this Book:
Reset