This chapter provides an overview of the diverse applications of ML in UAVs for object and people detection and tracking. It begins by examining the current landscape of ML-driven UAV technologies and their potential. The related work section discusses the advancements in object and people detection and tracking. The subsequent sections delve into the technical aspects, focusing on the next generation of UAV convolutional neural network (CNN) backbones, including the contextual multi-scale region-based CNN (CMSRCNN), single shot multibox detector (SSD), and you only look once (YOLO), highlighting their significance in enhancing detection capabilities. Furthermore, it explores practical applications of ML in UAVs, encompassing object and people detection and tracking, path planning, navigation, and image and video analysis. Challenges and complexities in vision-based UAV navigation are addressed. Additionally, it showcases the potential for UAV networks to locate objects in real time.
Top1. Introduction
Unmanned Aerial Vehicles (UAVs), commonly known as drones, have witnessed extensive utilization across both military and civilian domains, ranging from search and rescue missions to exploration and surveillance (Telli et al., 2023). The integration of Machine Learning (ML) techniques into UAVs has unlocked a myriad of possibilities, including advanced object detection and tracking capabilities . The amalgamation of UAVs and ML has ushered in a new era of enhanced situational awareness and operational efficiency. Particularly noteworthy are the applications of ML in UAV-based object detection and tracking (Khalil et al., 2021) (Mohan et al., 2021)
(Zhu et al., 2017)introduced a Contextual Multi-Scale Region-based Convolutional Neural Network (CMS-RCNN) for unconstrained face detection, which was later extended for UAVs). Moreover, the capabilities of UAVs extend beyond individual object detection and tracking. (Oubbati et al., 2022) leveraged deep reinforcement learning for UAV-enabled mobile relaying systems, while Alqurashi et al. (2021) focused on Machine Learning Techniques in Internet of UAVs for Smart Cities Applications. These studies indicate the diversity of ML applications in UAVs, ranging from communication improvement to navigation assistance (Saeed M. M. et al., 2023).
However, while the synergy of UAVs and ML holds immense promise, the field is not devoid of challenges. The integration of ML algorithms in UAV systems necessitates addressing computational constraints and energy efficiency, ensuring real-time processing for swift decision-making, and maintaining data privacy and security (Bakri Hassan et al., 2022) . Thus, this introduction provides a glimpse into the exciting realm of ML applications in UAVs, while acknowledging the existing challenges that researchers and practitioners must navigate.
1.1 Motivation
The goal for this chapter is to explore the applications of machine learning in UAV-based object and people detection and tracking. The usage of unmanned aerial vehicles (UAVs) outfitted with thermal cameras has attracted interest in the field of object identification, recognition, and tracking. However, there are limitations in the existing frameworks, such as the inability to track things outside the field of view (FOV) of the UAV camera and the issue of discriminating between many objects in close proximity (Saeed M. M., et al., 2022).
To solve these shortcomings, this research provides a robust system that extends the automatic detection, recognition, and tracking architecture. The technology improves the tracking module's capacity to distinguish objects during revisitation by the UAV. It also extends the tracking method to track several objects in Earth-fixed coordinates, even when they are outside the FOV of the camera. Additionally, modifications are made to the UAV payload to enable onboard and real-time image processing (Saeed M. M. et al., 2023).
The combination of algorithms and UAV payload proposed in this paper forms a state-of-the-art system that excels in robustness. The system can track things for extended periods of time, even when they are outside the FOV of the camera(F. Huang et al., 2023). It also leverages thermal imaging properties for object detection, enabling improved difference between items in close proximity and recognizing objects re-entering the FOV. The report provides several experimental results from field tests that involve multi-object tracking scenarios with a fixed-wing UAV. The results illustrate the effectiveness and reliability of the suggested solution (Leira et al., 2021).
In summary, this chapter tries to overcome issues in UAV-based object detection and tracking by applying machine learning approaches. The suggested system achieves resilience by extending object tracking capabilities beyond FOV restrictions and exploiting thermal image properties for improved object detection. The testing findings demonstrate the usefulness of this strategy in numerous field tests scenarios (Guo et al., 2023; Wu et al., 2021).