Currently, transfering a huge amount of video data from on premise to the public cloud is very slow. In this article, the researcher uses a set of self-developed software written in Python, C, and Bash to improve the speed of data transmission and analysis of drone-generated videos taken in eight different cities on the American continent to the public cloud of Amazon Web Services. The author uses several tools, compression, parallel threads execution, and the autoscaling feature of the public cloud vendor to tune the process of transmission and analysis in 78% of the speed compared to the common sequential transmission and single node option.
TopIntroduction
There has been great growth in the use of drones for research. The drone is “a remote-controlled pilotless aircraft or missile” capable of visualizing extended areas of study in less time than the human eye (OxfordDictionaries.com, 2018). Many drones can send live video streaming to mobile phones or computers. Data collected by drones provides useful information and patterns to deploy new technologies and generate deeper data analysis. These data, including information on movement, color, or area patterns, can be processed per frame and mapped as color pixels or represented as a set of Extensible Markup Language (XML) files.
According to Mell and Grance (2011, p. 2) from the National Institute of Standards and Technology (NIST):
Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.
The public cloud, such as Amazon Web Services (AWS) or the Google Cloud Platform (GCP), offers reliable and nearly unlimited storage (www.aws.amazon.com).
Photogrammetry is the use of photography in surveying and mapping to ascertain measurements between objects (OxfordDictionaries.com, 2018) and is an example of how a machine can be used to process images collected from drones. Monthly data can be efficiently processed by integrating and analyzing terabytes (TB) or petabytes (PB) from drones on the public cloud. However, there is not a faster way to send large amounts of data from a premise server to the public cloud.
The current method of processing data from drones on the cloud use raw data and these were sent in sequential order. This method depends on the speed of transference of every packet from the base station to the cloud. Also, if the company use only one node in the cloud for processing, then the overall process is sequential and very slow. This amount of time could be very long with a regular video of 300 MB. Using a set of videos of regular size can take several hours or days, depending on the bandwidth and the flow between the intermediate network devices.
In this article, the researcher will use a set of tools and an automation process to study a unique method of compress the output of drone videos and improve the transference of data from the on-premise base station to the cloud to analyze them in a concurrent environment on the public cloud. After discussing alternatives, the chapter will review process data calculations to compare faster methods. With modifications to upload times, the analyzed data time is reduced by an average of 78% as compared to a common sequential process. This valuable time can be used by companies to be focused on their experiment and business results instead of the processing of the data sets.
TopBackground
Several studies have discussed the integration of drones or robots and the public cloud. In the internet of drones theory (Gharibi, Boutaba, & Waslander, 2016), the authors created a system to manage and control access to drones and their activities to navigate service locations. A study by Yinong and Garcia (2010) reviewed the field of robots as a service in cloud computing. Through distributed systems, the researchers controlled a network of segregated robots and sent various programs to the robots to deliver different services.
In a service-oriented architecture (SOA) study, researchers created a Web-based system hosted in the public cloud to manage drone networks (Koubaa, Qureshi, Sriti, Javed, & Tovar, 2017). In a robust study by Mahmoud, Mohamed, and Al-Jaroodi (2015), the authors designed a technological system using unmanned aerial vehicles (UAVs) as nodes based on the theory of the internet of things (IoT). Using the RESTful HTTP technology and Arduino boards, the UAVs transmitted data with the internet.