Big Data Analytics and Its Applications in IoT

Overview Of Big Data:

In general, the Big Data is defined using 3Vs such as Velocity, Volume and Variety. Velocity is the rate at which the data grows and how fast the data is gathered for analysis. Volume refers to the enormous data being generated, whereas Variety refers to the different types of data being generated like structured, semi- structured and unstructured data. There is a fourth V to describe the big data, it is referred as veracity, which includes availability and accountability. In general, data generated by sensors in IoT applications will be considered as raw data. This raw data need to be fine-tuned before it is used by the decision makers. In general, during data collection, it is noticed that data missing, data redundant, data in the wrong format, etc. Hence, preprocessing is needed to get the relevant data in required format to avoid erroneous and misleading outputs that reduce the efficiency. According to the B2B report of data quality index test, it was shown that every data repository has got 40% bad data in which 15% are duplicate, 8% are missing, 11% are invalid and 6% comes from malicious or unauthorized users. These data weaken the organization's marketing and financial automation, increase the resource consumption and cost and leads to lower customer satisfaction and invalid reports. Thus, there is a great need of data preprocessing. Apache Hadoop, Apache Spark, etc are used for this purpose. Hence, we use techniques such as data mining, analytical tools and machine learning to extract the useful information from the big data.