Comparing Big Data Analysis Techniques

1. Introduction

Analyzing the big data is nothing but processing raw data to get meaningful and relevant data. Retrieved meaningful data can be useful in many applications which are based on different approaches like decision making, probabilistic analysis, and deterministic analysis. With the help of existing data analysis techniques, one can form the business strategies like identifying customer segments, customer retention, possible future business growth etc. However, it is suggested one can use data analysis technique as per the requirement i.e. type of input data and insights to uncover.

With the increasing size of users and applications data size is also increasing & in future it could be more than expected. Therefore, in future there would be a requirement of machines with good configurations & best analysis techniques. There are many existing analysis techniques available like regression analysis, time series analysis, sentiment analysis, descriptive analysis, predictive analysis, diagnostic analysis, association analysis with sampling, ensemble method for analysis, machine learning, visualization techniques, classification, and genetic algorithms and qualitative & quantitative analysis.

Big data refers to the large volume of data and can be characterized on the basis of volume, variety and velocity (V3 model of big data). Volume refers to the quantity of data and keeping the storage as per the requirement. Variety refers to the different data i.e. structured and unstructured data and can have different forms. Velocity refers to the keeping an eye on incoming data with the ever increasing size of users and applications. Big data analysis techniques must deal with the metadata i.e. data about data. Metadata reveals the quality, and characteristics about the data. To process and retrieve the meaningful data, it is expected that analysis techniques must be capable enough to process the metadata too. Identifying the ‘relevance’ among the different segments (different variables) data is very important & known as ‘association rule’ mining. It is also expected to process the real time data (RT data), qualitative and quantitative data in order to get the relevant data from large volume. Big data refers to processing the ‘n’ large datasets & more sophisticated big data analysis techniques can be applied to extract the relevant data with less processing time. Examples of big data are as follows – server web logs, patient records in hospitals, data related to surveillance, institutional data, large time series data, audio and video archives, and the most popular datasets from ‘e-commerce’ industries.