Improvement of K-Means Algorithm for Accelerated Big Data Clustering

Improvement of K-Means Algorithm for Accelerated Big Data Clustering

Chunqiong Wu, Bingwen Yan, Rongrui Yu, Zhangshu Huang, Baoqin Yu, Yanliang Yu, Na Chen, Xiukao Zhou
DOI: 10.4018/IJITSA.2021070107
OnDemand:
(Individual Articles)
Available
$37.50
No Current Special Offers
TOTAL SAVINGS: $37.50

Abstract

With the rapid development of the computer level, especially in recent years, “Internet +,” cloud platforms, etc. have been used in various industries, and various types of data have grown in large quantities. Behind these large amounts of data often contain very rich information, relying on traditional data retrieval and analysis methods, and data management models can no longer meet our needs for data acquisition and management. Therefore, data mining technology has become one of the solutions to how to quickly obtain useful information in today's society. Effectively processing large-scale data clustering is one of the important research directions in data mining. The k-means algorithm is the simplest and most basic method in processing large-scale data clustering. The k-means algorithm has the advantages of simple operation, fast speed, and good scalability in processing large data, but it also often exposes fatal defects in data processing. In view of some defects exposed by the traditional k-means algorithm, this paper mainly improves and analyzes from two aspects.
Article Preview
Top

1. Introduction

At present, with the rapid development of information technology such as computers, networks, and communications, information processing is rapidly industrializing on the entire social scale (Wilson et al., 2015; Zheng, 2015). In particular, the rapid rise of platforms such as “Internet +” has put people in the “ocean” of data all the time (Liu, 2019). Behind these huge amounts of data often contains very rich information. Relying on traditional data retrieval and analysis methods can no longer solve our need to obtain information. Traditional data management models are no longer suitable for today's data management needs. Therefore, data mining technology, the rapid rise has become one of the efficient solutions to how to quickly obtain useful information in today's society (Shu et al., 2017).

Data mining is to extract valid and potentially useful information from a large amount of raw data, and then express it through an understandable model (Chen, Feng, & Mao, 2019). Therefore, data mining has become more and more important and favored. Data mining is both a data processing method and a data processing process, and cluster analysis is an important method in data mining. K-means algorithm is one of the most classic and simple methods in cluster analysis. Although the K-means algorithm is simple to operate and easy to understand, there are some problems, such as ambiguity and subjective experience in determining the value of k, and the random selection of k initial center points has a great impact on the clustering result.

The text data clustering research has been developed for a long time since it was proposed, and its related technical methods have been initially formed, but the existing technical methods have more or less certain deficiencies and defects. In recent years, with the continuous deepening of data mining technology, more and more neighborhoods can see the application of k-means algorithm. Diab Abuaiadah (Abuaiadah, 2016) studied the analysis of five commonly used similarity and distance functions (Pearson's correlation coefficient, cosine, Jacquard coefficient, Euclidean distance, and average Kullback-Leibler divergence) when analyzing Arabic documents. The performance of the bisect k-means clustering algorithm and the traditional k-means algorithm are presented. The Lu team (Lu & Lu, 2016) proposed a fast genetic K-means algorithm (FGKA). The algorithm is improved on the GKA algorithm. Experiments show that FGKA and GKA always always converge to the global optimum, and that FGKA runs much faster than GKA. The Mexicano A team (Mexicano et al., 2015) proposed a fast mean algorithm based on the K-means algorithm, which can reduce the transaction data set time by up to 99.02%, while reducing the quality by only 7.62%. The Chunhua P team (Chunhua et al., 2015) aimed at the fluctuation of DG (distributed power generation) output and the uncertainty of load demand, which may cause the planned DG capacity to become larger or the improvement rate of the voltage curve to be reduced. K-means clustering and multi-scenario probability analysis are proposed and used to reduce the impact of volatility and uncertainty on the distribution network. The Dimitri J team (Dimitri, 2018) applied the k-means algorithm to the latest commercial mixed integer programming solver and achieved significant improvements. Wang Y's team (Wang & Deng, 2015) has some limitations on HMAX models in modeling V2 neurons or higher-level visual cortex during the simulation of visual recognition in primate visual cortex. The k-means algorithm is proposed to be applied to the model. The experimental results show that the method effectively solves these limitations. The Niu B team (Niu et al., 2016) proposed to mix five PSO and K-Means, which have shown good prospects in continuous function optimization, respectively, and thus obtained five PSO-KM-based clustering methods.

Complete Article List

Search this Journal:
Reset
Volume 17: 1 Issue (2024)
Volume 16: 3 Issues (2023)
Volume 15: 3 Issues (2022)
Volume 14: 2 Issues (2021)
Volume 13: 2 Issues (2020)
Volume 12: 2 Issues (2019)
Volume 11: 2 Issues (2018)
Volume 10: 2 Issues (2017)
Volume 9: 2 Issues (2016)
Volume 8: 2 Issues (2015)
Volume 7: 2 Issues (2014)
Volume 6: 2 Issues (2013)
Volume 5: 2 Issues (2012)
Volume 4: 2 Issues (2011)
Volume 3: 2 Issues (2010)
Volume 2: 2 Issues (2009)
Volume 1: 2 Issues (2008)
View Complete Journal Contents Listing