Parallel K-Means for Big Data: On Enhancing Its Cluster Metrics and Patterns

Abstract

K-Means clustering algorithm has been enhanced based on MapReduce such that it works in distributed Hadoop cluster for clustering big data. We found that the existing algorithm have not included techniques for computing the cluster metrics necessary for evaluating the quality of clusters and finding interesting patterns. This research adds this capability. Few metrics are computed in every iteration of k-Means in the Hadoop’s Reduce function such that when it is converged, the metrics are ready to be evaluated. We have implemented the proposed parallel k-Means and the experiments results show that the proposed metrics are useful for selecting clusters and finding interesting patterns.