Enhancing parallel k-Means using map reduce for discovering knowledge from big data

Abstract

Knowledge discovery from data using clustering algorithm include stages of data preprocessing, clustering the preprocessed dataset and evaluating patterns for obtaining knowledge. Along with the popularity of Hadoop, k-Means algorithm has been enhanced based on MapReduce for clustering big dataset. We enhance this existing algorithm such that it includes the capabilities for performing data preprocessing, generating patterns and measures such that these can be used for evaluating the quality of clusters. Our preliminary experiment results in small Hadoop cluster indicate that our proposed technique performs well for clustering a case study big dataset.