Beehive, A MapReduce framework for highly distributed swarms

Abstract

The exponential increase in data generated gives rise to more demanding requirments for processing it. A parallel programming paradigm known as MapReduce, has been used widely to process this data. There exist frameworks that have incorporated MapReduce style processing. However, these frameworks are not focused to be energy-efficient. We investigate a novel approach for processing large amounts of data. This is done by developing a framework that supports distributed processing, with focus on utilizing green energy. The main aim of this project is to facilitate processing in data centers powered by renewable energy. This is attempted by combining technologies used in green computing, grid computing and cloud computing. The results from this project were a fully developed task tracker along with reference implementation of the other components. At the current stage of development there was only found to be a 2% decrease in performance when compared to a local cluster. This project was also proved ready for future work, that would contribute to the main aim of the framework.