The Tool For Processing Big Data – Hadoop

In our previous blog we learned that the platform that processes and organizes Big Data is  Hadoop. Here we will learn more about Hadoop which is a core platform for structuring Big Data and solves the problems of utilizing it for analytic purposes. It is an Open Source software framework for distributed storage and distributed processing of Big Data on clusters of commodity hardware.

Main characteristics of Hadoop:

• Highly scalable (scaled out)
• Commodity hardware based
• Open Source, low acquisition and storage costs

Hadoop is basically divided into two parts namely : HDFS et Mapreduce framework. A Hadoop cluster is specially designed for storing and analyzing huge amounts of unstructured data. Workload is distributed across multiple cluster nodes that work to process data in parallel.

History of Hadoop

Doug Cutting is the brains behind Hadoop which has its origin in Apache et Nutch. Nutch was started in 2002 and it itself is an Open Source web search engine. Google published the paper that introduced the Mapreduce to the world.  In early 2005  Nutch developers had a working Mapreduce implementation in Nutch. In February 2006  Hadoop was formed as an independent project by Nutch. In January 2008 Hadoop has made its own top level project at Apache and by this time major companies like Yahoo and Facebook started using  Hadoop.

HDFS is the first aspect and Mapreduce  is the secondary aspect of  Hadoop.  HDFS has an architecture which helps it in processing the data and organizing it. To get into details of  HDFS, its architecture, functioning and several other concepts, keep an eye on the blogs that will be published in coming days.

 Prenez contact avec nous.

Manasa Heggere

Développeur senior Ruby on Rails