This post explains the functions of different types of Hadoop Daemons which run inside Hadoop cluster.
Note: This covers the Hadoop architecture for Hadoop version <1.x series . For latest architecture Hadoop 0.23 ( 2.x ) series there would be separate post.
When we submit job to Hadoop cluster our task go to Job Tracker which handles assigns the tasks to various Task Trackers.
The TaskTrackers start map reduce jobs and complete the processing for the submitted job.
Job Tracker also keeps track incase something fails at any TT and assign that part of job to someone else
There is only one JobTracker in the system
Namenode keeps track of data blocks for particular file . File blocks are actually stored
in Datanodes which tell NN from time to time about health of blocks , and which all blocks
are stored in them.
Incase some block is lost due to some reason , NN can replicate that from copy of block which
is stored in some other machine
In core-site.xml and mapred-site.xml we specify the hostname and port of the NameNode and the JobTracker.
Note: This covers the Hadoop architecture for Hadoop version <1.x series . For latest architecture Hadoop 0.23 ( 2.x ) series there would be separate post.
When we submit job to Hadoop cluster our task go to Job Tracker which handles assigns the tasks to various Task Trackers.
The TaskTrackers start map reduce jobs and complete the processing for the submitted job.
Job Tracker also keeps track incase something fails at any TT and assign that part of job to someone else
There is only one JobTracker in the system
Namenode keeps track of data blocks for particular file . File blocks are actually stored
in Datanodes which tell NN from time to time about health of blocks , and which all blocks
are stored in them.
Incase some block is lost due to some reason , NN can replicate that from copy of block which
is stored in some other machine
In core-site.xml and mapred-site.xml we specify the hostname and port of the NameNode and the JobTracker.
No comments:
Post a Comment
Please share your views and comments below.
Thank You.