JobTracker UI not showing progress of hadoop job

Question

I am testing my MR jobs under a single node cluster. Once I installed mahout 9 version Mapreduce jobs stopped showing the progress in jobtracker.(Dont know if that happened after mahout installation)

When ever I run a job in my hadoop cluster it wont show the status in job tacker UI as previous and the execution log displaying in the console is also different (similar to mahout logs)

Why is it so?

Thanks In Advance.

Answer 1

Most probably you job might be running using LocalJobRunner. If your job is using LocalJobRunner you'll notice mapred.LocalJobRunner events in the mapreduce job output. This could of couple of reasons:

In case of MapReduce v1:

Make sure your mapred-site.xml has the property mapred.job.tracker and is pointing to appropriate JobTracker host and port:

<property>  
  <name>mapred.job.tracker</name>  
  <value>[fqdn_of_jobtracker]:[port_of_jobtracker]</value>  
</property>

Generally, LocalJobRunner will be choosen by the mapreduce if the hadoop cannot read the property mapreduce.framework.name or if the property has been set to local.

If this does not pan out then try explicitly specifying the property using hadoop's generic command line option like:

hadoop jar hadoop-examples.jar pi -jt [fqdn_of_jt]:[port_of_jt] 2 1000

In case of YARN and MapReduce v2:

If you are running MapReduce on top of YARN in that case, the job would be run by LocalJobRunner if you haven't specified the mapreduce framework name in mapred-site.xml, so make sure you have the following property:

<property>  
  <name>mapreduce.framework.name</name>  
  <value>yarn</value>  
</property>