I downloaded the CDH4 tar for Hadoop with Yarn, and jobs are running fine, but I can't figure out where to view the logs from my job. In MRv1, I simply went to the JobTracker web app, and it had the job history. Individual jobs' logs were accessible from here as well, or by going to logs/userlogs directory.
In my new Yarn setup (just running on single computer), I have the logs directory, but no logs/userlogs folder.
When I go to the ResourceManager web page, localhost:8088, there is an "All Applications" section, but no entries show here. When I go to the JobHistory web page, localhost:19888, there are no jobs listed.
I'm wondering if there is a configuration issue. Here's my yarn-site.xml entries:
<property>
<name>yarn.nodemanager.local-dirs</name>
<value>/Users/dolan/cdh4/_hadoop_data/yarn/local</value>
</property>
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>/Users/dolan/cdh4/_hadoop_data/yarn/logs</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>localhost:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>localhost:19888</value>
</property>
<property>
<name>yarn.app.mapreduce.am.staging-dir</name>
<value>/user</value>
</property>
Any tips on what to debug are greatly appreciated.
Update: I've checked the nodemanager log dir, and it is empty. Additionally, the jobhistory directories are empty. One odd thing is that Hadoop tells me, INFO mapreduce.Job: The url to track the job: http://localhost:8080/
, but that address isn't available when I try from browser.
Update #2: I noticed that each job I run has the same Job ID ("job_local_0001"). This seems like a configuration issue somewhere.