Job queue for Hive action in oozie

2019-01-20 17:25发布

I have a oozie workflow. I am submitting all the hive actions with

<name>mapred.job.queue.name</name>
<value>${queueName}</value>

But for few hive actions, the job launched is not in specified queue; it is invoked in default queue.

Please suggest me the cause behind this behavior and solution.

1条回答
对你真心纯属浪费
2楼-- · 2019-01-20 17:51

A. Oozie specifics Oozie propagates the "regular" Hadoop properties to a "regular" MapReduce Action.

But for other types of Action (Shell, Hive, Java, etc.) where Oozie runs a single Mapper task in YARN, it does not consider that it's a real MapReduce job. Hence it uses a different set of undocumented properties always prefixed with oozie.launcher. Look into that post for example.

So in your case the actual property to set would be oozie.launcher.mapred.job.queue.name

B. TEZ specifics Excerpt from HortonWorks documentation:

For example, in Hive you can use the tez.queue.name property in hive-site.xml to specify the queue to use for Hive-on-Tez jobs. To assign Hive-on-Tez jobs to use the "engineering" queue, add the following property to hive-site.xml:

<property>  <name>tez.queue.name</name> <value>engineering</value> </property>
查看更多
登录 后发表回答