Hadoop job information for Stage-1: number of redu

问题:

I am using Hive over a hadoop cluster. Whenever I am trying the run a hive query the it always shows as

Hadoop job information for Stage-1: number of reducers: 1

I had used following Hive configuration :

hive.exec.reducers.bytes.per.reducer=1000000000
hive.exec.reducers.max=999

please tell me how can of increase the number reducer.

Thanks.

回答1:

Have you tried to set mapred.reduce.tasks parameter?

回答2:

Make sure you have gone through the points given below:

Your mapred.reduce.tasks defaults to -1. By setting this property to -1, Hive will automatically figure out what should be the number of reducers.

A hive query with no grouping will be using only 1 reducer.

And also note that when mapred.job.tracker is "local", hadoop sets mapred.reduce.tasks value to 1 by default.