“Too many fetch-failures” while using Hive

问题:

I'm running a hive query against a hadoop cluster of 3 nodes. And I am getting an error which says "Too many fetch failures". My hive query is:

  insert overwrite table tablename1 partition(namep)
  select id,name,substring(name,5,2) as namep from tablename2;

that's the query im trying to run. All i want to do is transfer data from tablename2 to tablename1. Any help is appreciated.

回答1:

This can be caused by various hadoop configuration issues. Here a couple to look for in particular:

• DNS issue : examine your /etc/hosts
• Not enough http threads on the mapper side for the reducer

Some suggested fixes (from Cloudera troubleshooting)

• set mapred.reduce.slowstart.completed.maps = 0.80
• tasktracker.http.threads = 80
• mapred.reduce.parallel.copies = sqrt (node count) but in any case >= 10

Here is link to troubleshooting for more details

http://www.slideshare.net/cloudera/hadoop-troubleshooting-101-kate-ting-cloudera