Hadoop 2.0 JAR files

问题:

I am getting an unexpected error when trying to instantiate a Configuration object in my Map/Reduce program.

The error is as follows:

java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory

I am using Hadoop 2.2.0. The program worked fine with an earlier version of Hadoop. However, now I need to write this job submitter for the version 2, and this does not seem to be working.

As a side note, I could not find all the required JAR files for this version. I have read the following solution to a similar problem: Where are hadoop jar files in hadoop 2?. However, this did not work for me. It fails with the above runtime error.

I tried using older JAR files to resolve these issues, but new error keep occurring.

Does anyone have a solution to this problem? I am using NetBeans, if that makes any difference from some reason.

回答1:

Including JARs in folders listed below worked for me: share\hadoop\common
share\hadoop\common\lib
share\hadoop\mapreduce
share\hadoop\mapreduce\lib share\hadoop\yarn
share\hadoop\yarn\lib

回答2:

I resolved the problem by downloading the JAR files from commons.apache directory and from Maven repositories. Each time I added a new JAR to the project, the application crashed with a different error (a different class was not found) until I downloaded all of the JAR files from the Maven repository. After downloading a lot of JAR files from the online repositories, most of the JAR related problems seem to be resolved.

回答3:

you can try right-clicking the Project, ->Build Path -> Configure Build Path

Go to your src folder, point to "Native Library", then edit the location to the location of your hadoop native library folder (normally: ~/hadoop-x.x.x/lib/native/"folder-depending-on-your-system")