CDH4 - Exception: java.lang.IncompatibleClassChang

Question

I am getting a java issue when I launch a pig script, it appears to be some dependency or version conflict, Running Debian/Cloudera CDH4/ Apache Pig

java.lang.Exception: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.Counter, but class was expected

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:406)
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.Counter, but class was expected

Answer 1

The IncomaptibleClassChangeError is almost always caused when your code is compiled against one version of Hadoop and are running against a different version.

Please check to make sure that you compiled your code against the same version of Hadoop as the version that you're running.

Answer 2

If you are using Maven, you need to make changes in the POM File.

Earlier I was using the following dependency:

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>1.2.1</version>
</dependency>

When I changed it to:

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-client</artifactId>
    <version>2.5.2</version>
</dependency>

I think this might help. The reason, according to me, is that the later version of Hadoop treats Counter as Interface and not a class. So we cannot create an object of the Counter.