No such method exception Hadoop

Question

When I am running a Hadoop .jar file from the command prompt, it throws an exception saying no such method StockKey method.

StockKey is my custom class defined for my own type of key.

Here is the exception:

12/07/12 00:18:47 INFO mapred.JobClient: Task Id :   
attempt_201207082224_0007_m_000000_1, Status : FAILED

java.lang.RuntimeException: java.lang.NoSuchMethodException: SecondarySort$StockKey.      
<init>()
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
    at org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:109)
    at org.apache.hadoop.io.WritableComparator.<init>(WritableComparator.java:95)
    at org.apache.hadoop.io.WritableComparator.get(WritableComparator.java:51)
    at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:795)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:817)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:383)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at     
   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
    at org.apache.hadoop.mapred.Child.main(Child.java:264)

Answer 1

For scala too, I fixed the problem adding default constructor as below,

class IntPair (first : IntWritable, second : IntWritable) extends WritableComparable[IntPair] {

     def this() = this(first = new IntWritable(), second = new IntWritable())

     def getFirst () : IntWritable = {
         first
     }

     def getSecond () : IntWritable = {
         second
     }

}

Answer 2

No any answer helped me.

In my case it happened when me reduced constructor visibility by mistake or hurry.

E.g. parent constructor is public and inherited class's one is default or protected !

Answer 3

There's another thing to check when getting errors like this for classes which are writables, mappers, reducers, etc.

If the class is an inner class, make sure it's declared static (i.e. doesn't need an instance of the enclosing class). Otherwise, Hadoop cannot instantiate your inner class and will give this same error - that a zero-arg constructor is needed.

Answer 4

You have to provide an empty default constructor in your key class. Hadoop is using reflection and it can not guess any parameters to feed.

So just add the default constructor:

public StockKey(){}

Answer 5

Make sure you have the default constructor, but I also had to add the static keyword to my class declaration. That is,

public class SecondarySort {
  public static void main(String[] args) {...}

  public static class StockKey extends ... {}
}

Answer 6

Was facing this same issue. Fixed by following pointers from @Thomas and @Chris.

Looks like both these solutions are needed to solve the problem :

• Answer from @Thomas is required as Hadoop is using reflection and when building large projects.

• Answer from @Chris is required when using inner classes and invoking Mappers/Reducers from main().