Hadoop: No Such Method Exception

hadoop, mapreduce

Solution

Your mapper and reducer classes need to be defined static, otherwise the compile creates a constructor with a single argument (the parent `MaxTemperature` class). Hence there now is not a default constructor.

public static class MaxTemperatureMapper extends Mapper<....

public static class MaxTemperatureReducer extends Reducer<....

Problem

I have written a MapReduce program, code is below: ``` import java.io.IOException; import java.util.Iterator; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.*; import org.apache.hadoop.fs.Path; import org.apache.hadoop.conf.*; import org.apache.hadoop.io.*; import org.apache.hadoop.mapred.Reporter; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; public class MaxTemperature { public class MaxTemperatureMapper extends Mapper <LongWritable, Text, Text, IntWritable>{ private static final int MISSING = 9999; public void map(LongWritable key, Text Value, Context context, Reporter reporter) throws IOException, InterruptedException{ String line = Value.toString(); String year = line.substring(15,19); int airTemperature; if(line.charAt(87)=='+') airTemperature = Integer.parseInt(line.substring(88,92)); else airTemperature = Integer.parseInt(line.substring(87,91)); String quality = line.substring(92,93); if(airTemperature!=MISSING && quality.matches("[01459]")) context.write(new Text(year), new IntWritable(airTemperature)); } } public class MaxTemperatureReducer extends Reducer<Text, IntWritable, Text, IntWritable>{ public void reduce (Text Key, Iterator<IntWritable> Values, Context context, Reporter reporter) throws IOException, InterruptedException{ int maxValue = Integer.MIN_VALUE; while(Values.hasNext()) maxValue = Math.max(maxValue, Values.next().get()); context.write(Key, new IntWritable(maxValue)); } } public static void main(String[] args) throws Exception { if(args.length!=2){ System.err.println("Usage: WeatherTemperature <input path> <output path>"); System.exit(-1); } Configuration conf = new Configuration(); Job job = new Job(conf, "Maximum Temperature Calculator"); job.setJarByClass(MaxTemperature.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); job.setMapperClass(MaxTemperatureMapper.class); job.setReducerClass(MaxTemperatureReducer.class); // job.setInputFormatClass(TextInputFormat.class); //job.setOutputFormatClass(TextOutputFormat.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); } } ``` I run the .jar of this program using the following command: ``` hadoop jar weather.jar MaxTemperature input output ``` And I'm getting the following error: ``` 12/06/13 00:52:05 INFO mapred.JobClient: Task Id : attempt_201206121354_0007_m_000000_0, Status : FAILED java.lang.RuntimeException: java.lang.NoSuchMethodException: MaxTemperature$MaxTemperatureMapper.<init>() at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:602) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: java.lang.NoSuchMethodException: MaxTemperature$MaxTemperatureMapper.<init>() at java.lang.Class.getConstructor0(Class.java:2706) at java.lang.Class.getDeclaredConstructor(Class.java:1985) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109) ... 7 more ``` What does this error mean? And what am I doing wrong, how do I rectify it? Thanks!

Original source