Hadoop configuration: mapred.* vs mapreduce.*
hadoop, mapreduce
Solution
Examining the source for 0.20.2, there are only a few `mapreduce.*` properties, and they revolve around configuring the job input/output format, mapper/combiner/reducer and partitioner classes (they also signal to the job client that the new API is being used by the user - look through the source for `o.a.h.mapreduce.Job`, `setUseNewAPI()` method)
- `mapreduce.inputformat.class`
- `mapreduce.outputformat.class`
- `mapreduce.partitioner.class`
- `mapreduce.map.class`
- `mapreduce.combine.class`
- `mapreduce.reduce.class`
There are some more properties but they are secondary configuration
The input and output formats, whether it be new or old API versions, typically use `mapred.*` properties
For example, the signal your map reduce input paths you use `mapred.input.dir` (whether you're using the new or old API). Same for the output property `mapred.output.dir`
So the long and the short of if is, if there isn't a utility method to configure the property (`FileInputFormat.setInputPaths(Job, String)`) then you'll need to check the source
Problem
I noticed that there are two sets of Hadoop configuration parameters: one with mapred.* and the other with mapreduce.. I am guessing these might be due to old API vs. new API but if I am not mistaken, these seem to coexist in the new API. Am I correct? If so, is there a generalized statement what is used for mapred. and what is for mapreduce.*?