Difference between Threading and Map-Reduce processing?
mapreduce, multithreading
Solution
You can find related paper for this, Comparing Fork/Join and MapReduce.
The paper compares the performance, scalability and programmability of three parallel paradigms: fork/join, MapReduce, and a hybrid approach.
What they find is basically that Java fork/join has low startup latency and scales well for small inputs (<5MB), but it cannot process larger inputs due to the size restrictions of shared-memory, single node architectures. On the other hand, MapReduce has significant startup latency (tens of seconds), but scales well for much larger inputs (>100MB) on a compute cluster.
Threading offers facilities to partition a task into several subtasks, in a recursive-looking fashion; more tiers, possibility of 'inter-fork' communication at this stage, much more traditional programming. Does not extend (at least in the paper) beyond a single machine. Great for taking advantage of your eight-core.
M-R only does one big split, with the mapped splits not talking between each other at all, and then reduces everything together. A single tier, no inter-split communication until reduce, and massively scalable. Great for taking advantage of your share of the cloud.
Problem
One of my collegue is arguing with me for introducing map-reduce concept in our application(text processing). His opinion is why we should not use threading concepts instead.We both are new to this map-reduce paradigm. I thought that using map-reduce concept helps the developer from the overhead of handling thread synchronisation,dead lock,shared data. Is there anything other than this for going to map-reduce concept rather than threading?