Java equivalent for Python pool.map/ Multiprocessing
concurrency, java, multiprocessing, parallel-processing, python
Solution
There's no exactly-compatible class, but `ExecutorService` gives you everything you need to implement it.
In particular, there's no function to map a `Callable` over a `Collection` and wait on the results, but you can easily build a `Collection<Callable<T>>` out of a `Callable<T>` and `Collection<T>`, then just call `invokeAll`, which returns you a `List<Future<T>>`.
(If you want to emulate some of the other functions from `multiprocessing.Pool`, you will need to loop around `submit` instead and build your own collection of things to wait on. But `map` is simple.)
Problem
I was wondering if somebody could point me to a simple equivalent of python's multiprocessing module in java. I have a simple parallel processing scenario (where no 2 processes interact): Take a data set and split it into 12 and apply a java method to the 12 datasets, collect results and join them in a list of some sort with the same ordering. Java being a "pro" language appears to have multiple libraries and methods - anyone who can help this java newbie get started? I would like to do this with minimal of coding - as i said my requirement is pretty straightforward. Update: how to do multiprocessing in java, and what speed gains to expect? This seems to indicate threads is the way to go. I expect I have no choice but wade into a bunch of locks (pun unintended) and wait for my ship to sail. Simple examples are welcome nevertheless.