Python multiprocessing - Passing a list of dicts to a pool
multiprocessing, pool, python
Solution
You could use a tiny proxy function:
def proxy(Dict):
return compute(**Dict)
pool.map(proxy, args)
Or, since you don't need the `proxy` function polluting the namespace:
pool.map(lambda Dict: compute(**Dict), args)
Problem
This question may be a duplicate. However, I read lot of stuff around on this topic, and I didn't find one that matches my case - or at least, I didn't understood it. Sorry for the inconvenance. What I'm trying to do is fairly common, passing a list of kwargs to pool.starmap(), to achieve multiprocessing. Here's my reduced case: ``` def compute(firstArg, **kwargs): # A function that does things # Fancy computing stuff... # Saving to disk... return True if __name__ == '__main__': args = [{ 'firstArg': "some data", 'otherArg': "some other data" 'andAnotherArg': x*2 } for x in range(42)] pool = Pool(4) pool.starmap(compute, args) pool.close() pool.terminate() ``` I supposed starmap() will unpack the dict and pass it to compute() as keyword args, but looking at the source code (see also l.46), it sends only keys (or values ?). So it raises : ``` TypeError: compute() takes 1 positional argument but 3 were given ``` It must be a clear, straight forward way to do this... Any help would be appreciated. Here's a quite similar question : Python Multiprocessing - How to pass kwargs to function?