Callback with multiprocessing.pools.map_async not working?

python

Solution

Ok I took a chance and logged a bug for it. Turns out it is actually a bug in 3.3 and a patch is in progress.

http://bugs.python.org/issue16307

Problem

I can't seem to get my callbacks to work when using `map_async()`. It works when I use slightly modified code to loop over my array adding tasks via `apply_async()` instead. From the documentation it seems I should be able to use the callbacks with `map_async()`, but maybe it's some kind of newb mistake... ``` from multiprocessing import Pool,TimeoutError from time import sleep servers=["s1","s2","s3","s4","s5","s6"] def f(x): print("start f(" + x + ")") sleep(5) print("end f(" + x + ")") return "did " + x def mycallback(x): print("My callback " + str(x)) def myerrorcallback(r): print("My errorcallback " + str(r)) if __name__ == '__main__': pool = Pool(processes=4) results = pool.map_async(f, servers, chunksize=1, callback=mycallback, error_callback=myerrorcallback) print(results.get(timeout=11)) ``` When run I get: ``` D:\python> f.py start f(s1) start f(s2) start f(s3) start f(s4) end f(s1) start f(s5) end f(s2) start f(s6) end f(s4) end f(s3) end f(s5) end f(s6) ['did s1', 'did s2', 'did s3', 'did s4', 'did s5', 'did s6'] ``` When I use modified code with `apply_async()` instead I get the print output from the callbacks. Modified code is to just change the last part to: ``` if __name__ == '__main__': pool = Pool(processes=4) for server in servers: pool.apply_async(f, (server,), callback=mycallback, error_callback=myerrorcallback) pool.close() pool.join() ``` Results in: ``` D:\python\>fb.py start f(s1) start f(s2) start f(s3) start f(s4) end f(s1) start f(s5) My callback did s1 end f(s2) My callback did s2 start f(s6) end f(s3) My callback did s3 end f(s4) My callback did s4 end f(s5) My callback did s5 end f(s6) My callback did s6 ```

Original source