Profiling a python multiprocessing pool
multiprocessing, pool, profiling, python
Solution
Try this:
def square_wrapper(i):
result = [None]
cProfile.runctx("result[0] = square(i)", globals(), locals(), "file_%d" % i)
return result[0]
Problem
I'm trying to run cProfile.runctx() on each process in a multiprocessing pool, to get an idea of what the multiprocessing bottlenecks are in my source. Here is a simplified example of what I'm trying to do: ``` from multiprocessing import Pool import cProfile def square(i): return i*i def square_wrapper(i): cProfile.runctx("result = square(i)", globals(), locals(), "file_"+str(i)) # NameError happens here - 'result' is not defined. return result if __name__ == "__main__": pool = Pool(8) results = pool.map_async(square_wrapper, range(15)).get(99999) print results ``` Unfortunately, trying to execute "result = square(i)" in the profiler does not affect 'result' in the scope it was called from. How can I accomplish what I am trying to do here?