Very simple concurrent programming in Python

concurrency, python

Solution

import multiprocessing

import Foo
import Bar

results = {}

def get_a():
    results['a'] = Foo.get_something()



def get_b():
    results['b'] = Bar.get_something_else()

process_a = multiprocessing.Process(target=get_a)
process_b = multiprocessing.Process(target=get_b)


process_b.start()
process_a.start()


process_a.join
process_b.join

Here is the process version of your program.

NOTE: that in threading there are shared datastructures so you have to worry about locking which avoids wrong manipulation of data plus as amber mentioned above it also has a GIL (Global interpreter Lock) problem and since both of your tasks are CPU intensive then this means that it will take more time because of the calls notifying the threads of thread acquisition and release. If however your tasks were I/O intensive then it does not effect that much.

Now since there are no shared datastructures in a process thus no worrying about LOCKS and since it works irrespective of the GIL so you actually enjoy the real power of multiprocessors.

Simple note to remember: process is the same as thread just without using a shared datastructures (everything works in isolation and is focused on messaging.)

check out https://www.dabeaz.com/tutorials.html he gave a good presentation on concurrent programming once.

Problem

I have a simple Python script that uses two much more complicated Python scripts, and does something with the results. I have two modules, Foo and Bar, and my code is like the following: ``` import Foo import Bar output = [] a = Foo.get_something() b = Bar.get_something_else() output.append(a) output.append(b) ``` Both methods take a long time to run, and neither depends on the other, so the obvious solution is to run them in parallel. How can I achieve this, but make sure that the order is maintained: Whichever one finishes first must wait for the other one to finish before the script can continue. Let me know if I haven't made myself clear enough, I've tried to make the example code as simple as possible.

Original source