Strange problems when using requests and multiprocessing
kill, linux, multiprocessing, python, python-requests
Solution
Requests async module uses gevent. If you look at the source code of gevent you will see that it monkey patches many of Python's standard library functions, including sleep:
request.async module during import executes:
from gevent import monkey as curious_george
# Monkey-patch.
curious_george.patch_all(thread=False, select=False)
Looking at the monkey.py module of gevent you can see:
https://bitbucket.org/denis/gevent/src/f838056c793d/gevent/monkey.py#cl-128
def patch_time():
"""Replace :func:`time.sleep` with :func:`gevent.sleep`."""
from gevent.hub import sleep
import time
patch_item(time, 'sleep', sleep)
Take a look at the code from the gevent's repository for details.
Problem
Please check this python code: ``` #!/usr/bin/env python import requests import multiprocessing from time import sleep, time from requests import async def do_req(): r = requests.get("http://w3c.org/") def do_sth(): while True: sleep(10) if __name__ == '__main__': do_req() multiprocessing.Process( target=do_sth, args=() ).start() ``` When I press Ctrl-C (wait 2sec after run - let Process run), it doesn't stop. When I change the import order to: ``` from requests import async from time import sleep, time ``` it stops after Ctrl-C. Why it doesn't stop/kill in first example? It's a bug or a feature? Notes: - Yes I know, that I didn't use async in this code, this is just stripped down code. In real code I use it. I did it to simplify my question. - After pressing Ctrl-C there is a new (child) process running. Why? - `multiprocessing.__version__ == 0.70a1`, `requests.__version__ == 0.11.2`, `gevent.__version__ == 0.13.7`