Segfault in multithreaded python extension in C
multithreading, python
Solution
I'm not positive that this will be relevant to your question, but one thing that looks suspicious is the PyEval_ReleaseLock() call in your module initializer function. I doubt that Python expects your module initializer to release the GIL out from underneath it, and a quick look at some example code here doesn't show anything along these lines. Could you try removing that PyEval_ReleaseLock() call and tell us what happens?
BTW, I agree that the PyGILState_*() calls within run_thread() should have no effect; you should be able to just remove them.
Problem
I have a very trimmed down example that creates a segfault that I can't seem to get rid of. Python script calls a C function in an extension, which creates a new thread using pthreads. I use PyGILState_Ensure and PyGILState_Release around my python call (PyRun_SimpleString) in the new thread, but perhaps I'm not using them correctly or have missed some other step. Commenting out the python calls in the receive_audio function, the segfault no longer happens. Any ideas? Output: python lib/test.py (Main Thread) initmodule complete (Main Thread) Calling run_thread() (Main Thread) Creating thread (New Thread) In receive_audio() - acquiring GIL (New Thread) python print! (New Thread) In receive_audio() - released GIL (New Thread) Looping 0 Segmentation fault C Code as follows: ``` PyMODINIT_FUNC streamaudio() { PyObject *m = Py_InitModule("streamaudio", methods); PyEval_InitThreads(); mainThreadState = PyThreadState_Get(); PyEval_ReleaseLock(); printf("(Main Thread) initmodule complete\n"); } static PyObject* run_thread(PyObject* self, PyObject* args) { int ok, stream_id; PyGILState_STATE gstate; gstate = PyGILState_Ensure(); ok = PyArg_ParseTuple(args, "i", &stream_id); PyRun_SimpleString("print '(Main Thread) Creating thread'\n"); int rc = pthread_create(&thread, NULL, receive_audio, (void*)stream_id); PyRun_SimpleString("print '(Main Thread) Thread created'\n"); PyGILState_Release(gstate); return Py_BuildValue("i", rc); } void* receive_audio(void *x) { printf("(New Thread) In receive_audio() - acquiring GIL\n"); PyGILState_STATE gstate; gstate = PyGILState_Ensure(); PyRun_SimpleString("print '(New Thread) python print!'\n"); PyGILState_Release(gstate); printf("(New Thread) In receive_audio() - released GIL\n"); int i; for (i = 0; i < 100; i++) { printf("(New Thread) Looping %d\n", i); sleep(1); } } ```