How do I close the stdout-pipe when killing a process started with python subprocess Popen?

kill, pipe, python, stdout, subprocess

Solution

I think the problem is that process.kill() only kills the immediate child process (bash), not the sub-processes of the bash script.

The problem and solution are described here:

- http://www.doughellmann.com/PyMOTW/subprocess/#process-groups-sessions

- How to terminate a python subprocess launched with shell=True

Use Popen(..., preexec_fn=os.setsid) to create a process group and os.pgkill to kill the entire process group. eg

import os
import signal
import subprocess
import time
from threading import Thread

process = None

def executeCommand(command, runCommand):
    Thread(target=runCommand, args=(command,)).start()

def runCommand(command):
    global process
    args = command.strip().split()
    process = subprocess.Popen(
        args, shell=False, stdout=subprocess.PIPE, preexec_fn=os.setsid)

    for line in process.communicate():
        if line:
            print "process:", line,

if __name__ == '__main__':
    executeCommand("./ascript.sh", runCommand)
    time.sleep(1)
    os.killpg(process.pid, signal.SIGKILL)

$ time python poc.py 
process: sleeping five

real    0m1.051s
user    0m0.032s
sys 0m0.020s

Problem

I wonder if it is possible to shut down the communication pipe when killing a subprocess started in a different thread. If I do not call communicate() then kill() will work as expected, terminating the process after one second instead of five. I found a discussion of a similar problem here, but I got no real answers. I assume that I either have to be able to close the pipe or to explicitly kill the sub-subprocess (which is "sleep" in the example) and kill that to unblock the pipe. I also tried to find the answer her on SO, but I only found this and this and this, which do not directly address this problem as far as I can tell (?). So the thing I want to do is to be able to run a command in a second thread and get all its output, but be able to kill it instantly when I so desire. I could go via a file and tail that or similar, but I think there should be a better way to do this? ``` import subprocess, time from threading import Thread process = None def executeCommand(command, runCommand): Thread(target=runCommand, args=(command,)).start() def runCommand(command): global process args = command.strip().split() process = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE) for line in process.communicate(): if line: print "process:", line, if __name__ == '__main__': executeCommand("./ascript.sh", runCommand) time.sleep(1) process.kill() ``` This is the script: ``` #!/bin/bash echo "sleeping five" sleep 5 echo "slept five" ``` Output ``` $ time python poc.py process: sleeping five real 0m5.053s user 0m0.044s sys 0m0.000s ```

Original source

Related problems