Getting realtime output using subprocess

Question

I am trying to write a wrapper script for a command line program (svnadmin verify) that will display a nice progress indicator for the operation. This requires me to be able to see each line of output from the wrapped program as soon as it is output. I figured that I'd just execute the program using `subprocess.Popen`, use `stdout=PIPE`, then read each line as it came in and act on it accordingly. However, when I ran the following code, the output appeared to be buffered somewhere, causing it to appear in two chunks, lines 1 through 332, then 333 through 439 (the last line of output) ``` from subprocess import Popen, PIPE, STDOUT p = Popen('svnadmin verify /var/svn/repos/config', stdout = PIPE, stderr = STDOUT, shell = True) for line in p.stdout: print line.replace('
', '') ``` After looking at the documentation on subprocess a little, I discovered the `bufsize` parameter to `Popen`, so I tried setting bufsize to 1 (buffer each line) and 0 (no buffer), but neither value seemed to cha…

Accepted Answer

I tried this, and for some reason while the code

```
for line in p.stdout:
  ...
```

buffers aggressively, the variant

```
while True:
  line = p.stdout.readline()
  if not line: break
  ...
```

does not. Apparently this is a known bug: http://bugs.python.org/issue3907 (The issue is now "Closed" as of Aug 29, 2018)

Getting realtime output using subprocess

Solution

Problem

Related problems