Can I pipe a io.BytesIO() stream to subprocess.popen() in Python?

bytesio, pipe, popen, python, subprocess

Solution

As @falsetru said you can't stream `BytesIO()` object directly; you need to get a bytestring from it first. It implies that all content should be already written to `stream` before you call `stream.getvalue()` to pass to `process.communicate()`.

If you want to stream instead of providing all input at once then you could drop `BytesIO()` object and write to the pipe directly:

from subprocess import Popen, PIPE

process = Popen(['command', 'arg1'], stdin=PIPE, bufsize=-1)
someStreamCreatingProcess(stream=process.stdin) # many `stream.write()` inside
process.stdin.close() # done (no more input)
process.wait()

`someStreamCreatingProcess()` should not return until it is done writing to the stream. If it returns immediately then it should call `stream.close()` at some point in the future (remove `process.stdin.close()` in your code):

from subprocess import Popen, PIPE

process = Popen(['command', 'arg1'], stdin=PIPE, bufsize=-1)
someStreamCreatingProcess(stream=process.stdin) # many `stream.write()` inside
process.wait() # stream.close() is called in `someStreamCreatingProcess`

Problem

I'm trying to pipe a io.BytesIO() bytetream to a separate program using subprocess.popen(), but I don't know how or if this is at all possible. Documentation and examples are all about text and newlines. When I whip up something like this: ``` import io from subprocess import * stream = io.BytesIO() someStreamCreatingProcess(stream) command = ['somecommand', 'some', 'arguments'] process = Popen(command, stdin=PIPE) process.communicate(input=stream) ``` I get ``` Traceback (most recent call last): File "./test.py", line 9, in <module> procOut = process.communicate(input=stream) File "/usr/lib/python2.7/subprocess.py", line 754, in communicate return self._communicate(input) File "/usr/lib/python2.7/subprocess.py", line 1322, in _communicate stdout, stderr = self._communicate_with_poll(input) File "/usr/lib/python2.7/subprocess.py", line 1384, in _communicate_with_poll chunk = input[input_offset : input_offset + _PIPE_BUF] TypeError: '_io.BytesIO' object has no attribute '__getitem__' ``` I think popen() is only for text. Am I wrong? Is there a different way to do this?

Original source