Python - HTTP multipart/form-data POST request

python, python-3.x

Solution

I had a look at this module

class HTTPConnection:
    # ...
    def send(self, data): # line 820
        """Send `data' to the server.
        ``data`` can be a string object, a bytes object, an array object, a
        file-like object that supports a .read() method, or an iterable object.
        """

data is exactly body. You may pass an iterator like this: (I did not try it out)

def body():
  for fileName in fileList:
    # Add boundary and header
    yield('--' + boundary) + '\r\n'
    yield('Content-Disposition: form-data; name={0}; filename=    {0}'.format(fileName)) + '\r\n'

    fileType = mimetypes.guess_type(fileName)[0] or 'application/octet-stream'
    yield('Content-Type: {}'.format(fileType)) + '\r\n'
    yield('\r\n')

    with open(fileName) as f: 
        # Bad for large files
        yield f.read()
    yield('--'+boundary+'--') + '\r\n'
    yield('') + '\r\n'

Problem

I would like to upload a file to a web server. From what I have read, the best way to do this is to use the multipart/form-data encoding type on an HTTP POST request. My research seems to indicate that there is no simple way to do this using the Python standard library. I am using Python 3. (Note: see a package called requests (PyPI Link) to easily accomplish this) I am currently using this method: ``` import mimetypes, http.client boundary = 'wL36Yn8afVp8Ag7AmP8qZ0SA4n1v9T' # Randomly generated for fileName in fileList: # Add boundary and header dataList.append('--' + boundary) dataList.append('Content-Disposition: form-data; name={0}; filename={0}'.format(fileName)) fileType = mimetypes.guess_type(fileName)[0] or 'application/octet-stream' dataList.append('Content-Type: {}'.format(fileType)) dataList.append('') with open(fileName) as f: # Bad for large files dataList.append(f.read()) dataList.append('--'+boundary+'--') dataList.append('') contentType = 'multipart/form-data; boundary={}'.format(boundary) body = '\r\n'.join(dataList) headers = {'Content-type': contentType} conn = http.client.HTTPConnection('http://...') req = conn.request('POST', '/test/', body, headers) print(conn.getresponse().read()) ``` This works to send text. There are two issues: This is text only, and the whole text file must be stored in memory as a giant string. How can I upload any binary file? Is there a way to do this without reading the whole file into memory?

Original source