Post request with multipart/form-data in appengine python not working

django, google-app-engine, post, python

Solution

Here is some code I tested locally that should do the trick (I used a different handler than webapp2 but tried to modify it to webapp2. You'll also need the poster lib found here http://atlee.ca/software/poster/):

In your POST handler on GAE:

from google.appengine.api import urlfetch
from poster.encode import multipart_encode
payload = {}
payload['test_file'] = self.request.POST['test_file']
payload['user_id'] = self.request.POST['user_id']
to_post = multipart_encode(payload)
send_url = "http://127.0.0.1:8000/"
result = urlfetch.fetch(url=send_url, payload="".join(to_post[0]), method=urlfetch.POST, headers=to_post[1])
logging.info(result.content)

Make sure your HTML form contains `method="POST" enctype="multipart/form-data"`. Hope this helps!

EDIT: I tried using the webapp2 handler and realized the way files are served are different than how the framework I used to test with works (KAY). Here is updated code that should do the trick (tested on production):

import webapp2
from google.appengine.api import urlfetch
from poster.encode import multipart_encode, MultipartParam

class UploadTest(webapp2.RequestHandler):
  def post(self): 
    payload = {}
    file_data = self.request.POST['test_file']
    payload['test_file'] = MultipartParam('test_file', filename=file_data.filename,
                                          filetype=file_data.type,
                                          fileobj=file_data.file)
    payload['name'] = self.request.POST['name']
    data,headers= multipart_encode(payload)
    send_url = "http://127.0.0.1:8000/"
    t = urlfetch.fetch(url=send_url, payload="".join(data), method=urlfetch.POST, headers=headers)
    self.response.headers['Content-Type'] = 'text/plain'
    self.response.out.write(t.content)
  def get(self):
    self.response.out.write("""
    <html>
        <head>
            <title>File Upload Test</title>
        </head>
        <body>
            <form action="" method="POST" enctype="multipart/form-data">
                <input type="text" name="name" />
                <input type="file" name="test_file" />
                <input type="submit" value="Submit" />
            </form>
        </body>
    </html>""")

Problem

I'm attempting to send a multipart post request from an appengine app to an external (django) api hosted on dotcloud. The request includes some text and a file (pdf) and is sent using the following code ``` from google.appengine.api import urlfetch from poster.encode import multipart_encode from libs.poster.streaminghttp import register_openers register_openers() file_data = self.request.POST['file_to_upload'] the_file = file_data send_url = "http://127.0.0.1:8000/" values = { 'user_id' : '12341234', 'the_file' : the_file } data, headers = multipart_encode(values) headers['User-Agent'] = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)' data = str().join(data) result = urlfetch.fetch(url=send_url, payload=data, method=urlfetch.POST, headers=headers) logging.info(result.content) ``` When this method runs Appengine gives the following warning (I'm not sure if it's related to my issue) ``` Stripped prohibited headers from URLFetch request: ['Content-Length'] ``` And Django sends through the following error ``` <class 'django.utils.datastructures.MultiValueDictKeyError'>"Key 'the_file' not found in <MultiValueDict: {}>" ``` The django code is pretty simple and works when I use the postman chrome extension to send a file. ``` @csrf_exempt def index(request): try: user_id = request.POST["user_id"] the_file = request.FILES["the_file"] return HttpResponse("OK") except: return HttpResponse(sys.exc_info()) ``` If I add ``` print request.POST.keys() ``` I get a dictionary containing user_id and the_file indicating that the file is not being sent as a file. if I do the same for FILES i.e. ``` print request.FILES.keys() ``` I get en empty list []. EDIT 1: I've changed my question to implement the suggestion of someone1 however this still fails. I also included the headers addition recommended by the link Glenn sent, but no joy. EDIT 2: I've also tried sending the_file as variations of ``` the_file = file_data.file the_file = file_data.file.read() ``` But I get the same error. EDIT 3: I've also tried editing my django app to ``` the_file = request.POST["the_file"] ``` However when I try to save the file locally with ``` path = default_storage.save(file_location, ContentFile(the_file.read())) ``` it fails with ``` <type 'exceptions.AttributeError'>'unicode' object has no attribute 'read'<traceback object at 0x101f10098> ``` similarly if I try access the_file.file (as I can access in my appengine app) it tells me ``` <type 'exceptions.AttributeError'>'unicode' object has no attribute 'file'<traceback object at 0x101f06d40> ```

Original source