Scrapy FormRequest sending JSON

json, python, scrapy

Solution

FormRequest is for simulating an HTML form (e.g. application/x-www-form-urlencoded). It sounds like you are simply wanting to POST data with your Request. Since you mention a content type of 'application/json' you probably want to do something like this:

request = Request( url, method='POST', 
                   body=json.dumps(my_data), 
                   headers={'Content-Type':'application/json'} )

Problem

I am trying to create a FormRequest that can send content-type:application/json. Here is what I try: ``` yield FormRequest("abc.someurl.com", formdata=json.dumps({"referenceId":123,"referenceType":456}), headers={'content-type':'application/json'}, callback=self.parseResult2) ``` If I use `json.dumps()` to process the form data in the formdata=, the error I get is "exceptions.ValueError: need more than 1 value to unpack" I can't just use the list as in ``` formdata={"referenceId":123,"referenceType":456} ``` The FormRequest works but is not accepted by the server. ``` import requests import json result = requests.post(url, json.dumps({"referenceId":123,"referenceType":456}), headers={'content-type':'application/json'}) ``` It works from the python command prompt as in the above. Any ideas? -KM

Original source