Python interface to PayPal - urllib.urlencode non-ASCII characters failing
paypal, python, unicode, urllib, urllib2
Solution
Try converting the params dictionary to utf-8 first... urlencode seems to like that better than unicode:
params = urllib.urlencode(dict([k, v.encode('utf-8')] for k, v in params.items()))
Of course, this assumes your input is unicode. If your input is something other than unicode, you'll want to decode it to unicode first, then encode it:
params['foo'] = my_raw_input.decode('iso-8859-1')
params = urllib.urlencode(dict([k, v.encode('utf-8')] for k, v in params.items()))
Problem
I am trying to implement PayPal IPN functionality. The basic protocol is as such: - The client is redirected from my site to PayPal's site to complete payment. He logs into his account, authorizes payment. - PayPal calls a page on my server passing in details as POST. Details include a person's name, address, and payment info etc. - I need to call a URL on PayPal's site internally from my processing page passing back all the params that were passed in abovem and an additional one called 'cmd' with a value of '_notify-validate'. When I try to urllib.urlencode the params which PayPal has sent to me, I get a: ``` While calling send_response_to_paypal. Traceback (most recent call last): File "<snip>/account/paypal/views.py", line 108, in process_paypal_ipn verify_result = send_response_to_paypal(params) File "<snip>/account/paypal/views.py", line 41, in send_response_to_paypal params = urllib.urlencode(params) File "/usr/local/lib/python2.6/urllib.py", line 1261, in urlencode v = quote_plus(str(v)) UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 9: ordinal not in range(128) ``` I understand that urlencode does ASCII encoding, and in certain cases, a user's contact info can contain non-ASCII characters. This is understandable. My question is, how do I encode non-ASCII characters for POSTing to a URL using urllib2.urlopen(req) (or other method) Details: I read the params in PayPal's original request as follows (the GET is for testing): ``` def read_ipn_params(request): if request.POST: params= request.POST.copy() if "ipn_auth" in request.GET: params["ipn_auth"]=request.GET["ipn_auth"] return params else: return request.GET.copy() ``` The code I use for sending back the request to PayPal from the processing page is: ``` def send_response_to_paypal(params): params['cmd']='_notify-validate' params = urllib.urlencode(params) req = urllib2.Request(PAYPAL_API_WEBSITE, params) req.add_header("Content-type", "application/x-www-form-urlencoded") response = urllib2.urlopen(req) status = response.read() if not status == "VERIFIED": logging.warn("PayPal cannot verify IPN responses: " + status) return False return True ``` Obviously, the problem only arises if someone's name or address or other field used for the PayPal payment does not fall into the ASCII range.