Python unicode string operation in App Engine failing
character-encoding, google-app-engine, python-2.7
Solution
Put these 3 lines on the top of your Python 27 code to use unicode :
#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
# And this code will not give you any problems
s1 = 'É, Á, Ö, Ü. Ó,'
logging.info (s1)
s2 = s1 + ':' + s1
logging.info ("%s,%s", s1, s2)
And never user str(). Only if you realy need to!
And read this blogpost from Nick Johnson. This was before Python 27. He did not use the `from __future__ import unicode_literals` , which makes using unicode with Python so easy.
Problem
Python code that runs in Development/local machine, but fails after installing to Appengine : 1st line in my File : ``` # -*- coding: utf8 -*-O ``` Lines later in the code : ``` s1 = u'Ismerőseid' logging.info (s1) s2 = s1 + u':' + s1 logging.info (s2) logging.info ("%s,%s", s1, s2) ``` In Dev (localhost): ``` INFO 2012-12-18 04:01:17,926 AppRun.py:662] Ismerőseid, INFO 2012-12-18 04:01:17,926 AppRun.py:664] Ismerőseid:Ismerőseid INFO 2012-12-18 04:01:17,926 AppRun.py:665] Ismerőseid,Ismerőseid. Ó, ``` On App Engine after install/run : ``` I 2012-12-21 06:52:07.730 É, Á, Ö, Ü. Ó, E 2012-12-21 06:52:07.736 Traceback (most recent call last): File "....", line 672, in xxxx s3 = s1 + u':' + s1 UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128) ``` I have tried to various combination of encoding/decoding/etc.. I have also chardet on the pasted string 'Ismerőseid' and it gives me `{'confidence': 0.7402600692642154, 'encoding': 'ISO-8859-2'}` Any help is greatly appreciated!