django-pyodbc: MSSQL unicode issue
django, django-pyodbc, freetds, odbc, sql-server
Solution
I would also recommend including the following in options, to ensure Unicode results:
'OPTIONS': {
'host_is_server': True,
'autocommit': True,
'unicode_results': True,
'extra_params': 'tds_version=8.0'
},
The autocommit option is also necessary for Django 1.6. When you connect in this manner, it bypasses any DSNs you may have set up.
Edit: Django’s documentation for v1.6 reports “Since Django 1.6, autocommit is turned on by default”, so I guess you don't need to set this
Problem
I'm having problems logging in users on Django using `django-pyodbc`, `freetds` and `unixodbc`. I can run `syncdb` and connect with `pyodbc` without any problems. The full error that I get is: ``` ('42000', '[42000] [FreeTDS][SQL Server]Unicode data in a Unicode-only collation or ntext data cannot be sent to clients using DB-Library (such as ISQL) or ODBC version 3.7 or earlier. (4004) (SQLExecDirectW) ``` freetds conf file: ``` [PS123] host = <ip number> port = 2254 tds version = 8.0 instance = T_INS01 client charset = UTF-8 # somebody suggested this had to be global so I put it here as well [global] client charset = UTF-8 tds version = 8.0 ``` odbcinst.ini ``` [ODBC] Trace = Yes TraceFile = /tmp/odbc.log [FreeTDS] Description = TDS driver (Sybase/MS SQL) Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so UsageCount = 1 tds_version = 8.0 # several people suggested using underscore ``` odbc.ini ``` [PS123] Driver = FreeTDS Description = A wonderful description goes here tds_version = 8.0 # also tried without underscore here Database = database Servername = PS123 Trace = Yes TraceFile = /tmp/freetdssql.log ``` If I log in the user through `django-debug-toolbar's debugsqlshell`, I can see these are the queries that happens before the exception: ``` In [10]: login(request, user) SELECT TOP 1 (1) AS [a] FROM [django_session] WHERE [django_session].[session_key] = magyul563p13z6e33t6rexesaxx1kszx [5.03ms] INSERT INTO [django_session] ([session_key], [session_data], [expire_date]) VALUES (magyul563p13z6e33t6rexesaxx1kszx, ZjgwN2E1NGZhNTE4YTI2ZWQxMDM3M2ZlZThiNWVlY2NlYTlmZWQ1YzqAAn1xAS4=, 2014-02-14 10:20:31+00:00) [3.23ms] SELECT [django_session].[session_key], [django_session].[session_data], [django_session].[expire_date] FROM [django_session] WHERE [django_session].[session_key] = magyul563p13z6e33t6rexesaxx1kszx [1.56ms] ``` But when I run those queries using only `pyodbc` like below: ``` import pyodbc cnx = pyodbc.connect("DSN=PS123;UID=username;PWD=password") cursor = cnx.cursor() cursor.execute("SELECT TOP 1 (1) AS [a] FROM [django_session] WHERE [django_session].[session_key] = 'azozj51b9a5y9lnbq2b4hydhrryrplpz'") cursor.execute("INSERT INTO [django_session] ([session_key], [session_data], [expire_date]) VALUES ('azozj51b9a5y9lnbq2b4hyd', 'ZwN2E1NGZhNTE4YTI2ZWQxMDM3M2ZlZThiNWVlY2NlYTlmZWQ1YzqAAn1x', '2014-02-15')") cursor.execute("SELECT [django_session].[session_key], [django_session].[session_data], [django_session].[expire_date] FROM [django_session] WHERE [django_session].[session_key] = 'azozj51b9a5y9lnbq2b4hyd'") ``` It fetches and inserts without any problem. The only issue I had was that I had to remove the time and timezone in the date string. I am assuming this is unrelated to my problem. These are my Django database settings: ``` DATABASES = { 'default': { 'ENGINE': 'sql_server.pyodbc', # also tried django_pyodbc 'NAME': 'database', 'USER': 'username', 'PASSWORD': 'password', 'HOST': 'PS123', 'PORT': '2254', 'OPTIONS': { 'host_is_server': True }, } } ``` Full traceback: ``` Traceback: File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response 115. response = callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/sites.py" in wrapper 219. return self.admin_view(view, cacheable)(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view 91. response = view_func(request, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/cache.py" in _wrapped_view_func 89. response = view_func(request, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/sites.py" in inner 196. if not self.has_permission(request): File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/sites.py" in has_permission 149. return request.user.is_active and request.user.is_staff File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py" in inner 204. self._setup() File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py" in _setup 270. self._wrapped = self._setupfunc() File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/middleware.py" in <lambda> 18. request.user = SimpleLazyObject(lambda: get_user(request)) File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/middleware.py" in get_user 10. request._cached_user = auth.get_user(request) File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/__init__.py" in get_user 136. user_id = request.session[SESSION_KEY] File "/usr/local/lib/python2.7/dist-packages/django/contrib/sessions/backends/base.py" in __getitem__ 44. return self._session[key] File "/usr/local/lib/python2.7/dist-packages/django/contrib/sessions/backends/base.py" in _get_session 167. self._session_cache = self.load() File "/usr/local/lib/python2.7/dist-packages/django/contrib/sessions/backends/db.py" in load 18. expire_date__gt=timezone.now() File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py" in get 143. return self.get_query_set().get(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py" in get 398. num = len(clone) File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py" in __len__ 106. self._result_cache = list(self.iterator()) File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py" in iterator 317. for row in compiler.results_iter(): File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py" in results_iter 775. for rows in self.execute_sql(MULTI): File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py" in execute_sql 846. cursor.execute(sql, params) File "/usr/local/lib/python2.7/dist-packages/django/db/backends/util.py" in execute 41. return self.cursor.execute(sql, params) File "/usr/local/lib/python2.7/dist-packages/sql_server/pyodbc/base.py" in execute 396. raise utils.DatabaseError(*e.args) Exception Type: DatabaseError at /admin/ Exception Value: ('42000', '[42000] [FreeTDS][SQL Server]Unicode data in a Unicode-only collation or ntext data cannot be sent to clients using DB-Library (such as ISQL) or ODBC version 3.7 or earlier. (4004) (SQLExecDirectW)') ```