How to resolve "iterator should return strings, not bytes"

Question

I am trying to import a CSV file, using a form to upload the file from the client system. After I have the file, I'll take parts of it and populate a model in my app. However, I'm getting an "iterator should return strings, not bytes" error when I go to iterate over the lines in the uploaded file. I've spent hours trying different things and reading everything I could find on this but can't seem resolve it (note, I'm relatively new to Django- running 1.5- and python - running 3.3). I stripped out things to get to just the error and ran it like this to make sure it is still there. The error is displayed when executing the line "for clubs in club_list" in tools_clubs_import(): The following is the corrected views.py that works, based on answer marked below: ``` import csv from io import TextIOWrapper from django.shortcuts import render from django.http import HttpResponseRedirect from django.core.urlresolvers import reverse from rank.forms import ClubImportForm def tools_clubs_import(re…

Accepted Answer

`request.FILES` gives you binary files, but the `csv` module wants to have text-mode files instead.

You need to wrap the file in a `io.TextIOWrapper()` instance, and you need to figure out the encoding:

```
from io import TextIOWrapper

f = TextIOWrapper(request.FILES['filename'].file, encoding=request.encoding)
```

It'd probably be better if you took the `charset` parameter from the `Content-Type` header if provided; that is what the client tells you the character set is.

You cannot work around needing to know the correct encoding for the file data; you can force interpretation as ASCII, for example, by providing a `errors` keyword as well (setting it to 'replace' or 'ignore'), but that does lead to data loss:

```
f = TextIOWrapper(request.FILES['filename'].file, encoding='ascii', errors='replace')
```

Using TextIOWrapper will only work when using Django 1.11 and later (as this changeset added the required support). In earlier versions, you can monkey-patch the support in after the fact:

```
from django.core.files.utils import FileProxyMixin

if not hasattr(FileProxyMixin, 'readable'):
    # Pre-Django 1.11, add io.IOBase support, see
    # https://github.com/django/django/commit/4f474607de9b470f977a734bdd47590ab202e778        
    def readable(self):
        if self.closed:
            return False
        if hasattr(self.file, 'readable'):
            return self.file.readable()
        return True

def writable(self):
        if self.closed:
            return False
        if hasattr(self.file, 'writable'):
            return self.file.writable()
        return 'w' in getattr(self.file, 'mode', '')

def seekable(self):
        if self.closed:
            return False
        if hasattr(self.file, 'seekable'):
            return self.file.seekable()
        return True

FileProxyMixin.closed = property(
        lambda self: not self.file or self.file.closed)
    FileProxyMixin.readable = readable
    FileProxyMixin.writable = writable
    FileProxyMixin.seekable = seekable
```

How to resolve "iterator should return strings, not bytes"

Solution

Problem