how to open a csv in universal new line mode through django upload?

csv, django, django-forms, python

Solution

You can use `str.splitlines()` -- which automatically splits on universale line-breaks -- in the following manner:

def clean(self):
    file_csv = self.cleaned_data['csv_file']
    lines = file_csv.read().splitlines()
    records = csv.reader(lines, dialect=csv.excel_tab)

If you are worried about the memory cost of creating the `lines` variable, you can force Django to save the file to a local file on disk changing the `FILE_UPLOAD_MAX_MEMORY_SIZE` variable in settings.py (more on this variable here):

# add to your settings.py
FILE_UPLOAD_MAX_MEMORY_SIZE = 0
FILE_UPLOAD_TEMP_DIR = '/tmp'

Then to process the file from it's tmp folder using universal mode:

def clean(self):
    file_csv = open(self.cleaned_data['csv_file'].temporary_file_path, 'rU')
    records = csv.reader(file_csv, dialect=csv.excel_tab)

Problem

I am trying to upload a csv file in a django form: ``` class CSVUploadForm(forms.Form): csv_file = forms.FileField(label='Select a CSV file to import:',) def clean(self): file_csv = self.cleaned_data['csv_file'] records = csv.reader(open('/mypath/'+file_csv.name, 'rU'), dialect=csv.excel_tab) ``` I need to open the file in universal new line mode. I can do that with "open" method above, but that will not work for this form because the file I am dealing with is an in memory uploaded version of the csv. How do I pass the universal new line mode flag rU to something like this: ``` records = csv.reader(file_csv, dialect=csv.excel_tab) ``` ?

Original source