How to convert tab separated, pipe separated to CSV file format in Python

csv, python

Solution

I fear that you can't identify the delimiter without knowing what it is. The problem with CSV is, that, quoting ESR:

the Microsoft version of CSV is a textbook example of how not to design a textual file format.

The delimiter needs to be escaped in some way if it can appear in fields. Without knowing, how the escaping is done, automatically identifying it is difficult. Escaping could be done the UNIX way, using a backslash '\', or the Microsoft way, using quotes which then must be escaped, too. This is not a trivial task.

So my suggestion is to get full documentation from whoever generates the file you want to convert. Then you can use one of the approaches suggested in the other answers or some variant.

Edit:

Python provides csv.Sniffer that can help you deduce the format of your DSV. If your input looks like this (note the quoted delimiter in the first field of the second row):

a|b|c
"a|b"|c|d
foo|"bar|baz"|qux

You can do this:

import csv

csvfile = open("csvfile.csv")
dialect = csv.Sniffer().sniff(csvfile.read(1024))
csvfile.seek(0)

reader = csv.DictReader(csvfile, dialect=dialect)
for row in reader:
    print row,
# => {'a': 'a|b', 'c': 'd', 'b': 'c'} {'a': 'foo', 'c': 'qux', 'b': 'bar|baz'}
# write records using other dialect

Problem

I have a text file (.txt) which could be in tab separated format or pipe separated format, and I need to convert it into CSV file format. I am using python 2.6. Can any one suggest me how to identify the delimiter in a text file, read the data and then convert that into comma separated file. Thanks in advance

Original source