Pandas read_csv expects wrong number of columns, with ragged csv file

Question

I have a csv file that has a few hundred rows and 26 columns, but the last few columns only have a value in a few rows and they are towards the middle or end of the file. When I try to read it in using read_csv() I get the following error. "ValueError: Expecting 23 columns, got 26 in row 64" I can't see where to explicitly state the number of columns in the file, or how it determines how many columns it thinks the file should have. The dump is below ``` In [3]: infile =open(easygui.fileopenbox(),"r") pledge = read_csv(infile,parse_dates='true') --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () 1 infile =open(easygui.fileopenbox(),"r") 2 ----> 3 pledge = read_csv(infile,parse_dates='true') C:\Python27\lib\site-packages\pandas-0.8.1-py2.7-win32.egg\pandas\io\parsers.pyc in read_csv(filepath_or_buffer, sep, dialect, header, index_col, names, skiprows, na_values, thousands,…

Accepted Answer

You can use `names` parameter. For example, if you have csv file like this:

```
1,2,1
2,3,4,2,3
1,2,3,3
1,2,3,4,5,6
```

And try to read it, you'll receive and error

```
>>> pd.read_csv(r'D:/Temp/tt.csv')
Traceback (most recent call last):
...
Expected 5 fields in line 4, saw 6
```

But if you pass `names` parameters, you'll get result:

```
>>> pd.read_csv(r'D:/Temp/tt.csv', names=list('abcdef'))
   a  b  c   d   e   f
0  1  2  1 NaN NaN NaN
1  2  3  4   2   3 NaN
2  1  2  3   3 NaN NaN
3  1  2  3   4   5   6
```

Hope it helps.

Pandas read_csv expects wrong number of columns, with ragged csv file

Solution

Problem

Related problems