pandas.read_csv: how to skip comment lines
pandas, python
Solution
So I believe in the latest releases of pandas (version 0.16.0), you could throw in the `comment='#'` parameter into `pd.read_csv` and this should skip commented out lines.
These github issues shows that you can do this:
- https://github.com/pydata/pandas/issues/10548
- https://github.com/pydata/pandas/issues/4623
See the documentation on `read_csv`: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html
Problem
I think I misunderstand the intention of read_csv. If I have a file 'j' like ``` # notes a,b,c # more notes 1,2,3 ``` How can I pandas.read_csv this file, skipping any '#' commented lines? I see in the help 'comment' of lines is not supported but it indicates an empty line should be returned. I see an error ``` df = pandas.read_csv('j', comment='#') ``` CParserError: Error tokenizing data. C error: Expected 1 fields in line 2, saw 3 I'm currently on ``` In [15]: pandas.__version__ Out[15]: '0.12.0rc1' ``` On version'0.12.0-199-g4c8ad82': ``` In [43]: df = pandas.read_csv('j', comment='#', header=None) ``` CParserError: Error tokenizing data. C error: Expected 1 fields in line 2, saw 3