How to read a dataframe of encoded strings from csv in python

Question

Suppose I read an html website and I get a list of names, such as: 'Amiel, Henri-Frédéric'. In order to get the list of names I decode the html using the following code: ``` f = urllib.urlopen("http://xxx.htm") html = f.read() html=html.decode('utf8') t.feed(html) t.close() lista=t.data ``` At this point, the variable lista contains a list of names like: [u'Abatantuono, Diego', ... , u'Amiel, Henri-Frédéric'] Now I would like to: - put these names inside a DataFrame; - save the DataFrame in a csv file; - read the csv in Python through a DataFrame For simplicity, let's take in consideration just the above name to complete steps 1 to 3. I would use the following code: ``` name=u'Amiel, Henri-Fr\xe9d\xe9ric' name=name.encode('utf8') array=[name] df=pd.DataFrame({'Names':array}) df.to_csv('names') uni=pd.read_csv('names') uni #trying to read the csv file in a DataFrame ``` At this point i get the following error: ``` UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 67:…

Accepted Answer

Both`to_csv` method and `read_csv` function take an `encoding` argument. Use it. And work with unicode internally. If you don't, trying to encode/decode inside your program will get you.

```
import pandas as pd

name = u'Amiel, Henri-Fr\xe9d\xe9ric'
array = [name]
df = pd.DataFrame({'Names':array})
df.to_csv('names', encoding='utf-8')
uni = pd.read_csv('names', index_col = [0], encoding='utf-8')
print uni  # for me it works with or without print
```

out:

```
                   Names
0  Amiel, Henri-Frédéric
```

How to read a dataframe of encoded strings from csv in python

Solution

Problem