which encoding does the python lxml module use internally?
encoding, lxml, python
Solution
`lxml.etree._ElementUnicodeResult` is a class that inherits from `unicode`:
$ pydoc lxml.etree._ElementUnicodeResult
lxml.etree._ElementUnicodeResult = class _ElementUnicodeResult(__builtin__.unicode)
| Method resolution order:
| _ElementUnicodeResult
| __builtin__.unicode
| __builtin__.basestring
| __builtin__.object
In Python, it's fairly common to have classes that extend from base types to add some module-specific functionality. It should be safe to treat the object like a regular Unicode string.
Problem
When I get a webpage, I use UnicodeDammit to convert it to utf-8 encoding, just like: ``` import chardet from lxml import html content = urllib2.urlopen(url).read() encoding = chardet.detect(content)['encoding'] if encoding != 'utf-8': content = content.decode(encoding, 'replace').encode('utf-8') doc = html.fromstring(content, base_url=url) ``` but when I use: ``` text = doc.text_content() print type(text) ``` The output is `<type 'lxml.etree._ElementUnicodeResult'>`. why? I thought it would be a utf-8 string.