Reading non-ASCII characters from a text file

character-encoding, python

Solution

- First of all - detect the file's encoding

  from chardet import detect
  encoding = lambda x: detect(x)['encoding']
  print encoding(line)

- then - convert it to unicode or your default encoding str:

  n_line=unicode(line,encoding(line),errors='ignore')
  print n_line
  print n_line.encode('utf8')

Problem

I'm using python 2.7. I've tried many things like codecs but didn't work. How can I fix this. myfile.txt ``` wörd ``` My code ``` f = open('myfile.txt','r') for line in f: print line f.close() ``` Output ``` s\xc3\xb6zc\xc3\xbck ``` Output is same on eclipse and command window. I'm using Win7. There is no problem with any characters when I don't read from a file.

Original source