UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte error in python while reading a csv file
byte-order-mark, python
Solution
Without seeing the binary content of the file it is difficult to guess the actual encoding but UTF-8, with or without a BOM (Byte order Marker) cannot start with an 0xFF.
If it starts with an 0xFF, then that would suggest that it is probably in Little Endian UTF-16 to UTF-32 which are the only Unicode serialisations that have a byte order marker starting with 0xFF.
https://en.wikipedia.org/wiki/Byte_order_mark gives some explanation.
It is also possible that it is a Persian specific character set. National character sets should be avoided if a Unicode option is available, for the generation of your source CSV files.
Problem
``` StopWords = pd.read_csv('stopwords.csv',encoding='UTF-8', quotechar='|',names=['StopWords']) ``` I am trying to read a CSV file that contains Persian language text, and this is the error I get: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte