Python: Replace typographical quotes, dashes, etc. with their ascii counterparts

python, string

Solution

What about this? It creates translation table first, but honestly I don't think you can do this without it.

transl_table = dict( [ (ord(x), ord(y)) for x,y in zip( u"‘’´“”–-",  u"'''\"\"--") ] ) 

with open( "a.txt", "w", encoding = "utf-8" ) as f_out : 
    a_str = u" ´funny single quotes´ long–-and–-short dashes ‘nice single quotes’ “nice double quotes”   "
    print( " a_str = " + a_str, file = f_out )

    fixed_str = a_str.translate( transl_table )
    print( " fixed_str = " + fixed_str, file = f_out  )

I wasn't able to run this printing to a console (on Windows) so I had to write to txt file. The output in the a.txt file looks as follows:

a_str = ´funny single quotes´ long–-and–-short dashes ‘nice single quotes’ “nice double quotes” fixed_str = 'funny single quotes' long--and--short dashes 'nice single quotes' "nice double quotes"

By the way, the code above works in Python 3. If you need it for Python 2, it might need some fixes due to the difference in handling Unicode strings in both versions of the language

Problem

On my website people can post news and quite a few editors use MS word and similar tools to write the text and then copy&paste into my site's editor (simple textarea, no WYSIWYG etc.). Those texts usually contain "nice" quotes instead of the plain ascii ones (`"`). They also sometimes contain those longer dashes like `–` instead of `-`. Now I want to replace all those characters with their ascii counterparts. However, I do not want to remove umlauts and other non-ascii character. I'd also highly prefer to use a proper solution that does not involve creating a mapping dict for all those characters. All my strings are unicode objects.

Original source