How can I iterate over every character in a given encoding using Python?

encoding, python, unicode

Solution

All Unicode characters can be represented in `UTF-n` for all defined `n`. What are you trying to achieve?

If you really want to do something like print all the valid characters in a particular encoding, without needing to know whether the encoding is "single byte" or "multi byte" or whether its size is fixed or not:

import unicodedata as ucd
import sys

def dump_encoding(enc):
    for i in xrange(sys.maxunicode):
        u = unichr(i)
        try:
            s = u.encode(enc)
        except UnicodeEncodeError:
            continue
        try:
            name = ucd.name(u)
        except:
            name = '?'
        print "U+%06X %r %s" % (i, s, name)

if __name__ == "__main__":
    dump_encoding(sys.argv[1])

Suggestions: Try it out on something small, like `cp1252`. Redirect stdout to a file.

Problem

Is there a way to iterate over every character in a given encoding, and print it's code? Say, UTF8?

Original source