Why is text utf-16 as opposed to utf-8

haskell, text

Solution

There was a project to convert `text` to using utf8 internally, because that's irrelevant to the API it provides. After completing enough to benchmark, the project was considered not an improvement and not integrated with the mainline at this time. There is a chance it will be in the future, if it can become a sufficient improvement. Here's the full story: http://jaspervdj.be/posts/2011-08-19-text-utf8-the-aftermath.html

Problem

The `text` library uses utf-16 internally. utf-8 is a more commonly used encoding, especially in C libraries. In addition, utf-8 uses less memory most of the time. Why does `text` use utf-16?

Original source