Converting "normal" std::string to utf-8

Question

Let's see if I can explain this without too many factual errors... I'm writing a string class and I want it to use `utf-8` (stored in a std::string) as it's internal storage. I want it to be able to take both "normal" `std::string` and `std::wstring` as input and output. Working with std::wstring is not a problem, I can use `std::codecvt_utf8` to convert both from and to std::wstring. However after extensive googling and searching on SO I have yet to find a way to convert between a "normal/default" C++ std::string (which I assume in Windows is using the local system localization?) and an utf-8 std::string. I guess one option would be to first convert the std::string to an std::wstring using `std::codecvt` and then convert it to utf-8 as above, but this seems quite inefficient given that at least the first 128 values of a char should translate straight over to utf-8 without conversion regardless of localization if I understand correctly. I found this similar que…

Accepted Answer

If your "normal string" is encoded using the system's code page and you want to convert it to UTF-8 then this should work:

```
std::string codepage_str;
int size = MultiByteToWideChar(CP_ACP, MB_COMPOSITE, codepage_str.c_str(),
                               codepage_str.length(), nullptr, 0);
std::wstring utf16_str(size, '\0');
MultiByteToWideChar(CP_ACP, MB_COMPOSITE, codepage_str.c_str(),
                    codepage_str.length(), &utf16_str[0], size);

int utf8_size = WideCharToMultiByte(CP_UTF8, 0, utf16_str.c_str(),
                                    utf16_str.length(), nullptr, 0,
                                    nullptr, nullptr);
std::string utf8_str(utf8_size, '\0');
WideCharToMultiByte(CP_UTF8, 0, utf16_str.c_str(),
                    utf16_str.length(), &utf8_str[0], utf8_size,
                    nullptr, nullptr);
```

Converting "normal" std::string to utf-8

Solution

Problem

Related problems