How to match unicode characters with boost::spirit?

Question

How can I match utf8 unicode characters using `boost::spirit`? For example, I want to recognize all characters in this string: ``` $ echo "На берегу пустынных волн" | ./a.out Н а б е р е гу п у с т ы н н ы х в о л н ``` When I try this simple `boost::spirit` program it will not match the unicode characters correctly: ``` #include #include #include namespace qi = boost::spirit::qi; int main() { std::cin.unsetf(std::ios::skipws); boost::spirit::istream_iterator begin(std::cin); boost::spirit::istream_iterator end; std::vector letters; bool result = qi::phrase_parse( begin, end, // input +qi::char_, // match every character qi::space, // skip whitespace letters); // result BOOST_FOREACH(char letter, letters) { std::cout << letter << " "; } std::cout << std::endl; } ``` It behaves like this: ``` $ echo "На берегу пустынных волн" | ./a.out | less <9D>

Accepted Answer

I haven't got much experience with it, but apparently Spirit (SVN trunk version) supports Unicode.

```
#define BOOST_SPIRIT_UNICODE // We'll use unicode (UTF8) all throughout
```

See, e.g. the sexpr parser sample which is in the scheme demo.

```
BOOST_ROOT/libs/spirit/example/scheme
```

I believe this is based on the demo from a presentation by Bryce Lelbach1, which specifically showcases:

- wchar support

- utree attributes (still experimental)

- s-expressions

There is an online article about S-expressions and variant.

1 In case it is indeed, here is the video from that presentation and the slides (pdf) as found here (odp)

How to match unicode characters with boost::spirit?

Solution

Problem