Using boost::tokenizer with string delimiters

boost, c++, string, tokenize

Solution

It looks like you will need to write your own TokenizerFunction to do what you want.

Problem

I've been looking boost::tokenizer, and I've found that the documentation is very thin. Is it possible to make it tokenize a string such as "dolphin--monkey--baboon" and make every word a token, as well as every double dash a token? From the examples I've only seen single character delimiters being allowed. Is the library not advanced enough for more complicated delimiters?

Original source