String split using multiple delimiters in java

java, stop-words, string

Solution

Construct a regular expression of the form

delim1|delim2|delim3

then use `String`'s `split()` method to split the text by any of the delimiters.

In order to construct the regexp, read each delimiter, and pass it to `Pattern.quote` before appending to the regex that you build. This would let your delimiters use regex metacharacters as well.

Problem

I am working on a data mining algorithm where I need to tokenize the string using multiple words. I have a separate file which contain all the stopwords. What I need to do is to tokenize the input string by any of the word (stopword) working as delimiter. For eg. If the file contains stopwords as a is and of that and the input string comes to be "a computer cluster consists of a set of loosely connected computers that work together" the output comes to be computer cluster consists set loosely connected computers work together Checking string against all the stopwords recursively would be very time consuming? Is there any good method for this?

Original source