Extracting food items from sentences
algorithm, nlp
Solution
You can attempt it without using a trained set which contains a corpus of food items, but the approach shall work without it too.
Instead of doing simple POS tagging, do a dependency parsing combined with POS tagging. That way would be able to find relations between multiple tokens of the phrase, and parsing the dependency tree with restricted conditions like noun-noun dependencies you shall be able to find relevant chunk.
You can use spacy for dep parsing. Here is output from displacy :
https://demos.explosion.ai/displacy/?text=peanut%20butter%20and%20jelly%20sandwich%20is%20delicious&model=en&cpu=1&cph=1
- You can use freely available data here, or something better: https://en.wikipedia.org/wiki/Lists_of_foods as a training set to create a base set of food items (the hyperlinks in the crawled tree)
- Based on the dependency parsing on your new data, you can keep enriching the base data. For example: if 'butter' exists in your corpus, and 'peanut butter' is a frequently encountered pair of tokens, then 'peanut' and 'peanut butter' also get added to the corpus.
- The corpus can be maintained in a file which can be loaded in memory while processing, or database like redis,aerospike etc.
- Make sure you work with normalized i.e. small cased, special characters cleaned, words lemmatized/stemmed, both in corpus and the processing data. That would increase your coverage and accuracy.
Problem
Given a sentence: I had peanut butter and jelly sandwich and a cup of coffee for breakfast I want to be able to extract the following food items from it: peanut butter and jelly sandwich coffee Till now, using POS tagging, I have been able to extract the individual food items, i.e. peanut, butter, jelly, sandwich, coffee But like I said, what I need is peanut butter and jelly sandwich instead of the individual items. Is there some way of doing this without having a corpus or database of food items in the backend?