How to remove list of words from a list of strings
list-comprehension, python, regex, stop-words
Solution
Here is my stab at it. This uses regular expressions.
import re
pattern = re.compile("(of|the|in|for|at)\W", re.I)
phrases = ['of New York', 'of the New York']
map(lambda phrase: pattern.sub("", phrase), phrases) # ['New York', 'New York']
Sans `lambda`:
[pattern.sub("", phrase) for phrase in phrases]
Update
Fix for the bug pointed out by gnibbler (thanks!):
pattern = re.compile("\\b(of|the|in|for|at)\\W", re.I)
phrases = ['of New York', 'of the New York', 'Spain has rain']
[pattern.sub("", phrase) for phrase in phrases] # ['New York', 'New York', 'Spain has rain']
@prabhu: the above change avoids snipping off the trailing "in" from "Spain". To verify run both versions of the regular expressions against the phrase "Spain has rain".
Problem
Sorry if the question is bit confusing. This is similar to this question I think this the above question is close to what I want, but in Clojure. There is another question I need something like this but instead of '[br]' in that question, there is a list of strings that need to be searched and removed. Hope I made myself clear. I think that this is due to the fact that strings in python are immutable. I have a list of noise words that need to be removed from a list of strings. If I use the list comprehension, I end up searching the same string again and again. So, only "of" gets removed and not "the". So my modified list looks like this ``` places = ['New York', 'the New York City', 'at Moscow' and many more] noise_words_list = ['of', 'the', 'in', 'for', 'at'] for place in places: stuff = [place.replace(w, "").strip() for w in noise_words_list if place.startswith(w)] ``` I would like to know as to what mistake I'm doing.