How to find spans with a specific class containing specific text using beautiful soup and re?
beautifulsoup, python, regex
Solution
import re
from bs4 import BeautifulSoup
html_doc = """
<html>
<body>
<span class="blue">here is a lot of text that i don't need</span>
<span class="blue">this is the span i need because it contains 04/18/13 7:29pm</span>
<span class="blue">04/19/13 7:30pm</span>
<span class="blue">Posted on 04/20/13 10:31pm</span>
</body>
</html>
"""
# parse the html
soup = BeautifulSoup(html_doc)
# find a list of all span elements
spans = soup.find_all('span', {'class' : 'blue'})
# create a list of lines corresponding to element texts
lines = [span.get_text() for span in spans]
# collect the dates from the list of lines using regex matching groups
found_dates = []
for line in lines:
m = re.search(r'(\d{2}/\d{2}/\d{2} \d+:\d+[a|p]m)', line)
if m:
found_dates.append(m.group(1))
# print the dates we collected
for date in found_dates:
print(date)
output:
04/18/13 7:29pm
04/19/13 7:30pm
04/20/13 10:31pm
Problem
how can I find all span's with a class of `'blue'` that contain text in the format: ``` 04/18/13 7:29pm ``` which could therefore be: ``` 04/18/13 7:29pm ``` or: ``` Posted on 04/18/13 7:29pm ``` in terms of constructing the logic to do this, this is what i have got so far: ``` new_content = original_content.find_all('span', {'class' : 'blue'}) # using beautiful soup's find_all pattern = re.compile('<span class=\"blue\">[data in the format 04/18/13 7:29pm]</span>') # using re for _ in new_content: result = re.findall(pattern, _) print result ``` I've been referring to https://stackoverflow.com/a/7732827 and https://stackoverflow.com/a/12229134 to try and figure out a way to do this, but the above is all i have got so far. edit: to clarify the scenario, there are span's with: ``` <span class="blue">here is a lot of text that i don't need</span> ``` and ``` <span class="blue">this is the span i need because it contains 04/18/13 7:29pm</span> ``` and note i only need `04/18/13 7:29pm` not the rest of the content. edit 2: I also tried: ``` pattern = re.compile('<span class="blue">.*?(\d\d/\d\d/\d\d \d\d?:\d\d\w\w)</span>') for _ in new_content: result = re.findall(pattern, _) print result ``` and got error: ``` 'TypeError: expected string or buffer' ```