How to get the option text using BeautifulSoup
beautifulsoup, html-parsing, python
Solution
You don't have to use `lxml` here. I have trouble installing it on my machine, so my answer does not make use of it.
from bs4 import BeautifulSoup as BS
import urllib2
soup = BS(urllib2.urlopen("./test.html").read())
contents = [str(x.text) for x in soup.find(id="start_dateid").find_all('option')]
With this, you avoid the issue of multiple select areas in the html file, since we're first limiting by `id='start_dateid'`, which guarantees for you that you have the right `<select>`, since within each html document each html element must have a unique `id` attribute if it has an `id` attribute. Then, we're searching for all of the `<option>` tags only within that `<select>` tag, and then we get all of the values from each `<option>`.
Problem
I want to using BeautifulSoup to get the option text in the following html. For example: I'd like to get 2002/12 , 2003/12 etc. ``` <select id="start_dateid"> <option value="0">2002/12</option> <option value="1">2003/12</option> <option value="2">2004/12</option> <option value="3">2005/12</option> <option value="4">2006/12</option> <option value="5" selected="">2007/12</option> <option value="6">2008/12</option> <option value="7">2009/12</option> <option value="8">2010/12</option> <option value="9">2011/12</option> </select> ``` What's the best way to get the contents? Now I'm using the following code but I don't know how to use beautiful soup for that. If there are more than one selected areas in the html file, the result will be incorrect. Here is what I have so far: ``` import urllib2 from bs4 import BeautifulSoup import lxml soup = BeautifulSoup(urllib2.urlopen("./test.html").read(),"lxml"); for item in soup.find_all('option'): print(''.join(str(item.find(text=True)))); ```