Using Python and BeautifulSoup (saved webpage source codes into a local file)
beautifulsoup, python
Solution
With Chandan's help, the problem has been solved. All the credits shall go to him. :)
the "urllib2.url" is useless here.
from bs4 import BeautifulSoup
import re
# import urllib2
url = "C:\example.html"
page = open(url)
soup = BeautifulSoup(page.read())
cities = soup.find_all('span', {'class' : 'city-sh'})
for city in cities:
print city
Problem
I am using Python 2.7 + BeautifulSoup 4.3.2. I am trying to use Python and BeautifulSoup to pick up information on a webpage. Because the webpage is in the company website and requires login and redirection, I copied the target page's source code page into a file and saved it as “example.html” in C:\ for the convenience of practicing. This is a part of the original code: ``` <tr class="ghj"> <td><span class="city-sh"><sh src="./citys/1.jpg" alt="boy" title="boy" /></span><a href="./membercity.php?mode=view&u=12563">port_new_cape</a></td> <td class="position"><a href="./search.php?id=12563&sr=positions" title="Search positions">452</a></td> <td class="details"><div>South</div></td> <td>May 09, 1997</td> <td>Jan 23, 2009 12:05 pm </td> </tr> ``` The code I worked out so far is: ``` from bs4 import BeautifulSoup import re import urllib2 url = "C:\example.html" page = urllib2.urlopen(url) soup = BeautifulSoup(page.read()) cities = soup.find_all('span', {'class' : 'city-sh'}) for city in cities: print city ``` This is just the first stage of testing, so it's somewhat incomplete. However, when I run it, it gives an error message. Seems it’s improper to use `urllib2.urlopen` to open a local file. ``` Traceback (most recent call last): File "C:\Python27\Testing.py", line 8, in <module> page = urllib2.urlopen(url) File "C:\Python27\lib\urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "C:\Python27\lib\urllib2.py", line 404, in open response = self._open(req, data) File "C:\Python27\lib\urllib2.py", line 427, in _open 'unknown_open', req) File "C:\Python27\lib\urllib2.py", line 382, in _call_chain result = func(*args) File "C:\Python27\lib\urllib2.py", line 1247, in unknown_open raise URLError('unknown url type: %s' % type) URLError: <urlopen error unknown url type: c> ``` How can I practice using a local file?