Why do I get an AttributeError when trying to use BeautifulSoup's `.find` to find text in a page?
beautifulsoup, python, web-scraping
Solution
The `find` and `find_all` methods do not search for arbitrary text in the document, they search for HTML tags. The documentation makes that clear (my italics):
Pass in a value for name and you’ll tell Beautiful Soup to only consider tags with certain names. Text strings will be ignored, as will tags whose names that don’t match. This is the simplest usage:
soup.find_all("title")
# [<title>The Dormouse's story</title>]
That's why your `soup.find("Born")` is returning `None` and hence why it complains about `NoneType` (the type of `None`) having no `findNext()` method.
That page you reference contains (at the time this answer was written) eight copies of the word "born", none of which are tags.
Looking at the HTML source for that page, you'll find the best option may be to look for the correct span (formatted for readabilty):
<th scope="row" style="text-align: left;">Born</th>
<td>
<span class="nickname">Steven Paul Jobs</span><br />
<span style="display: none;">(<span class="bday">1955-02-24</span>)</span>February 24, 1955<br />
</td>
Problem
I am trying to scrape a website with BeautifulSoup but am having a problem. I was following a tutorial done in python 2.7 and it had exactly the same code in it and had no problems. ``` import urllib.request from bs4 import * htmlfile = urllib.request.urlopen("http://en.wikipedia.org/wiki/Steve_Jobs") htmltext = htmlfile.read() soup = BeautifulSoup(htmltext) title = (soup.title.text) body = soup.find("Born").findNext('td') print (body.text) ``` If I try to run the program I get, ``` Traceback (most recent call last): File "C:\Users\USER\Documents\Python Programs\World Population.py", line 13, in <module> body = soup.find("Born").findNext('p') AttributeError: 'NoneType' object has no attribute 'findNext' ``` Is this a problem with python 3 or am i just too naive?