BeautifulSoup does not work for some web sites

Question

I have this sript: ``` import urrlib2 from bs4 import BeautifulSoup url = "http://www.shoptop.ru/" page = urllib2.urlopen(url).read() soup = BeautifulSoup(page) divs = soup.findAll('a') print divs ``` For this web site, it prints empty list? What can be problem? I am running on Ubuntu 12.04

Accepted Answer

Actually there are quite couple of bugs in BeautifulSoup which might raise some unknown errors. I had a similar issue when working on apache using `lxml` parser

So, just try to use other couple of parsers mentioned in the documentation

```
soup = BeautifulSoup(page, "html.parser")
```

This should work!

BeautifulSoup does not work for some web sites

Solution

Problem