Elegant way to try/except a series of BeautifulSoup commands?

Question

I'm parsing webpages on a site displaying item data. These items have about 20 fields which may or may not occur -- say: price, quantity, last purchased, high, low, etc. I'm currently using a series of commands; about 20 lines of `soup.find('div',{'class':SOME_FIELD_OF_INTEREST})` to look for each separate field of interest. (Some are in `div`, `span`, `dd`, and so on, so it's difficult to just do a `soup.find_all('div')` command.) My question: Is there an elegant way to `try` and `except` everything such that the viewing of said code can be more compact or concise? Right now a sample line would look like: ``` try: soup.find('div', {'id':'item-pic'}).img["src"] except: "" ``` I was hoping to combine everything in one line. I don't think I can syntactically run try: ` except:

`, and I'm not sure how I'd write a function that goes `try_command(soup.find('div',{'id':'item-pic'}).img["src"])` without actually running the command. I'd love to hear if anybody has any adv…

Accepted Answer

maybe something like that:

```
def try_these(start_obj, *args) :
        obj = start_obj
        for trythat in args :
            if obj is None :
                return None
            try :
                if isinstance(trythat, str) :
                    obj = getattr(obj, trythat)
                else :
                    method, opts = trythat
                    obj = getattr(obj, method)(*opts)
            except :
                return None
        return obj    
src = try_these(soup, ('find', ({'id':'item-pic'},),), 
                      'img', 
                      ('get', ('src',),) )
```

where you can pass `str` to get attribute from object or `tuple` (str method, tuple params), finally you'll get `None` or result. I'm not familiar with soup so I'm not sure if `get('src')` would be a good approach (as probably its not a dict), anyway you can easily modify that snippet to accept something more than only 'call or attr'.

Inspired by your question I wrote simple python module that helps to deal with such situation, you can find it here

```
import silentcrawler

wrapped = silentcrawler.wrap(soup)
# just return None on failure
print wrapped.find('div', {'id':'item-pic'}).img["src"].value_

# or
def on_success(value) :
    print 'found value:', value
wrapped = silentcrawler.wrap(soup, success=on_success)
# call on_success if everything will be ok
wrapped.find('div', {'id':'item-pic'}).img["src"].value_ 
```

there is more possibilities

Elegant way to try/except a series of BeautifulSoup commands?

Solution

Problem