python, "urlparse.urlparse(url).hostname" return None value

python, urlparse

Solution

Another good way without any try and except block -

Replace `hostname = urlparse.urlparse(url).hostname` with

hostname = urlparse.urlparse(url).hostname or ''

and similarly path = `urlparse.urlparse(url).path` with

path = urlparse.urlparse(url).path or ''

Hope this helps !

Problem

After loging in on a website I want to collect its links. This I do with this function (using mechanize and urlparse libraries): ``` br = mechanize.Browser() . . #logging in on website . for link in br.links(): url = urlparse.urljoin(link.base_url, link.url) hostname = urlparse.urlparse(url).hostname path = urlparse.urlparse(url).path #print hostname #by printing this I found it to be the source of the None value mylinks.append("http://" + hostname + path) ``` and I get this error message: ``` mylinks.append("http://" + hostname + path) TypeError: cannot concatenate 'str' and 'NoneType' objects ``` I am not sure on how to fix this, or even if it can be fixed at all. Is there any way to force the function to append even if it would produce a nonworking and weird result for the None value? Alternatively, what I'm really after in the link is what the link ends with. for example, the html code for one of the links look like this (what I am after is the world "lexik"): ``` <td class="center"> <a href="http://UnimportantPartOfLink/lexik>>lexik</a> </td> ``` so an alternative route would be if mechanize can just collect this value directly, bypassing the links and None value troubles

Original source