NullReferenceException in HtmlAgilityPack

Question

I am trying to extract a `link` using `xpath` from the below mentioned url ``` string url = "http://www.album-cover-art.org/search.php?q=Ruin+-+Live+Album+Version+Lamb+of+God" ``` My code: ``` HtmlAgilityPack.HtmlWeb web = new HtmlAgilityPack.HtmlWeb(); HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument(); htmlDoc = web.Load(url); //Exception generated here Line 23 if (htmlDoc.DocumentNode != null) { HtmlNode linkNode = htmlDoc.DocumentNode.SelectSingleNode(".//*[@id='related_search_row']/img/@src"); if (linkNode != null) Console.WriteLine(linkNode.InnerText); } ``` The above code compiles fine but when I try to run it generates a exception ``` Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object. ``` Complete stacktrace ``` System.NullReferenceException: Object reference not set to an instance of an object. at HtmlAgilityPack.HtmlDocument.ReadDocumentEncoding(HtmlNode node) in C:\Source\htmlagilitypack\Trunk\Htm…

Accepted Answer

This is a bug in HtmlAgilityPack. The document you're trying to parse has `<meta http-equiv="Content-Type" content="text/html; charset=iso-utf-8">` where the `charset` value (`iso-utf-8`) couldn't be parsed as a valid encoding name by AgilityPack. As Simon Mourier said, this is a bug introduced in 1.4.0.0.

To avoid this, manually load your document from a stream and set the encoding manually like this:

```
var htmlDoc = new HtmlDocument();
htmlDoc.OptionReadEncoding = false;
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
using (var response = (HttpWebResponse)request.GetResponse())
{
    using (var stream = response.GetResponseStream())
    {
        htmlDoc.Load(stream, Encoding.UTF8);
    }
}
```

NullReferenceException in HtmlAgilityPack

Solution

Problem