How do I decode an encoded HttpWebResponse?

Question

I have this piece of code to fetch a Page HTML from an URL, however the response content looks encoded. Code: ``` HttpWebRequest xhr = (HttpWebRequest) WebRequest.Create(new Uri("https://www.youtube.com/watch?v=_Ewh75YGIGQ")); xhr.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate; //xhr.CookieContainer = request.Account.CookieContainer; xhr.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; xhr.Headers["Accept-Encoding"] = "gzip, deflate, br"; xhr.Headers["Accept-Language"] = "en-US,en;q=0.5"; xhr.Headers["Upgrade-Insecure-Requests"] = "1"; xhr.KeepAlive = true; xhr.UserAgent = "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)"; xhr.Host = "www.youtube.com"; xhr.Referer = "https://www.youtube.com/watch?v=6aCpYxzRkf4"; var response = xhr.GetResponse(); string html; using (StreamReader reader = new StreamReader(response.GetResponseStream())) { html = reader.ReadToEnd(); } ``` These are the response headers: ``` X-XSS-Protection:…

Accepted Answer

The answer is in the response header: Content-Encoding: br -> This means Brotli compression.

There is a .NET implementation (NuGet package) for it:

Install this to your project add "using Brotli; " and replace the "using (StreamReader....." with this code:

```
       using (BrotliStream bs = new BrotliStream(response.GetResponseStream(), System.IO.Compression.CompressionMode.Decompress)) {
            using (System.IO.MemoryStream msOutput = new System.IO.MemoryStream()) {
                bs.CopyTo(msOutput);
                msOutput.Seek(0, System.IO.SeekOrigin.Begin);
                using (StreamReader reader = new StreamReader(msOutput)) {
                    html = reader.ReadToEnd();
                }
            }
        }
```

How do I decode an encoded HttpWebResponse?

Solution

Problem

Related problems