Invalid and/or missing SSL certificate

Question

I'm trying to build a datastore on Google App Engine to collect some stream data off of StockTwits for a bunch of companies. I'm basically replicating one I did with Twitter, but it's giving me an HTTPException: Invalid and/or missing SSL certificate error for one of the URLs. I changed the URL to look at another company, but got the same result. Here's the code that pulls the data: ``` class StreamHandler(webapp2.RequestHandler): def get(self): tickers = ['AAPL','GOOG', 'IBM', 'BAC', 'INTC', 'DELL', 'C', 'JPM', 'WFM', 'WMT', 'AMZN', 'HOT', 'SPG', 'SWY', 'HTSI', 'DUK', 'CEG', 'XOM', 'F', 'WFC', 'CSCO', 'UAL', 'LUV', 'DAL', 'COST', 'YUM', 'TLT', 'HYG', 'JNK', 'LQD', 'MSFT', 'GE', 'LVS', 'MGM', 'TWX', 'DIS', 'CMCSA', 'TWC', 'ORCL', 'WPO', 'NYT', 'GM', 'JCP', 'LNKD', 'OPEN', 'NFLX', 'SBUX', 'GMCR', 'SPLS', 'BBY', 'BBBY', 'YHOO', 'MAR', 'L', 'LOW', 'HD', 'HOV', 'TOL', 'NVR', 'RYL', 'GIS', 'K', 'POST', 'KRFT', 'CHK', 'GGP', 'RSE', 'RWT', 'AIG', 'CB', 'BRK.A', 'CAT'] for i in set(tickers):…

Accepted Answer

I don't know why GAE is having a problem with it, but I notice the certificate returned by api.stocktwits.com doesn't match the server name on its Subject's Common Name (which is ssl2361.cloudflare.com), but only on one of its Subject Alternative Names ("DNS Name=*.stocktwits.com"). Maybe Subject Alternatives Names are not supported, or don't work with wildcard names as used here. (This would be a Google bug / missing feature.)

I was able to reproduce your problem and find a workaround by calling GAE urlfetch.fetch API. (As you may know, on GAE urllib2 is implemented as a wrapper for urlfetch.)

Starting at the line with your `urllib2.Request` up to your `jason.load`, replace with:

```
sttweets = urlfetch.fetch(tickerstringst, validate_certificate=False)
stcode = sttweets.status_code

if stcode == 200:
    stresults = json.loads(sttweets.content, 'utf-8')
```

And your error goes away, along with any assurance you are actually taking with the real site (though the traffic should still be encrypted).

Currently the `urlfetch.fetch` GAE API docs say:

`validate_certificate` The underlying implementation currently defaults to False, but will default to True in the near future.

Well, welcome to the future because validate_certificate now seems to be defaulting to `True`.

This is probably a bug (or missing feature, if you want to be kind) in GAE urlfetch.fetch and I encourage you to report it to Google as such.

Invalid and/or missing SSL certificate

Solution

Problem