Using Python to Collect Historical 30yr Treasury Bond Prices

input, post, python, web

Solution

Page required `submit=Show+Prices` in POST data.

I tested it with `curl` on linux.

Without `submit=Show+Prices` this give me normal page:

curl -k https://www.treasurydirect.gov/GA-FI/FedInvest/selectSecurityPriceDate.htm -d "priceDate.month=7&priceDate.day=8&priceDate.year=2013"

With `submit=Show+Prices` this give me page with data:

curl -k https://www.treasurydirect.gov/GA-FI/FedInvest/selectSecurityPriceDate.htm -d "priceDate.month=7&priceDate.day=8&priceDate.year=2013&submit=Show+Prices"

Problem

I am using the website https://www.treasurydirect.gov/GA-FI/FedInvest/selectSecurityPriceDate.htm. This website gives me access to historical bond prices based on the CUSIP number of the bond. I am trying to construct a chart to display the historical prices of a specific bond over time, however the website does not support this function. Instead it allows the user to look up specific dates. I am curious if there is a way in Python to input the dates I want to lookup and then "post" these dates to the website and read the resulting webpage to search for my specific CUSIP and construct a dict of dates and prices in order to graphically visualize and interpret this data. This would be an easy task if inputting a date took you to a specific directory of the website that could be manipulated in the address, but unfortunately the way the site is setup it seems to be a built in program that displays the corresponding charts for each day. If anyone could help me out with this I would greatly appreciate it! I have tried using urllib2 and the Request function to send a dict with the parameters for priceData.day, month, and year, but it does not open the correct webpage. ``` import urllib2 def URLRequest(url, params, method="GET"): if method == "POST": return urllib2.Request(url, data=urllib.urlencode(params)) else: return urllib2.Request(url + "?" + urllib.urlencode(params)) data = URLRequest("https://www.treasurydirect.gov/GA-FI/FedInvest/selectSecurityPriceDate.htm",{"priceData.month":"7","priceData.day":"8","priceData.year":"2013"}, method="POST") response = urllib2.urlopen(data) response.read() [Out]: The source file of the website without displaying the information I need ```

Original source