Python / web scrape / aspx -- is it humanly possible when there are no forms?

Question

Total noob, obviously. Teaching self Python for web scraping in the interest of open records/government transparency/reporting/etc. There's an .aspx page I want to scrape, a week-by-week calendar for January - March 2012 But it has no forms ... Perhaps you fine people can tell me if a solution is even possible before I spend days fighting with it. http://webmail.legis.ga.gov/Calendar/default.aspx?chamber=house The only way to see the appointments on the calendar is by choosing a day on a picture of a calendar. But, at least, if you click on Monday, it shows all the week's appointments. (I would like to gather all those appointments in order to count how often each committee meets, a bit of a proxy for counting what kind of legislation gets attention and what kind is ignored.) But so, what strategy to use? It appears that each month at least down in its bowels is assigned to a sequential four-digit number prepended with a "V", like V4414, and days with a non-prepended number. I'm only…

Accepted Answer

The form contains four `input`s with `name`s of: - `__EVENTTARGET` - `__EVENTARGUMENT` - `__VIEWSTATE` - `__EVENTVALIDATION` The latter two have initial values. You need to scrape those. The former two are set by those links. For example, you have a link: ``` 1 ``` Look at the `href`: ``` javascript:__doPostBack('calMain','4504') ``` Somehow, parse those two strings out of it. The former is `__EVENTTARGET`. The latter is `__EVENTARGUMENT`. Once you have all four pieces of data, you can issue a `POST` request to get the next page.

Python / web scrape / aspx -- is it humanly possible when there are no forms?

Solution

Problem