Reading a website with asyncore
python, sockets, web-services
Solution
Have you looked at http://asynchttp.sourceforge.net/?
"Asynchronous HTTP Client for Python
The 'asynchttp'' module is a logical extension of the Python library 'asynchat' module which is built on the 'asyncore' and 'select' modules. Our goal is to provide the functionality of the excellent 'httplib' module without using blocking sockets."
The project's last commit was 2001-05-29, so it looks dead. But it might be of interest anyway.
Disclaimer: I have not used it myself.
Also, this blog post has some information on async HTTP.
Problem
I would like to read a website asynchronously, which isnt possible with urllib as far as I know. Now I tried reading with with plain sockets, but HTTP is giving me hell. I run into all kind of funky encodings, for example transfer-encoding: chunked, have to parse all that stuff manually, and I feel like coding C, not python at the moment. Isnt there a nicer way like URLLib, asynchronously? I dont really feel like re-implementing the whole HTTP specification, when it's all been done before. Twisted isnt an option currently. Greetings, Tom