python: httplib.CannotSendRequest when nesting threaded SimpleXMLRPCServers

Question

I am intermittently receiving a httplib.CannotSendRequest exception when using a chain of SimpleXMLRPCServers that use the SocketServer.ThreadingMixin. What I mean by 'chain' is the following: I have a client script which uses xmlrpclib to call a function on a SimpleXMLRPCServer. That server, in turn, calls another SimpleXMLRPCServer. I realise how convoluted that sounds, but there are good reasons that this architecture has been selected, and I don't see a reason it shouldn't be possible. ``` (testclient)client_script ---calls--> (middleserver)SimpleXMLRPCServer ---calls---> (finalserver)SimpleXMLRPCServer --- does something ``` - If I do not use SocketServer.ThreadingMixin then this issue doesn't occur (but I need the requests to be multi-threaded so this doesn't help.) - If I only have a single level of services (ie just client script calling final server directly) this doesn't happen. I have been able to reproduce the issue in the simple test code below. There are three snippets:…

Accepted Answer

Okay, I'm a bit stupid. I think I was staring at the code for to protracted a period of time that I missed the obvious solution staring me in the face (quite literally, because the answer is actually in the actual question.)

Basically, the CannotSendRequest occurs when an httplib.HTTPConnection is interrupted by an intervening 'request' operation. Basically, each httplib.HTTPConnection.request must be paired with a .getresponse() call. If that pairing is interrupted by another request operation, the second request will produce the CannotSendRequest error. so:

```
connection = httplib.HTTPConnection(...)
connection.request(...)
connection.request(...)
```

will fail because you have two requests on the same connection before any getresponse is called.

Linking that back to my question:

- the only place in the three programs where such connections are being made are in the serverproxy calls.

- the problem only occurs during threading, so it's likely a race condition.

- the only place a serverproxy call is shared is in middleserver.py

The solution then, is obviously to have each thread create it's own serverproxy. The fixed version of middleserver is below, and it works:

```
import SocketServer
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
import xmlrpclib

class AsyncXMLRPCServer(SocketServer.ThreadingMixIn,SimpleXMLRPCServer): pass

# Create server
server = AsyncXMLRPCServer(('', 8888), SimpleXMLRPCRequestHandler)
server.register_introspection_functions()

def call_waste():
    # Each call to this function creates its own serverproxy.
    # If this function is called by concurrent threads, each thread
    # will safely have its own serverproxy.
    s = xmlrpclib.ServerProxy('http://localhost:9999')
    s.waste_time()
    return True

server.register_function(call_waste, 'call_waste')
server.serve_forever()
```

Since this version results in each thread having its own xmlrpclib.serverproxy, there is no risk of the serverproxy invoking HTTPConnection.request more than once in succession. The programs work as intended.

Sorry for the bother.

python: httplib.CannotSendRequest when nesting threaded SimpleXMLRPCServers

Solution

Problem