Configure suds to use custom response xml parser for big response payloads

python, suds

Solution

For my task at hand I found solution for part of my question - I'm using `suds.plugin.MessagePlugin` implementation to clear save WebService response payload and prevent suds from parsing it into DOM. it is item 2 in my original question:

avoid running suds.sax.parser and do not build DOM object containing whole response tree

Here is example of using this approach with public Web Service.

See full code in gist

    import suds
    class PayloadInterceptor(suds.plugin.MessagePlugin):
        def __init__(self, *args, **kwargs):
            self.last_payload = None

        def received(self, context):
            #recieved xml as a string
            print "%s bytes received" % len(context.reply)
            self.last_payload = context.reply    
            #clean up reply to prevent parsing
            context.reply = ""
            return context

    if __name__=='__main__':    
        wsurl = "http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso?WSDL"
        payload_interceptor = PayloadInterceptor()
        client = suds.client.Client(wsurl, plugins=[payload_interceptor])
        print client
        res = client.service.CapitalCity("NL")
        print "received %s bytes" % len(payload_interceptor.last_payload)
        print "parsed result: %s" % res
        print "response payload: %s" % payload_interceptor.last_payload

Produce output:

    ...    
    received 336 bytes
    parsed result: None
    response payload: <?xml version="1.0" encoding="utf-8"?>
    <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
      <soap:Body>
        <m:CapitalCityResponse xmlns:m="http://www.oorsprong.org/websamples.countryinfo">
          <m:CapitalCityResult>Amsterdam</m:CapitalCityResult>
        </m:CapitalCityResponse>
      </soap:Body>
    </soap:Envelope>

Problem

I'm building python client application to consume data from the WebService using suds python. For now I'm able to do remote calls and obtain responses. My problem is that response size is big - 100Mb now and the size will grow in the future. I've requested WebService provider to update API that would allow me to request parts of me model but with no much luck for now. Please suggest how suds Client can be configured to: - to use custom stream parser for response messages - that will extract the information I need only - and avoid running suds.sax.parser and do not build DOM object containing whole response tree - also I'd like my parser to save response directly to file on disk and do not store it in memory (to implement the latter I think I would need to pass custom suds transport that will do so for me)

Original source