What's the best way to transfer data from python to another application in windows?

com, data-transfer, python, winapi

Solution

Staying within the Windows interprocess communication mechanisms, we had positive experience using windows named pipes. Using Windows overlapped IO and the `win32pipe` module from pywin32.

You can learn much about win32 and python in the Python Programming On Win32 book.

The sending part simply writes to `r'\\.\pipe\mypipe'`.

A listener (`ovpipe`) object holds an event handle, and waiting for a message with possible other events involves calling `win32event.WaitForMultipleObjects`.

rc = win32event.WaitForMultipleObjects(
    eventlist,    # Objects to wait for.
    0,            # Wait for one object
    timeout)      # timeout in milli-seconds.

Here is part of the python overlapped listener class:

import win32event
import pywintypes
import win32file
import win32pipe

class ovpipe:
"Overlapped I/O named pipe class"
def __init__(self):
    self.over=pywintypes.OVERLAPPED()
    evt=win32event.CreateEvent(None,1,0,None)
    self.over.hEvent=evt
    self.pname='mypipe'
    self.hpipe = win32pipe.CreateNamedPipe(
        r'\\.\pipe\mypipe',             # pipe name 
        win32pipe.PIPE_ACCESS_DUPLEX|   # read/write access
        win32file.FILE_FLAG_OVERLAPPED,
        win32pipe.PIPE_TYPE_MESSAGE|    # message-type pipe 
        win32pipe.PIPE_WAIT,            # blocking mode 
        1,                              # number of instances 
        512,                            # output buffer size 
        512,                            # input buffer size 
        2000,                           # client time-out
        None)                           # no security attributes
    self.buffer = win32file.AllocateReadBuffer(512)
    self.state='noconnected'
    self.chstate()

def execmsg(self):
    "Translate the received message"
    pass

def chstate(self):
    "Change the state of the pipe depending on current state"
    if self.state=='noconnected':
        win32pipe.ConnectNamedPipe(self.hpipe,self.over)
        self.state='connectwait'
        return -6

    elif self.state=='connectwait':
        j,self.strbuf=win32file.ReadFile(self.hpipe,self.buffer,self.over)
        self.state='readwait'
        return -6

    elif self.state=='readwait':
        size=win32file.GetOverlappedResult(self.hpipe,self.over,1)
        self.msg=self.strbuf[:size]
        ret=self.execmsg()
        self.state = 'noconnected'
        win32pipe.DisconnectNamedPipe(self.hpipe)
        return ret

Problem

I'm developing an application with a team in .Net (C++) and provide a COM interface to interact with python and other languages. What we've found is that pushing data through COM turns out to be pretty slow. I've considered several alternatives: - dumping data to a file and sending the file path through com - Shared Memory via mmap? - Stream data through a socket directly? From your experience what's the best way to pass around data?

Original source