stdout redirection with ctypes

ctypes, python, windows

Solution

Use the same C runtime that CPython 3.x uses (e.g. msvcr100.dll for 3.3). Also include a call to `fflush(NULL)` before and after redirecting `stdout`. For good measure, redirect the Windows `StandardOutput` handle, in case a program uses the Windows API directly.

This can get complicated if the DLL uses a different C runtime, which has its own set of POSIX file descriptors. That said, it should be OK if it gets loaded after you've redirected Windows `StandardOutput`.

Edit:

I've modified the example to run in Python 3.5+. VC++ 14's new "Universal CRT" makes it much more difficult to use C standard I/O via ctypes.

import os
import sys
import ctypes, ctypes.util

kernel32 = ctypes.WinDLL('kernel32')

STD_OUTPUT_HANDLE = -11

if sys.version_info < (3, 5):
    libc = ctypes.CDLL(ctypes.util.find_library('c'))
else:
    if hasattr(sys, 'gettotalrefcount'): # debug build
        libc = ctypes.CDLL('ucrtbased')
    else:
        libc = ctypes.CDLL('api-ms-win-crt-stdio-l1-1-0')

    # VC 14.0 doesn't implement printf dynamically, just
    # __stdio_common_vfprintf. This take a va_array arglist,
    # which I won't implement, so I escape format specificiers.

    class _FILE(ctypes.Structure):
        """opaque C FILE type"""

    libc.__acrt_iob_func.restype = ctypes.POINTER(_FILE)    

    def _vprintf(format, arglist_ignored):
        options = ctypes.c_longlong(0) # no legacy behavior
        stdout = libc.__acrt_iob_func(1)
        format = format.replace(b'%%', b'\0')
        format = format.replace(b'%', b'%%')
        format = format.replace(b'\0', b'%%')
        arglist = locale = None        
        return libc.__stdio_common_vfprintf(
            options, stdout, format, locale, arglist)

    def _printf(format, *args):
        return _vprintf(format, args)

    libc.vprintf = _vprintf
    libc.printf = _printf

def do_print(label):
    print("%s: python print" % label)
    s = ("%s: libc _write\n" % label).encode('ascii')
    libc._write(1, s, len(s))
    s = ("%s: libc printf\n" % label).encode('ascii')
    libc.printf(s)
    libc.fflush(None) # flush all C streams

if __name__ == '__main__':
    # save POSIX stdout and Windows StandardOutput
    fd_stdout = os.dup(1)
    hStandardOutput = kernel32.GetStdHandle(STD_OUTPUT_HANDLE)

    do_print("begin")

    # redirect POSIX and Windows
    with open("TEST.TXT", "w") as test:
        os.dup2(test.fileno(), 1)
        kernel32.SetStdHandle(STD_OUTPUT_HANDLE, libc._get_osfhandle(1))

    do_print("redirected")

    # restore POSIX and Windows
    os.dup2(fd_stdout, 1)
    kernel32.SetStdHandle(STD_OUTPUT_HANDLE, hStandardOutput)

    do_print("end")

Problem

I'm trying to redirect the output of printf functions to a file on Windows. I'm using ctypes with python3 to invoke the functions. My code is: ``` import os, sys from ctypes import * if __name__ == '__main__': print("begin") saved_stdout=os.dup(1) test_file=open("TEST.TXT", "w") os.dup2(test_file.fileno(), 1) test_file.close() print("python print") cdll.msvcrt.printf(b"Printf function 1\n") cdll.msvcrt.printf(b"Printf function 2\n") cdll.msvcrt.printf(b"Printf function 3\n") os.dup2(saved_stdout, 1) print("end") ``` But when I run the code from Eclipse I get the following on the screen: ``` begin end Printf function 1 Printf function 2 Printf function 3 ``` ...and the following in the TEST.txt ``` python print ``` When I run this from cmd, this is what is on the screen: ``` begin end ``` ..and this is in the TEST.txt: ``` python print ``` When I comment out the second `dup2()` statement e.g. ``` import os, sys from ctypes import * if __name__ == '__main__': print("begin") saved_stdout=os.dup(1) test_file=open("TEST.TXT", "w") os.dup2(test_file.fileno(), 1) test_file.close() print("python print") cdll.msvcrt.printf(b"Printf function 1\n") cdll.msvcrt.printf(b"Printf function 2\n") cdll.msvcrt.printf(b"Printf function 3\n") #os.dup2(saved_stdout, 1) print("end") ``` From Eclipse, on the screen: ``` begin ``` ...and in the TEST.txt file: ``` python print end Printf function 1 Printf function 2 Printf function 3 ``` From cmd, on the screen: ``` begin ``` ...and in the TEST.txt file: ``` python print end ``` I'm totally confused now. I read all the redirection threads here on StackOverflow and I can't understand what's going on. Anyway, what I've gathered is that C functions access the stdout that is bind directly to the file descriptor, while python uses a special object for that - stdout File Object. So the elementary `sys.stdout=*something*` doesn't work with ctypes. I've even tried `os.fdopen(1)` on the dup2-ed output and then calling `flush()` after every `printf` statement but this isn't working again. I'm totally out of ideas now and would appreciate if someone have a solution for this.

Original source

Related problems