SetStdHandle has no effect on cout/printf

visual-c++, winapi, windows

Solution

Wow, after a while of searching for a way to manually set the `HANDLE` of a `FILE`, I finally discovered that there is a fairly straightforward way to do this using the C Run-Time Library:

std::vector<int> vfdStdOut;
void StartStdOutRedirection(_In_ LPCSTR lpFile)
{
    // Duplicate stdout and give it a new file descriptor
    int fdDup = _dup(_fileno(stdout));
    vfdStdOut.push_back(fdDup);

    // Re-open stdout to the new file
    freopen(lpFile, "w", stdout);
}

bool EndStdOutRedirection(void)
{
    if (vfdStdOut.size() != 0)
    {
        // Get last saved file descriptor and restore it
        int fdNext = vfdStdOut.back();
        _dup2(fdNext, _fileno(stdout));

        // Need to close the file associated with the saved file descriptor
        _close(fdNext);

        vfdStdOut.pop_back();
        return true;
    }

    return false;
}

This will also take care of calling `SetStdHandle` for you!

Problem

The title says it all. When I run the following code: ``` HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE); HANDLE hFile = CreateFile(TEXT("Foo.txt"), GENERIC_WRITE, FILE_READ_ACCESS | FILE_WRITE_ACCESS, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); SetStdHandle(STD_OUTPUT_HANDLE, hFile); std::cout << "Hello, "; printf("world!\n"); WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), "Hello, world!\n", 13, NULL, NULL); SetStdHandle(STD_OUTPUT_HANDLE, hOut); CloseHandle(hFile); ``` The result is that `Hello, world!` gets written to the console as a result of the calls to `cout` and `printf`, and `Hello, world!` also gets written to the file `Foo.txt` as a result of the call to `WriteFile`. My assumption is that when everything gets initialized at the very beginning, the `HANDLE` returned by `GetStdHandle` is cached and reused for both `cout` and `printf`. That's perfectly reasonable and exactly what I would want as I assume `GetStdHandle` requires a call to the operating system (which can be long!). The trouble is that I want to override that behavior and "sync" both cout and printf with the application's standard handles if possible. Before suggesting any alternatives, let me describe exactly what it is I am trying to do (yes, I know that it is possible to use `freopen` for this purpose). What I need to be able to do is to "save" the current standard output handle on a stack-like data structure before I change it so that I can be able to restore the previous output handle. Anything short of that is unacceptable for this situation (i.e. I can't restore to `CONOUT$`, etc.). This needs to have the ability to be recursive. I.e. the following should work as you would expect it to: ``` std::cout << "A1" << std::endl; StartStdOutRedirection(TEXT("Foo.txt")); std::cout << "B1" << std::endl; StartStdOutRedirection(TEXT("Bar.txt")); std::cout << "C1" << std::endl; EndStdOutRedirection(); std::cout << "B2" << std::endl; EndStdOutRedirection(); std::cout << "A2" << std::endl; ``` This would be excessively easy if there was a way to "re-sync" `stdout` as the following code should do the trick: ``` std::vector<HANDLE> vStdOutHandles; void StartStdOutRedirection(_In_ LPCTSTR lpFile) { vStdOutHandles.push_back(GetStdHandle(STD_OUTPUT_HANDLE)); SetStdHandle(STD_OUTPUT_HANDLE, CreateFile(lpFile, GENERIC_WRITE, FILE_WRITE_ACCESS | FILE_READ_ACCESS, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL)); } void EndStdOutRedirection(void) { CloseHandle(GetStdHandle(STD_INPUT_HANDLE)); SetStdHandle(STD_OUTPUT_HANDLE, vStdOutHandles.back()); vStdOutHandles.pop_back(); } ``` The correctness of the above code can be verified by using `WriteFile` with a call to `GetStdHandle(STD_OUTPUT_HANDLE)` in place of `cout`. What I ideally need is an equivalent of `freopen` that works on `HANDLE`s. That way I could use `DuplicateHandle` on the `HANDLE` returned by `GetStdHandle` and then this `MyReopenHandle` function to set the underlying file for that `HANDLE` to the file of my liking. I believe that would work as I assume both `printf` and `cout` have a `HANDLE` saved somewhere deep down. I tried to "fake it" by duplicating the standard output handle, closing that handle, and then calling `CreateFile` in hopes that it'll give me the same `HANDLE` value, but that works sporadically at best. Here's my code for that if you are interested: ``` std::vector<HANDLE> vStdOutHandles; bool StartStdOutRedirection(_In_ LPCTSTR lpFile) { bool fResult = false; HANDLE hProc = GetCurrentProcess(); HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE); if (hOut != INVALID_HANDLE_VALUE) { HANDLE hDup; if (DuplicateHandle(hProc, hOut, hProc, &hDup, 0, FALSE, DUPLICATE_SAME_ACCESS)) { // Need to close the current handle before we open the new one CloseHandle(hOut); HANDLE hFile = CreateFile(lpFile, GENERIC_WRITE, FILE_WRITE_ACCESS | FILE_READ_ACCESS, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); if (hFile != INVALID_HANDLE_VALUE) { // Should be same HANDLE; else we're screwed... assert(hFile == hOut); SetStdHandle(STD_OUTPUT_HANDLE, hFile); vStdOutHandles.push_back(hDup); fResult = true; } else { // Otherwise, reopen the previous output HANDLE on failure DuplicateHandle(hProc, hDup, hProc, &hFile, 0, FALSE, DUPLICATE_SAME_ACCESS); assert(hFile == hOut); CloseHandle(hDup); } } } return fResult; } bool EndStdOutRedirection(void) { bool fResult = false; HANDLE hProc = GetCurrentProcess(); HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE); if (hOut != INVALID_HANDLE_VALUE && vStdOutHandles.size() != 0) { HANDLE hDup; HANDLE hNext = vStdOutHandles.back(); // Close current handle and re-open previous one CloseHandle(hOut); if (DuplicateHandle(hProc, hNext, hProc, &hDup, 0, FALSE, DUPLICATE_SAME_ACCESS)) { // Again, we're screwed if these are not the same assert(hOut == hDup); SetStdHandle(STD_OUTPUT_HANDLE, hDup); vStdOutHandles.pop_back(); fResult = true; } } return fResult; } ``` The above assert fails about half the time (I wasn't really expecting or counting on that to work... I was just interested). That is about as far as I have gotten in terms of this problem. If anyone has any suggestions, please let me know :)

Original source