scico | Palettes for R based on the Scientific Colour-Maps | Data Visualization library

The new console implementation in Windows 10 has a bug in which high-level WriteConsole and WriteFile to a non-active screen buffer instead always writes to the active screen buffer. Low-level WriteConsoleOutput[Character] works correctly. Using the legacy console also works. You can enable the legacy console in the properties dialog.

Note that a process cannot use an inherited handle for a screen buffer in the parent's console if it allocates a new console due to the CREATE_NEW_CONSOLE flag. Trying to write to the screen-buffer file will fail because it's not bound to the caller's attached console (i.e. instance of conhost.exe).

Console files that are bound include "CON", "CONIN$", "CONOUT$", and a new screen buffer from CreateConsoleScreenBuffer. There are also unbound input and output console files, which are set as the standard handles when allocating a new console (e.g. via AllocConsole()). These handles access the input buffer and active screen buffer of any attached console [*]. Note that a process can have handles for console files that are bound to multiple consoles and can switch between consoles using AttachConsole.

Note also that some programs open "CONOUT$" instead of writing to the StandardOutput and StandardError handles, especially if they require a console instead of whatever the standard handles might be (e.g. a pipe or disk file). In this case it is not enough to set hStdOutput in STARTUPINFO. You have to temporarily make the new screen buffer active via SetConsoleActiveScreenBuffer. This does not affect the caller's standard handles. It sets the active screen buffer in the attached console, which is what "CONOUT$" opens. The previous screen buffer can be restored after the child process exits, or after you know the child has already opened and written to the new screen buffer.

[*] In Windows 8+, these console files are implemented by the condrv.sys device driver and opened on "\Device\ConDrv". They are respectively named "Console", "CurrentIn", "CurrentOut", "ScreenBuffer", "Input", and "Output". The filename for the console connection itself is "Connect". Internally the latter is opened as the process ConsoleHandle, which is used implicitly by the console API in some cases (e.g. GetConsoleWindow, GetConsoleCP, and GetConsoleTitle).