Julia: Fix color support on Windows via VT sequences

[davidanthoff]

This is a follow up on https://github.com/JuliaLang/julia/issues/7267#issuecomment-476935384. The issue is that right now that @KristofferC's https://github.com/KristofferC/Crayons.jl package doesn't print colors properly on Windows, because the julia process on Windows doesn't configure proper VT support on newer versions of Windows 10.

I'm sketching what I found so far, but don't have the time to fix this. I believe this needs to be fixed somewhere either in libuv or julia, not sure. But maybe what I found so far is helpful for doing that.

I started out with starting julia on Windows 10 via the start menu link. I then ran:

julia> hOutput = ccall(:GetStdHandle, Int32, (Int32,), -11)
88

julia> dwMode = Ref{UInt32}(0)
Base.RefValue{UInt32}(0x00000000)

julia> ccall(:GetConsoleMode, Int32, (Int32, Ptr{Nothing}), hOutput, dwMode)
1

julia> dwMode[]
0x00000003

The first thing to note is that by default the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag is not set for this process, which means that none of the new VT support in Windows 10 is active.

Alright, lets set that flag (the value of it is 0x004):

julia> newMode = dwMode[] | 0x0004
0x00000007

julia> ccall(:SetConsoleMode, Int32, (Int32, Int32), hOutput, newMode)
1

If I now call

julia> using Crayons

julia> Crayons.test_256_colors(false)

things still don't look good. BUT, if I do the following, I get the correct output:

julia> buf = IOBuffer()

julia> Crayons.test_256_colors(buf, false)

julia> tr_buf = transcode(UInt16, String(take!(buf)))

julia> ccall(:WriteConsoleW, Int32, (Int32, Ptr{UInt8}, Int32, Ptr{Nothing}, Ptr{Nothing}), hOutput, tr_buf , length(tr_buf ), C_NULL, C_NULL)

I'm not sure what is going on, but it looks like somehow that print and friends don't send that string in such a way to the console that the VT sequences make it there. Maybe libuv is intercepting these sequences in some form? I have no idea, but clearly whatever print is doing is preventing things from working as they should in this case.

I believe ENABLE_VIRTUAL_TERMINAL_INPUT is also not set by default. I don't think that matters for the color example, but it probably means that some other VT feature that is in principle supported on Windows is not active when julia runs.

[davidanthoff]

Maybe libuv is intercepting these sequences in some form?

I poked around a bit in the libuv code, and that seems to be the explanation. If I understand it properly, then libuv is actually intercepting VT sequences that one writes to the console and tries to translate them into Windows Console API calls. So these VT sequences never make it out of julia/libuv! I think for newer Windows versions the proper strategy is probably to just skip this custom tty implementation in libuv entirely, and instead set ENABLE_VIRTUAL_TERMINAL_PROCESSING for the julia process and let the windows console handle these VT sequences.

[mgkuhn]

Sounds like something that should be fixed upstream in libuv: if the Windows version is sufficiently new (e.g., 1809), then libuv should make use of the new ConPTY architecture, that is stop using the old Windows Console API, and instead pass on UTF-8 + VT100 codes directly to stdout, just as on Unix.

[davidanthoff]

Yes, agreed. I think on recent Windows versions one could also get rid of the whole UTF-8 to UCS-2 conversion that is happening in libuv on Windows, one can just pass UTF-8 strings directly to WriteConsoleA on modern windows system, as long as one sets the right code page for the console, see here.

CCing @vtjnash because he seems the resident expert on these issues.

[vtjnash]

Yeah, currently we're blocked on several libuv bugs with VTP handling (libuv/libuv#2129 and libuv/libuv#1965). I've lost track of what their status is. But if someone wants this to happen and can help push those over the line, we can maybe get this turned back on.

[musm]

So to be clear this is all upstream problems in UV and not julia related?

[davidanthoff]

That seems a fair assessment, as far as I can tell.

I think libuv just is not hooking into any of the new console improvements that have come to Windows lately. On a modern Windows, it really shouldn't need to handle TTY stuff itself, and it probably also shouldn't do any of the string encoding conversions. I suspect that the code in libuv could look a lot more like the one for the other unix systems on modern Windows systems.

The only thing in julia that I found that one probably doesn't want on modern Windows is this, assuming one just uses the UTF-8 versions of the API directly.

I don't understand this. It might be a way to disable the whole TTY handling that libuv provides on Windows? But I'm not sure... And in any case, it seems better if this would be fixed in libuv directly.

[musm]

What's the advantage or why are we using libuv for tty ? In order to resolve this issue, should we just wait until libuv fixes the problem or try to fix the issue by circumventing libuv and addressing the problem by handling tty stuff ourselves?

[musm]

https://github.com/JuliaLang/julia/blob/c44644442949238da70670b547312ca1ae9a9c7d/src/support/libsupportinit.c#L16 I wonder why this is set, it doesn't seem necessary to have a functioning repl

[mgkuhn]

So far, Windows console input/output has been very different from Unix (using a C Console API rather than VT100 sequences, no pty, forcing terminal emulators and sshd to screen-scrape a character matrix, no UTF-8 support), and libuv had to work hard to emulate Unix terminal semantics under Windows in a (limited) portable way. However, in 2018, Microsoft finally saw the light (a result of their efforts to create WSL and port OpenSSH to Windows), and they now aim to make the Windows 10 console work much more like xterm on Unix, with a lot of Microsoft-specific backwards-compatibility mechanics (ConPTY). Once that is completed, and Julia drops support for older (i.e., pre-2019) Windows and terminal versions, libuv should no longer be needed for tty output, but for a while the Windows console may remain a moving target. Initial ConPTY support came with Windows 10 1809, with more improvements and bug fixes expected in the 2019 releases.

[davidanthoff]

Probably best to just hope this gets addressed in libuv? It could simply detect at runtime what Windows version is running, and then, depending on that activate all its emulation (for old Windows versions) or not. I assume Julia will want to support older versions of Windows for a while, and it seems easiest if handling that complexity is happening in libuv.

[davidanthoff]

Fun fact: the julia fork of libuv just blanket disables support for this here :)

What I don't understand is why the julia fork took that route, instead of just applying https://github.com/libuv/libuv/pull/1965 on the julia fork, while at the same time leaving the existing libuv support for VT sequences on modern Windows system enabled?

@vtjnash As far as I can tell, all the pieces that are needed to make this work actually exist, the only thing we would need to do at this point is make two changes to the julia fork of libuv: a) undo https://github.com/JuliaLang/libuv/commit/587ebd34097f6bc9c674a215af9e7ec6ae7b087c and b) apply https://github.com/libuv/libuv/pull/1965 to the julia fork of libuv.

As far as I can tell https://github.com/libuv/libuv/pull/2129 really doesn't matter for any of this. It would provide a simple API to control some of this behavior, but as far as I can tell we wouldn't actually need that for julia.

[davidanthoff]

I also spent some time investigating what we would gain if our libuv just passed these VT sequences through.

1) it would fix the color issues in the default Windows terminal.
2) I tried out to alternative terminals FluentTerminal and Terminus. Both seem to use conpty, and for both the color issues is solved when the VT sequences are just passed through.
3) When I use VS Code Insider on Windows Insider, the color problems are solved in the integrated REPL. I think that is explained by the fact that VS Code is only using conpty on Windows 18309+ right now.
4) It wouldn't fix ConEmu. I think that is because conpty support hasn't landed there yet (but apparently they are working on it).

So my sense is that the various terminals on Windows are all in the process of moving over to conpty, and as they do, fixing this on the julia side would smooth things considerably for a large number of terminals.

[musm]

@davidanthoff thank you for spearheading this and the great examples / discussion

With the following patch to libsupportinit.c

// This file is a part of Julia. License is MIT: https://julialang.org/license

#include <locale.h>
#include "libsupport.h"
#ifdef _OS_WINDOWS_
#include "windows.h"
#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
#endif
#endif

#ifdef __cplusplus
extern "C" {
#endif

static int isInitialized = 0;

void libsupport_init(void)
{
    if (!isInitialized) {
#ifdef _OS_WINDOWS_
        HANDLE hConsole = { GetStdHandle(STD_OUTPUT_HANDLE) };
        DWORD consoleMode = 0;
        GetConsoleMode(hConsole, &consoleMode);
        SetConsoleMode(hConsole, consoleMode | ENABLE_VIRTUAL_TERMINAL_PROCESSING);
#endif
        setlocale(LC_ALL, ""); // set to user locale
        setlocale(LC_NUMERIC, "C"); // use locale-independent numeric formats

        ios_init_stdstreams();

        isInitialized=1;
    }
}

#ifdef __cplusplus
}
#endif

We then fix the issue of having the wrong Console Mode set. In other words, we can directly run the following example without manually changing the console mode.

using Crayons
buf = IOBuffer()
Crayons.test_256_colors(buf, false)
hOutput = ccall(:GetStdHandle, Int32, (Int32,), -11)
tr_buf = transcode(UInt16, String(take!(buf)))
ccall(:WriteConsoleW, Int32, (Int32, Ptr{UInt8}, Int32, Ptr{Nothing}, Ptr{Nothing}), hOutput, tr_buf , length(tr_buf ), C_NULL, C_NULL)

[davidanthoff]

@musm I think instead of adding more patches here in julia, we should just _undo_ the patch in julia's fork of libuv that disables setting the new VT mode for the console. Upstream libuv does properly set the console mode, as far as I can tell, it is just the julia fork of libuv that disables that.

I think there was a good reason for that: late in the 1.0 release process a bug in the UTF-8 processing of libuv was discovered, and disabling the VT mode was probably easier/safer than fixing that bug. But at this point, I think one could undo that, and instead carry the UTF-8 bug fix for libuv (https://github.com/libuv/libuv/pull/1965) that @vtjnash actually created in the julia fork.

[musm]

@davidanthoff Ok great I got color support to work in Julia

Modified the libuv as required and a minor tweak to libsupport_init

[musm]

BTW @davidanthoff are you sure FluenTerminal properly handles things? I tried to run the same code in FluentTerminal yet It does not display all the colors as in CMD.exe.

[davidanthoff]

Did you change the setting in FluentTerminal to use conpty instead of winpty? By default it doesn’t.

[musm]

Ok thanks, it's still a little janky compared to CMD. I'm not worried about though since Fluent feels very beta software at this point.

[musm]

BTW I opened a PR against our version of libuv at https://github.com/JuliaLang/libuv/pull/2
After that is reviewed a minimal PR in julia will be required to support 256 colors. I'm not sure if a conditional for older versions of windows will be required in the patch to libsupportinit

If anyone is interested in being a ginnie pig and test the new julia build, here is a dropbox link to the installer https://www.dropbox.com/s/w4zr2tkldz15f12/julia-a62b8ec9c6-win64.exe?dl=0. I want to confirm that older versions of Windows that don't support the VT term sequences still render unicode correctly. I.e. it would be good to confirm if any Windows 7 users could confirm the build is working fine.

[KristofferC]

How about Crayons.test_24bit_colors(false)?

[musm]

[musm]

I'd love to fix this, but it's beyond my hacking scope. The images posted above look good, but https://github.com/JuliaLang/libuv/pull/2 shows that there are issues with resetting the terminal state that neither I nor @davidanthoff have been able to figure out.

If anyone has any ideas here it would be appreciated.

[StefanKarpinski]

@vtjnash, any chance you could be of assistance here? It would be great to fix Windows terminals.

[mgkuhn]

Is it practical for Julia to limit Win32 terminal support to just the very latest version of Windows 10? That might soon make it possibly to just remove any Windows-specific terminal code, i.e. bypass libuv entirely for terminal I/O (and make sure that everything works just as well via OpenSSH for Windows).

[davidanthoff]

@mgkuhn I think libuv essentially provides a layer that makes it easy to support both old and new Windows 10 versions. And upstream libuv actually automatically switches between the "new" API (where Windows interprets the VT sequences) and its own VT emulator layer, depending on which version of Windows one is running on. The issue is just that the julia fork of libuv disables support for that (due to another bug, that I believe is now fixed). So I think the least effort way to resolve this here is actually to just stick with libuv, and make sure the existing support for the new Windows API that it provides is used by julia.

[davidanthoff]

This suggests that if we fix this julia issue things will improve in VS Code as well.

[davidanthoff]

I just tried the code that I posted in the original issue here at the top with the new Windows Termainl. As expected, everything is the same: if Julia were to fix how it emits VT sequences and were to configure things correctly, we would have full color support on Windows.