Terminal: Support querying the colors via OSC

[egmontkob]

OSC 4, 10, 11 and friends, with a ? instead of a color, respond with the current value of that color in xterm, vte and quite a few other terminals, always using the rgb:RRRR/GGGG/BBBB notation, and the same terminator (ST vs. BEL) as in the query.

Examples:

echo -ne '\e]4;1;?\e\\'; cat

echo -ne '\e]11;?\a'; cat

This allows apps (e.g. vim) to automatically pick a color scheme depending on whether the user has a dark-on-bright on bright-on-dark color scheme. I don't know if it's actually being used in practice, though :D

[j4james]

I should probably raise a separate issue for this, but I just wanted to note that there are standard DEC sequences for querying the color table (and setting it for that matter) which we may want to support as well.

[skyline75489]

Note that VTE responds with rgb:RRRR/GGGG/BBBB format. But iTerm2 responds with rgb:RR/GG/BB format. I don't have a daily Linux machine so I don't know what xterm is using.

[DHowett]

I just tested urxvt and it responded with rgba:0000/0000/0000/f332 :)

[skyline75489]

Well that's awkward. So I have to assume that XTerm has support for rgba: (and at the same time,rgbi, according to @egmontkob) ?

Personally I'd prefer rgb:RR/GG/BB since it's close to #RRGGBB which is everywhere on the web.

[TBBle]

From the XParseColor docs (via the Xterm docs for OSC 4), the RR vs RRR vs RRRR is about the bit-depth of the value. So rgb:RR/GG/BB is 24-bit colour (8/8/8/0), and anything longer is really Wide Colour Gamut, and should be scaled down. (Hence things like #302351 being rgba:3000/2300/5100/ee00 in urxvt tips-and-tricks, assuming a linear tone-mapping function.

That said, I guess we don't have native WCG support in Terminal right now, and I expect we'd return the 24-bit colours we are working with, not the tone-mapped resulting colours if we happen to be running on a WCG-based display.

So I also support rgb:RR/GG/BB as long as we're working from a 24-bit palette.

I'm guessing rgba: is a common extension for alpha... It's not in the XParseColor docs, and I remember back in the day, the best reason to use rxvt over xterm was transparent backgrounds... Perhaps rgba: is (or was) an rxvt-specific extension?

Looks like it was rxvt-specific, and unlike rgb:, rgba: is specified to always have 16 bits _per channel_. Someone was very forward-thinking in the 90's... Or lazy when writing a string parser.

[j4james]

I've tested the color table queries on 9 different terminal emulators - every one of them (including XTerm , Gnome Terminal, Konsole, and Rxvt) reported the colors as rgb:rrrr/gggg/bbbb (typically scaled up from the internal 24-bit representation). I see in the Rxvt source that there is a build option that will make it return rgba, but that didn't seem to be active on my test system - perhaps it's OS dependent? In any event, I'd think the rgb:rrrr/gggg/bbbb format is almost certainly the right choice if we want to be compatible with what most people are doing.

Regarding the other color formats, XTerm does support rgbi and the various cie variants, but the report format is always rgb.

[skyline75489]

Out of sheer curiosity, what is the reason that makes iTerm2 the weirdest kid in town?

CC @gnachman

[j4james]

I believe iTerm2 has an option to choose between reporting as rgb:rrrr/gggg/bbbb and rgb:rr/gg/bb. They got it wrong initially, and then were forced to add the option to support the correct format because it was breaking in vim.

[gnachman]

I used 8 bit codes originally for reasons I no longer recall. I was made aware of the issue in this post: https://groups.google.com/g/iterm2-discuss/c/xXzeBY2aRjo/m/zHBZreQWCQAJ

Commit 5c5785f3632b8e90dd69f458411a8b8b17aa0599 changed the default to 16 bits. It is now enabled in 3.4.0 betas, which will exit beta when Big Sur is released.

[TBBle]

To be clear, per the spec, _both_ rgb:rr/gg/bb and rgb:rrrr/gggg/bbbb are valid. So is rgb:rrr/ggg/bbb, and a few other options. The item2-discuss link from @gnachman shows that it was a bug in vim before 8.1, that it incorrectly implemented the XParseColor-compatible colour parser.

That said, the commonality of vim, particularly older vim, means it's probably worth being bug-compatible here from the start, even though it's being overly-precise.

My reading of the Xterm docs for OSC 4 is that if you call OSC 4 XXXX, then OSC 4 ? should be returning precisely that XXXX, but I can see how it can be read to allow returning some numerically-equivalent values, i.e. 48-bit RGB, even if the XXX was originally CIELab:....

[j4james]

My reading of the Xterm docs for OSC 4 is that if you call OSC 4 XXXX, then OSC 4 ? should be returning precisely that XXXX

Yeah, it certainly does read like that. But in practice the spec is defined by the reference implementation, which in this case is XTerm. If you want to be "spec compliant" you need to match whatever XTerm does. And when you think about it, it makes sense for your terminal to be liberal in what it accepts (i.e. supporting as many formats as possible), but conservative in what it produces as a response (you can't reasonably expect every application that queries the palette to understand all the different formats).

[TBBle]

That's fair. And given that (for an older version, I didn't check anything newer) xterm OSC 4 n ? produces XQueryColor formatted as rgb:%04x/%04x/%04x, that seems a safe thing to do.

you can't reasonably expect every application that queries the palette to understand all the different formats

I agree in general. In this case however, because it's not "all the different formats", it's a format defined by a specific API (XParseColor), I would absolutely expect of myself that a parser was prepared to handle those documented formats. Apps on X11 of course have it easy, because they can just _use_ XParseColor, but I'm unaware off-hand of anyone knocking together an X11-independent implementation of the API, which is a shame, but I guess the OSC colour codes are the only, highly-specific use-case for such a thing.

It may not be _kind_ to specify an API behaviour in terms of the API it's implemented with underneath, but that is what we have here. (Edit: If anyone's looking for the implementation side of XParseColor, the colour space parsing is farmed out to functions pointed to by the struct XcmsColorSpace instances, and libXRender separately defines rgba: in XRenderParseColor).

That said, Microsoft Terminal cannot _consume_ rgb:rrrr/gggg/bbbb for OSC 4 right now, so we probably need to fix that, or we end up in the situation where we cannot roundtrip our OSC 4 n ? back to OSC 4 x <string we saved earlier>.

Our local implementation of XParseColor could do with some love too.

[skyline75489]

I’m working on it. See #7578 if you have time 😄