Wezterm: iTerm image support does not match mintty

[klamonte]

Hello, I was poking around the libsixel page and saw that wezterm was listed. Looks very nice, and the AppImage download was VERY convenient! :)

Alas, I did not see images, even when sent with sequences that will get images on screen with mintty. Hope this report can help you out...

Describe the bug

I was looking at wezterm's image support, and it does not produce the same results as mintty. (I cannot test against iTerm2 so do not know who is right vs wrong here.)

Environment (please complete the following information):

• OS: Debian Linux 10.4
• Version: please run wezterm -V and include its output here wezterm 20200608-110940-3fb3a61

To Reproduce

Launch wezterm, then from inside it:

wget https://jexer.sourceforge.io/sixel/iTerm2_7x15.cap
cat iTerm2_7x15.cap

Configuration

No idea, I just grabbed the nightly AppImage.

Expected behavior

Images on screen. The default font for wezterm is not 7x15 on my system, so the images might be munged in some way, but they "ought" to be there. Assuming mintty's understanding of iTerm2 protocol is the same as yours and iTerm2's.

Screenshots

If run with a 7x15 font, it should look like one of the images at https://jexer.sourceforge.io/sixel.html .

[wez]

I can confirm that I don't see images from that capture on my system, except briefly once I saw them in a flash. Also, FWIW, cat'ing that same file under iTerm2 is super super slow; so slow that I didn't think it was doing anything at all, but after a while the images started to show up.

[wez]

Another thing that you may wish to try is wezterm imgcat <filename> in both wezterm and mintty; the imgcat subcommand just emits the OSC 1337 sequence for an image upload. That might add a datapoint about how both handle that situation.

I'll look at this in more detail after work.

[klamonte]

When running against Jexer itself, the following was also logged by WezTerm:

 2020-06-08T21:02:39.419Z ERROR term::terminalstate > unhandled Window CSI ReportCellSizePixels
 2020-06-08T21:02:39.419Z ERROR term::terminalstate > unhandled Window CSI ReportTextAreaSizePixels
 2020-06-08T21:02:39.419Z ERROR term::terminalstate > unhandled DecPrivateMode 1005
 2020-06-08T21:02:39.419Z ERROR term::terminalstate > unhandled DecPrivateMode 1036
 2020-06-08T21:02:39.419Z ERROR term::terminalstate > unhandled DecPrivateMode 1034
 2020-06-08T21:02:39.419Z ERROR term::terminalstate > unhandled DecPrivateMode 80
 2020-06-08T21:02:39.419Z ERROR term::terminalstate > unhandled DecPrivateMode 1070
 2020-06-08T21:02:42.825Z ERROR wezterm::mux        > read_pty EOF: tab_id 0
 2020-06-08T21:02:42.825Z ERROR wezterm::localtab   > is_dead: 0
 2020-06-08T21:02:42.825Z ERROR wezterm::mux        > removing window 0

Don't know if any of those might be part of the issue, because all of the unhandled DecPrivateMode's will be in the capture file too. (I strip out the ReportCellSize / ReportTextAreaSize codes when making the captures.)

[wez]

It's an integer division issue; internally the terminal considers there to be a lot of 7x15 pixel images. If your font height happens to be larger than 15 pixels we end up considering those images to be 0 cells heigh, so it ends up not rendering the images even though it is carrying the data through the model :-p

You should be able to see the images if you create ~/.wezterm.lua with these contents, which specify a smaller font size:

return {
   font_size = 8.0
}

(the font size is measured in points so the dpi can impact the size that you need to make the images show up).

I'll look at making this less surprising, but hopefully that unblocks what you were originally planning to look at!

[wez]

PS: you may also wish to raise the output flooding throttle when testing lots of, or big, images:

return {
   font_size = 8.0,
   ratelimit_output_bytes_per_second = 100000000,
}

[klamonte]

Wow, that works! Thank you. I will be adding WezTerm to my terminals list soon.

Two more questions:

1. Does WezTerm expose its iTerm2 image support somehow to the application through a sequence? I notice its DA1 response contains only 6 (selective erase). (Actually, I can't find any documented means of checking for iTerm2 images, so requested one on iTerm2: https://gitlab.com/gnachman/iterm2/-/issues/8940 )

2. I notice that mouse is not working, despite Jexer requesting SGR mouse protocol. Jexer requests mouse reporting with: "\033[?1002;1003;1005;1006h" : button-event + any-event + UTF-8 mode + SGR mode. This is to support terminals that don't have any-event, and/or don't have SGR: they will fallback to button-event and/or UTF-8. But WezTerm seems to be ignoring the 1006 request, and I end up with no mouse events. Is this a configuration thing I can change with WezTerm?

BTW-Really impressed with WezTerm's image performance.

[wez]

1. Does WezTerm expose its iTerm2 image support somehow to the application through a sequence? I notice its DA1 response contains only 6 (selective erase). (Actually, I can't find any documented means of checking for iTerm2 images, so requested one on iTerm2: https://gitlab.com/gnachman/iterm2/-/issues/8940 )

I didn't find a way to signal this to the app either. In termwiz, which is my nascent terminal client library for rust, I use the following logic to inspect the TERM_PROGRAM and TERM_PROGRAM_VERSION environment and activate based on that. It is imperfect for lots of reasons:

https://github.com/wez/wezterm/blob/master/termwiz/src/caps/mod.rs#L255-L272

I haven't put much thought into DA1 so far as I wasn't aware that many apps were using it. I'd appreciate any suggestions on you have on improving that aspect of wezterm!

1. I notice that mouse is not working, despite Jexer requesting SGR mouse protocol. Jexer requests mouse reporting with: "\033[?1002;1003;1005;1006h" : button-event + any-event + UTF-8 mode + SGR mode. This is to support terminals that don't have any-event, and/or don't have SGR: they will fallback to button-event and/or UTF-8. But WezTerm seems to be ignoring the 1006 request, and I end up with no mouse events. Is this a configuration thing I can change with WezTerm?

Oh, I suspect that bundling those modes together into the same escape is confusing wezterm and thus sounds like a bug. WezTerm supports SGR mouse; if you can try sending just the 1006h from your app for the sake of testing, it should work. I'll work up a fix for this!

BTW-Really impressed with WezTerm's image performance.

That's good to hear! I do want to revise how the output rate-limiting works so that we don't overly penalize images in the default configuration.

[klamonte]

Testing again just now, and noticed a few things:

• I believe the CSI 16 t response has the width/height reversed. Jexer is reporting "14x7" text cell size rather than 7x14. Those CSI responses are like "CSI Ps ; height ; width t".

• Running 'reset' shows "unhandled Code(FullReset)" in WezTerm's stderr, and does not restore the cursor visibility.

• I am getting mouse events (woo!), but not Any-Event mouse tracking (1003 mode). Is that something WezTerm is expected to support?

• Button-Event tracking does not appear to be reporting mouse release (button up) events.

[wez]

Testing again just now, and noticed a few things:

• I believe the CSI 16 t response has the width/height reversed. Jexer is reporting "14x7" text cell size rather than 7x14. Those CSI responses are like "CSI Ps ; height ; width t".

Whoops :) Thanks for reporting this; it was a latent bug! Should be fixed in b370c80

• Running 'reset' shows "unhandled Code(FullReset)" in WezTerm's stderr, and does not restore the cursor visibility.

Ah, interesting how that hasn't bubbled up for me so far! This should now be implemented in
9c99330

• I am getting mouse events (woo!), but not Any-Event mouse tracking (1003 mode). Is that something WezTerm is expected to support?
• Button-Event tracking does not appear to be reporting mouse release (button up) events.

These two are related; I was being lazy/principled and wanted only to implement SGR Mouse reporting encoding, so wezterm currently only generates mouse events when mode 1006 is activated, implicitly activating Button-Event but with SGR encoding when that mode is activated.

That means that it doesn't know about Any-Event or Button-Event modes distinct from SGR mode.
I can get this cleaned up and implemented correctly.

FWIW, button-up events are generated for me in SGR Mouse mode in eg: vim:
https://github.com/wez/wezterm/blob/master/term/src/terminalstate.rs#L420
I wonder if there is something else about your test scenario that we're missing in wezterm?

[klamonte]

I see what's going on with button release: WezTerm is always reporting button "3" release when only button 0 (say) is unpressed. What should happen instead is that a button release is exactly the same SGR sequence as a button press, except ending in "m" rather than "M", specifically to disambiguate which button is being released (which only SGR mode does).

WezTerm's behavior is the X10 mouse mode (and very similar to ncurses). I do see some patches floating around where other applications would treat button "3" almost the same as "all buttons released", and others where they look for the specific button to be released.

[wez]

I believe all of the issues you uncovered have now been addressed, so I'm closing this out.
I'd also like to note that wezterm now also has sixel support!
There have been some recent fixes to image handling in master, so if you feel inclined to play around, I'd recommend using a nightly build.