Hyper: Hyper doesn't correctly handle variable-width characters

[wyqydsyq]

• [X] I am on the latest Hyper.app version
• [X] I have searched the issues of this repo and believe that this is not a duplicate

• OS version and name: OpenSUSE Tumbleweed
• Hyper.app version: 1.3.3
• Link of a Gist with the contents of your .hyper.js: https://gist.github.com/wyqydsyq/69b2dae557654799c91cfd6249ab10d7
• The issue is reproducible in vanilla Hyper.app: Yes

Issue

Hyper doesn't correctly handle variable-width characters, namely, emojis from the "noto-emoji" font. This issue is also prevalent in libvte and some Qt based terminal emulators. It works in the st emulator which apparently uses a different wcwidth implementation.

Here's a comparison between my same Tmux session looking broken in Hyper and appearing as it should in st:

You can also see that the spacing on the tmux pane title is messed up, strangely resizing the window sometimes fixes this unpredictably.

[rauchg]

Thanks for the report. I believe @tibotiber is currently looking into this, and his PR is very close!

[tibotiber]

@wyqydsyq would you mind trying the code on #1536 to confirm if it fixes your issue?

[revelt]

We've got a bit of a mixup: #1536 talks about more about powerline, but comments mention emoji as well. This issue is titled as to cover quite wide base, including plain emoji, but then talks specifically about emulators and issues shown are not regarding the characters but the UI of the emulator.

So, since emoji is "variable-width", it also falls into this issue, I mean things like:

I would help to test @tibotiber question above but I have no clue how to get the emulators in question set up. @wyqydsyq please follow up to Thibaut's question.

[Borkason]

Windows 10 version 1709, Hyper 1.4.8

Display problem with emoji's: 🎆🎆🎆🎆🎆

Notice the cursor when repeating the last command with ARROW KEY UP.

[wyqydsyq]

Sorry for the late reply, I stopped using Hyper due to this issue. I just tried the latest stable version which apparently has #1536 merged, however still get undesirable results. Screenshot attached shows a comparison with the display from Tilix which is how it should look.

[Stanzilla]

Should be https://github.com/xtermjs/xterm.js/issues/1059

[ghostrick]

similar problem occurred on v3.0.2

[jquast]

I actually used hyper as an example in an article I wrote about varying Unicode Version level support in Terminals (and a solution that works for python), https://jeffquast.com/post/terminal_wcwidth_solution/

The gist of it is that Hyper supports only version 5.1.0 of the Unicode standard for wide characters, through its depenency on the wcwidth library, whereas version 9.0 introduces the specification of Emoji's, from the article above,

Chinese, Japanese, and Korean users, whose languages use full-width characters almost exclusively have been the most impacted by this problem, whereas the Western world only seemed to notice it with the introduction of emoticons in Unicode version 9.0.

The problem is the depenency on the wcwidth library, that library does not support any newer version. I haven't yet found a javascript version that has been brought up-to-date (Latest Unicode specification is version 13.0)

best wishes !!

[LabhanshAgrawal]

@wyqydsyq @jquast can you check again with the latest ci build
I ran the ucs-detect tool with the new changes and it showed UNICODE_VERSION=12.1.0; export UNICODE_VERSION

[jquast]

Everything looks great now, thanks

[LabhanshAgrawal]

Everything looks great now, thanks

What is this tool you're using?

[jquast]

pip install wcwidth blessed

then, this is the script: https://github.com/jquast/wcwidth/blob/master/bin/wcwidth-browser.py