Zettlr: Weird behavior in search field using Korean character (possibly CJK)

[vlwkaos]

Description

When typing a Korean letter, letter is duplicated when there is a match for auto-completion.
See

Reproducing

I type "코", then a matching note titled "코로나" appears. Then I try to use the found match, so presse enter or direction keys. Then a previous letter "코" is replicated and the search field is now filled with "코코" not "코로나"

Expected behaviour

Auto-complete matching note names.

Screenshots

Platform

• OS and version: Windows 10 Home
• Zettlr Version: 1.6.0
• Screen Resolution: 1080p

Additional information

Only tested with Korean characters, so not sure about other Asian inputs.

[nathanlesage]

Might very well be related to these characters being double in width. Or maybe it has to do with the fact that non-latin scripts are written using an IME — @ryota-abe, do you have experienced something similar?

In either case, I'll just drop the corresponding code in here:

https://github.com/Zettlr/Zettlr/blob/b3d551d2b59625d6b332274369455a5ca867ba19/source/renderer/zettlr-toolbar.js#L71-L79

[vlwkaos]

So I spent time investigating and Japanese characters do reproduce this behavior too, at least on my computer. And it does seem like it has to do with characters being full-width.

A quick hack would be to change line 76 to select from length-1 if the last character from the string is a full-width character.

So I tried using wcwidth to check if the last character matches the width defined by POSIX.1-2001. This worked okay, but I am not sure if it is okay performance-wise or will not cause other problems in the future development. Let me know what you think.

if (e.setSelectionRange) { 
              let d = this._oldval.charAt(this._oldval.length-1);
              if (d.length === wcwidth(d))
                e.setSelectionRange(this._oldval.length, this._searchbar.val().length) 
              else
                e.setSelectionRange(this._oldval.length-1, this._searchbar.val().length) 
            } 

Edit:
So there is a weird situation that when Zettlr is first opened it uses IME, but it does not once I minimize it once. When using IME keyup function recognize it as a single key press and it works fine as is and the above solution does not work.

The thing is it is not possible to know when Windows10 decides to use IME(which by the way is not the preferred option for Koreans since it inputs the whole character only when its completed)

I want to make it work for the both situation. I think it is somehow related to this. I will further investigate the issue and fix it if possible... (not sure on that part though)

[ryota-abe]

It is reproducible in my environment as well, but I hadn't noticed this until now because I was using only ASCII characters for file names.

[vlwkaos]

So I was able to cook up a solution that works well for a Korean character, but not with Japanese. This is due to the difference in how IME handles the characters.

For example, Korean characters are typed as is, whereas Japanese (my guess is Chinese too) characters need to be converted before it completes the composition.

Another universal solution that works for all IME would be to check if a composition of a character is finished then render auto-completion. However, this breaks a fluent feel of the auto-completion that appears as keys are typed. So I will try to find a way to make it work for all CJK.

Will keep this updated, and possilby a PR..

[nathanlesage]

However, this breaks a fluent feel of the auto-completion that appears as keys are typed.

But, as far as I understand IMEs, their main thing is that you actually have two "layers" of text input, hence, first the layer only from the IME, and then secondly the normal, underlying text field. But fear no more, look at what I found:

https://developer.mozilla.org/en-US/docs/Web/API/Element/compositionend_event

Implementing this should work for all IMEs and all scripts.

Gimme one sec …

[nathanlesage]

And tadaaaaaaa! Just tested it out, it now works with all scripts! @pizzakinggod Feel free to check out the commit, just a few lines have changed, to see how it works! I learned something good today :) The isComposing-part was the one I needed to find the perfect solution for this, so thanks again!

[vlwkaos]

Just a suggestion, I tested the committed change and something's off.

https://www.fxsitecompat.dev/en-CA/docs/2018/keydown-and-keyup-events-are-now-fired-during-ime-composition/

Looking at this, the isComposing is set to true only between compositionstart and compositionend. So I think listening for compositionend will run _applyAutocomplete() twice

First at compositionend then at input{ isComposing: false}

And I think that is why something feel's off. At least on my end.. do you mind testing it further?

[nathanlesage]

Huh? When I tested it out, the event only ran once, after composition end, because after the composition ended, the event fired, but no keyup was triggered on my site.

[vlwkaos]

Hmm, weird. Anyhow, removing the listener for compositionend gives the expected behavior of typed characters + the rest of the name appear selected. But with it on, there is no select. Functionality wise, there is no difference though. Well, I think I can live with it. Thanks for looking into it!

[nathanlesage]

Anyhow, removing the listener for compositionend gives the expected behavior of typed characters + the rest of the name appear selected.

In other words: Without that thingy everything works as expected? I mean, then I can surely remove that thing again, I'm not bound to it 🤷🏼‍♂️

[vlwkaos]

Yes. without it, it works as expected... Weird🤔