Terminal: Korean IME does not work as expected

[rkttu]

Environment

Windows build number: Windows 10 2004 19041.21
Windows Terminal version (if applicable): 0.8.10091.0

Any other software?

Steps to reproduce

• Run Windows PowerShell tab
• Typing text $x = '한글'

Expected behavior

• The resulted text should be $x = '한글'

Actual behavior

• But the resulted text was $x = '한그ㄱ글'

[yjh0502]

The bug breaks my daily workflow :(

[zadjii-msft]

Thanks for the bug report! @rkttu /@yjh0502 Does this repro in a legacy console (pwsh.exe) window? Or does this only happen in the Windows Terminal?

[rkttu]

@zadjii-msft Currently it only happens in WT, but legacy console's behavior also weird. It does not display character compositions. For example, to make a character '한', user can typing the key 'ㅎ', 'ㅏ', and 'ㄴ'. Usually, this sequence displayed like 'ㅎ' -> '하' -> '한', but currently it displays just a complete character directly (ex. '' -> '한').

FYI, I'm using Dubeolsik (두벌식) Korean IME.

[cupsos]

@zadjii-msft Same issue here. Terminal with WSL(Debian), pwsh, powershell, cmd not work expectly. //'한그ㄱ글'

Legacy WSL(Debian), pwsh, powershell, cmd can type properly, //'한글'
but not showing character compositions

[zadjii-msft]

This is another great repro gif from @juhokang in #4311

Windows build number: Microsoft Windows [Version 10.0.18362.592]
Windows Terminal version (if applicable): Windows Terminal (Preview) Version: 0.8.10091.0

ex) Type 안녕하세요 with the keyboard

Expected behavior

show 안녕하세요 on the terminal

Actual behavior

shows 안녀ㄴ녕하ㅎ하셍세세요요

[ioklo]

FYI, expected behavior during composition.

[hatsunearu]

https://github.com/microsoft/vscode/issues/89853 Possibly similar issue happening in VS Code

[rkttu]

@zadjii-msft Is there any progress on this issue? It seems like quite a lot of people encountered this problem.

[zadjii-msft]

@rkttu Nope, when there _is_ progress, someone will make sure to chime in on this thread. It's been triaged as a P1 bug for 1.0, so we won't be shipping 1.0 without a fix for this, so stay tuned. If anyone is particularly passionate about this bug, we'd be happy to review a PR. Until someone's been _assigned_ to this bug, you can be sure you won't be stepping on our toes ☺️

[hatsunearu]

Possible workaround (not tested): try removing Droid Sans Fallback from the fonts list of the app if it's there.

https://github.com/microsoft/vscode/issues/89853#issuecomment-581495374

[simnalamburt]

Note that this problem did not occur in version 0.7.3451.0.

[simnalamburt]

Possible workaround (not tested): try removing Droid Sans Fallback from the fonts list of the app if it's there.

microsoft/vscode#89853 (comment)

@hatsunearu Unfortunately https://github.com/microsoft/vscode/issues/89853 is caused by wrong glyph rendering and that workaround dosn't work here.

[simnalamburt]

@guswns0528 discovered that this bug is first caused by https://github.com/microsoft/terminal/commit/dfa7b4a1. https://github.com/microsoft/terminal/commit/dfa7b4a1 itself is right commit so we can't simply revert it.

Looks like this issue is caused by strange behavior of Core text APIs. Originally, the Composition complete event should only be fired when the letter composition is completed. Like this:

• If you are typing ㅎ ㅏ ㄴ ㄱ ㅡ ㄹ, composition event should be fired twice.
  한, and 글

But in here it's fired before that. Like this:

• If you type ㅎ ㅏ ㄴ ㄱ ㅡ ㄹ in here, composition event is fired three times.
  한, 그, and 글

It might be a bug of Core text APIs but core text API is just a wrapper of Text Services Framework, which is super stable framework inheritted from Windows XP era. So I need to investigate further.

I'm currently debugging this issue and I'll comment on here if I find something. Please feel free to share any information if you find something. Thanks

[DHowett-MSFT]

@simnalamburt thanks! Note that @leonMSFT is currently working in this area and has a couple pull requests out for this; perhaps you two can coordinate?

[simnalamburt]

Cool! Currently I am suspecting wrong parameter of NotifyTextChanged function as the cause of the bug. But in fact, since I first saw the Core text API today, I still don't know what's wrong. Any help or information is always welcomed!

[leonMSFT]

@simnalamburt thanks a lot for the investigation! I tried to see if I can repro the two vs three composition completed events issue, but I only see two composition completed events. Maybe you could provide a screenshot of the debugging logs you used to find this? Update: I just found this out, but you're likely seeing multiple composition completed events after the first character is finished because when we call NotifyTextChanged to reset the text server's buffer, it'll fire _another_ composition completed event. However, since we have the if-statement to check if there's anything in _inputBuffer, we don't do anything on our side.

I was also taking a look at this bug earlier, and perhaps my findings could explain why you're seeing the bug you're seeing. Here's the behavior I'm observing and why I _think_ we're messing up Korean IME input:

So, going through your example keysequence, pressing ㅎㅏㄴ, would result in three TextUpdated events being received, with the _inputBuffer and the _textBlock having the character 한.

Now the user presses ㄱ, and what will happen is the following:

1. TextUpdated is received with the character 한.
2. CompositionCompleted is received, signaling that 한 is the finished composition.
3. As part of our CompositionCompletedHandler, we send the contents of _inputBuffer (which is 한) to the terminal and reset the _inputBuffer and _textBlock. Then we also notify the text server that they should also make their "buffer" empty as well. (This is the NotifyTextChanged call that you mentioned).

What should happen now is that we should receive a TextUpdated event with the text as 한ㄱ.
However, we're actually not getting any TextUpdated events after our CompositionCompletedHandler is finished _because_ we're telling the text server to reset their text buffer. Since their buffer is empty, it won't tell us that we should update our text to be anything.

This is why you'll run into the weird issue where you'll be pressing ㅎㅏㄴ, which works fine, and you'll see 한 on the screen, but once you make another input, like ㄱ, nothing happens. The ㄱ keypress triggers the CompositionCompleted event for the previous character 한, which tells the text server to clear their buffer. So, you will need to press ㄱ again to make ㄱ show up on the screen.

So, the core of the problem is that we need to send the IME input to the terminal when we believe composition is finished, and we naturally also need to clear our buffer whenever we send some input to the terminal. We also need to keep the text server's buffer and our _inputBuffer in sync, so whenever we clear our buffer, we tell the text server to clear theirs as well.

As a small test, I've tried commenting out the code where we're telling the text server to reset their text buffer, and lo and behold, text comes out as you would expect, without having to double-press any characters. The only problem here is that if we don't reset our text buffer, every CompositionCompleted event will cause us to send the whole _inputBuffer (which included literally everything you've ever typed while in IME mode) to the terminal, resulting in lots of duplicate input.

I'm currently trying to think of a way around this, but I'm giving you a summary of my findings so maybe you can also repro and investigate further to see if I've missed something! 😄

[simnalamburt]

@leonMSFT Wow thanks for your detailed explanation! Now I understand what was going on in my development environment.

Currently I’m trying to leave some unfinished characters in text buffer instead of totally clearing it in CompositionCompletedHandler.

Please share any information or updates and let me help anything I can! Actually there are lots of people waiting for this issue to be resolved since there are not much options that Korean developer can choose in Windows. Any share will be helpful and the whole Korean developer community will be grateful to you! 😄

[leonMSFT]

@simnalamburt Yup, not clearing the whole text buffer, but leaving unfinished characters in is key! Luckily, I think I'm close to getting the fix for this out! 🎉I'm specifically testing out trying to type out this sequence: 안녕하세요, which was provided earlier in this thread.

It _seems_ to work as expected, but before I have a PR out for this fix, I'll need to make sure I haven't messed up any other IME input modes, so hang tight! 😃

One thing I would like your help on is letting me know of other sample character sequences that might possibly break the way I'm handling Korean IME! I don't know Korean at all, so having the sequence laid out in english characters like it was above with "dkssudgktpdy" (which comes out as 안녕하세요) really helped!

[simnalamburt]

Wow it was very fast! I lost my chance to become hero lol

Actually there are not many corner cases in Korean IME. And your sample
video (안녕하세요) showed that it perfectly handles one of very famous corner
cases in Korean IME called “도깨비불 현상”.

If 안녕하세요 works perfectly I expect the other cases to work fine, but I’ll
share you few more samples just in case.

gksrnrdj whgdk (한국어 좋아)
To test whether aborting composition with space works fine

gksrnrdjEnterwhgdk (한국어\n좋아)
To test whether aborting composition with enter works fine

Actually there are bunch of things to test further like

• Test if alternative Korean IME like 세벌식 works fine
• Test if swiching IME in the middle of composition works fine (it’s very
  common for Korean people)
• etc

But these cases might be complicated to ask you to test so just share your
patch or make the draft PR. I have bunch of Korean and Japanese friend
developers interested in this issue and they will battle test it for you!

2020년 2월 29일 (토) 09:27, Leon Liang [e-mail]님이 작성:

[simnalamburt]

I tried to reproduce the scenario that you described, but I'm having trouble. I typed ㅎㅏㄴㄱ and I got 2 textUpdate event instead of 0 after typing "한".

# Typed ㅎ
_compositionStartedHandler() called.
_textUpdatingHandler() called.
Text:  ㅎ
Range: [0, 0]

# Typed ㅏ
_textUpdatingHandler() called.
Text:  하
Range: [0, 1]

# Typed ㄴ
_textUpdatingHandler() called.
Text:  한
Range: [0, 1]

# Typed ㄱ
_compositionCompletedHandler() called.
_SendAndClearText() called.
inputBuffer was L"한" and cleared.
_compositionStartedHandler() called.
_textUpdatingHandler() called.
Text:  ㄱ
Range: [0, 0]
_textUpdatingHandler() called.
Text:  ㄱ
Range: [0, 0]
_compositionCompletedHandler() called.
_SendAndClearText() called.
inputBuffer was L"ㄱㄱ" and cleared.

Is there anything that I misunderstood, or did something changed with https://github.com/microsoft/terminal/commit/31c9d19a72ef5fce19ab480c0a5e407064e15941#diff-7708ccd4133d008adca4935827f7ddb7?

https://github.com/simnalamburt/terminal/commit/acf74bc8ad947 this is a patch that I used for tracing.

[leonMSFT]

That's really weird! I pulled your branch from your fork (and the branch called patch-4226) and tried to do the same thing you were doing and this is what I'm getting:

After pressing ㄱ, as you can see, _textUpdatingHandler, _compositionCompletedHandler and _SendAndClearText are called in sequence, and another _compositionCompletedHandler is called afterwards. I'm not seeing the two extra _textUpdatingHandler calls that you're seeing though. 😢

[simnalamburt]

That's strange. My issue (2 text update event) is being reproduced consistently on two computers, mine and PC of @guswns0528. I wonder what the difference is.

My Windows specifications:

• Windows Terminal version: https://github.com/simnalamburt/terminal/commit/acf74bc8ad947
• Windows Edition: Windows 10 Pro
• Windows Version: 1909
• Windows OS build: 18363.657
• Processor type: 64-bit operating system, x64-based processor
• Windows display language: English (United States)
• Default app language, Default input language: 한국어

[leonMSFT]

From the details you've provided, I don't really see a difference 😢. However! I finally have a PR out, so feel free to take a look and play around with it!

[simnalamburt]

Me and @guswns0528 tried https://github.com/microsoft/terminal/pull/4796 and it worked perfectly! Still don't know why the 2 TextUpdate events were fired but looks like your patch fixed it anyway. 👍