Conemu: wrong cursor position when typing chinese

Created on 23 Feb 2017  ·  18Comments  ·  Source: Maximus5/ConEmu

Versions

ConEmu build: 170118 x64
OS version: Windows 10 x64 15031
Used shell version (cmd)

Problem description

after typing in chinese, the cursor stops at a wrong position:

qq 20170223113226

using default configuration with additional CJK font setting (Main console font: Console, Alternative font 微软雅黑).

Expected results

cursor should stops at where those charactors end.

drawing-cjk other-windows
Source
picture terrywh

Most helpful comment

for me 170517 got it fixed

picture terrywh on 26 May 2017
👍2

All 18 comments

Please show text data from About/SysInfo.
And output from "ConEmuC -checkunicode".

Maximus5 picture Maximus5 on 23 Feb 2017

SysInfo:

ConEmu 170118 [64] Startup Info
  OsVer: 10.0.15031.x64, Product: 1, SP: 0.0, Suite: 0x100, SM_SERVERR2: 0
  CSDVersion: , ReactOS: 0 (), Rsrv: 0
  DBCS: 1, WINE: 0, PE: 0, Remote: 0, ACP: 936, OEMCP: 936, Admin: 0
  StartTime: 2017-02-23 20:04:01.564
  AppID: 85a194077d2e839c4867ea0081ae8e4f::162
  Desktop: `Winsta0\Default`, SessionId: 1, ConsoleSessionId: 1
  Title: `C:\Users\wuhao.PANDATV\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\开发\ConEmu64.lnk`
  Size: {0,0},{0,0}
  Flags: 0x00000C01, ShowWindow: 1, ConHWnd: 0x00000000
  char: 1, short: 2, int: 4, long: 4, u64: 8
  Handles: 0x00000000, 0x00010001, 0x00000000
  Current PID: 10696, TID: 9480
  Active HKL: 0x0000000008040804
  GetKeyboardLayoutList: 0x0000000008040804
CmdLine: "D:\Program Files\ConEmu\ConEmu64.exe" 
ExecMod: D:\Program Files\ConEmu\ConEmu64.exe
WorkDir: D:\Program Files\ConEmu
PathEnv: C:\ProgramData\Oracle\Java\javapath;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;D:\Program Files\MacType;C:\Users\wuhao.PANDATV\AppData\Local\Microsoft\WindowsApps;
ConFont: 0  Lucida Console  00  Consolas    932 *ＭＳ ゴシック    936 *新宋体
CMD's AutoRuns: {not defined}
Foreground: x0055075A {741,107}-{1317,524} 'VirtualConsoleClass' - cmd
MouseCursor: {1386,574} MouseMonitor: 00010001 StartMonitor: 00010001
Display: bpp=32, planes=1, align=1, vrefr=60, shade=x00000000, rast=x00007E99, dpi=96x96, per-mon-dpi=1
Monitors (dpi: WholeDesktop, Effective, Angular, RAW):
  00010001: {0,0}-{1920,1080} (1920x1080), Working: {0,0}-{1920,1040} (1920x1040), dpi: {96,96};{96,96};{91,91};{92,92} `\\.\DISPLAY1` <<== Primary
  00010003: {1920,0}-{3840,1080} (1920x1080), Working: {1920,0}-{3840,1040} (1920x1040), dpi: {96,96};{96,96};{91,91};{92,92} `\\.\DISPLAY2`
Modules:
  00007FF6A0FD0000-00007FF6A1261FFF   292000 D:\Program Files\ConEmu\ConEmu64.exe
  00007FFB61C30000-00007FFB61E0BFFF   1DC000 C:\WINDOWS\SYSTEM32\ntdll.dll
  00007FFB619D0000-00007FFB61A7DFFF    AE000 C:\WINDOWS\System32\KERNEL32.DLL
  00007FFB5E4D0000-00007FFB5E712FFF   243000 C:\WINDOWS\System32\KERNELBASE.dll
  00007FFB611D0000-00007FFB61319FFF   14A000 C:\WINDOWS\System32\USER32.dll
  00007FFB5F170000-00007FFB5F18DFFF    1E000 C:\WINDOWS\System32\win32u.dll
  00007FFB619A0000-00007FFB619C6FFF    27000 C:\WINDOWS\System32\GDI32.dll
  00007FFB5E7D0000-00007FFB5E953FFF   184000 C:\WINDOWS\System32\gdi32full.dll
  00007FFB5E720000-00007FFB5E7C0FFF    A1000 C:\WINDOWS\System32\msvcp_win.dll
  00007FFB5E3D0000-00007FFB5E4C4FFF    F5000 C:\WINDOWS\System32\ucrtbase.dll
  00007FFB5F220000-00007FFB5F2C0FFF    A1000 C:\WINDOWS\System32\ADVAPI32.dll
  00007FFB60940000-00007FFB609DCFFF    9D000 C:\WINDOWS\System32\msvcrt.dll
  00007FFB5F1C0000-00007FFB5F218FFF    59000 C:\WINDOWS\System32\sechost.dll
  00007FFB617A0000-00007FFB618C4FFF   125000 C:\WINDOWS\System32\RPCRT4.dll
  00007FFB5F480000-00007FFB608B8FFF  1439000 C:\WINDOWS\System32\SHELL32.dll
  00007FFB5E1B0000-00007FFB5E1F9FFF    4A000 C:\WINDOWS\System32\cfgmgr32.dll
  00007FFB60E50000-00007FFB60EF8FFF    A9000 C:\WINDOWS\System32\shcore.dll
  00007FFB61320000-00007FFB61617FFF   2F8000 C:\WINDOWS\System32\combase.dll
  00007FFB5F050000-00007FFB5F0B8FFF    69000 C:\WINDOWS\System32\bcryptPrimitives.dll
  00007FFB5E960000-00007FFB5F04BFFF   6EC000 C:\WINDOWS\System32\windows.storage.dll
  00007FFB61680000-00007FFB616D0FFF    51000 C:\WINDOWS\System32\shlwapi.dll
  00007FFB5E110000-00007FFB5E120FFF    11000 C:\WINDOWS\System32\kernel.appcore.dll
  00007FFB5E0A0000-00007FFB5E0EBFFF    4C000 C:\WINDOWS\System32\powrprof.dll
  00007FFB5E0F0000-00007FFB5E104FFF    15000 C:\WINDOWS\System32\profapi.dll
  00007FFB5F2D0000-00007FFB5F3D4FFF   105000 C:\WINDOWS\System32\COMDLG32.dll
  00007FFB61070000-00007FFB611B4FFF   145000 C:\WINDOWS\System32\ole32.dll
  00007FFB616E0000-00007FFB6179EFFF    BF000 C:\WINDOWS\System32\OLEAUT32.dll
  00007FFB57760000-00007FFB579C7FFF   268000 C:\WINDOWS\WinSxS\amd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.15031.0_none_108c4f3adfe7a5f8\COMCTL32.dll
  00007FFB59950000-00007FFB59959FFF     A000 C:\WINDOWS\SYSTEM32\VERSION.dll
  00007FFB596F0000-00007FFB59712FFF    23000 C:\WINDOWS\SYSTEM32\WINMM.dll
  00007FFB513F0000-00007FFB51406FFF    17000 C:\WINDOWS\SYSTEM32\NETAPI32.dll
  00007FFB58ED0000-00007FFB58EFAFFF    2B000 C:\WINDOWS\SYSTEM32\WINMMBASE.dll
  00007FFB59D60000-00007FFB59D77FFF    18000 C:\WINDOWS\SYSTEM32\SAMCLI.DLL
  00007FFB5D810000-00007FFB5D81DFFF     E000 C:\WINDOWS\SYSTEM32\NETUTILS.DLL
  00007FFB5F190000-00007FFB5F1BCFFF    2D000 C:\WINDOWS\System32\IMM32.DLL
  00007FFB5C820000-00007FFB5C8B5FFF    96000 C:\WINDOWS\system32\uxtheme.dll
  00007FFB59720000-00007FFB5974BFFF    2C000 C:\WINDOWS\SYSTEM32\dwmapi.dll
  00007FFB5F3E0000-00007FFB5F47DFFF    9E000 C:\WINDOWS\System32\clbcatq.dll
  00007FFB37880000-00007FFB37A9FFFF   220000 C:\WINDOWS\System32\msxml3.dll
  00007FFB5DC30000-00007FFB5DC53FFF    24000 C:\WINDOWS\System32\bcrypt.dll
  00007FFB60F00000-00007FFB61065FFF   166000 C:\WINDOWS\System32\MSCTF.dll
  00007FFB483A0000-00007FFB48823FFF   484000 C:\WINDOWS\system32\explorerframe.dll
  00007FFB4E7F0000-00007FFB4E86BFFF    7C000 C:\WINDOWS\SYSTEM32\textinputframework.dll
  00007FFB50E60000-00007FFB51132FFF   2D3000 C:\WINDOWS\System32\CoreUIComponents.dll
  00007FFB548B0000-00007FFB54992FFF    E3000 C:\WINDOWS\System32\CoreMessaging.dll
  00007FFB5D3F0000-00007FFB5D420FFF    31000 C:\WINDOWS\SYSTEM32\ntmarta.dll
  00007FFB58350000-00007FFB58486FFF   137000 C:\WINDOWS\SYSTEM32\wintypes.dll
  00007FFB585F0000-00007FFB58604FFF    15000 C:\WINDOWS\SYSTEM32\usermgrcli.dll
  000000006AE40000-000000006AE61FFF    22000 D:\Program Files\SublimeText\Data\Packages\IMESupport\imesupport_hook_x64.dll
  00007FFB48830000-00007FFB48876FFF    47000 C:\WINDOWS\system32\dataexchange.dll
  00007FFB526A0000-00007FFB5297BFFF   2DC000 C:\WINDOWS\system32\d3d11.dll
  00007FFB532F0000-00007FFB53415FFF   126000 C:\WINDOWS\system32\dcomp.dll
  00007FFB5CF20000-00007FFB5CFC5FFF    A6000 C:\WINDOWS\system32\dxgi.dll
  00007FFB5C970000-00007FFB5CAD9FFF   16A000 C:\WINDOWS\system32\twinapi.appcore.dll
  00007FFB5B970000-00007FFB5BB06FFF   197000 C:\WINDOWS\SYSTEM32\PROPSYS.dll
  00007FFB5B3C0000-00007FFB5B3CEFFF     F000 C:\WINDOWS\System32\perfproc.dll
  00007FFB4CB70000-00007FFB4CC61FFF    F2000 C:\WINDOWS\system32\msctfuimanager.dll
  00007FFB3F780000-00007FFB3F92AFFF   1AB000 C:\WINDOWS\system32\DUI70.dll
  00007FFB3DE90000-00007FFB3DF22FFF    93000 C:\WINDOWS\system32\DUser.dll
  00007FFB46F80000-00007FFB47140FFF   1C1000 C:\WINDOWS\SYSTEM32\UIAutomationCore.dll
  00007FFB5DEF0000-00007FFB5DF88FFF    99000 C:\WINDOWS\SYSTEM32\sxs.dll
  00007FFB50E10000-00007FFB50E56FFF    47000 C:\WINDOWS\System32\UIAnimation.dll
  00007FFB58DD0000-00007FFB58E39FFF    6A000 C:\Windows\System32\oleacc.dll
  00007FFB54260000-00007FFB5451DFFF   2BE000 C:\WINDOWS\system32\dwrite.dll
  00007FFB57DA0000-00007FFB57E2AFFF    8B000 C:\WINDOWS\system32\directmanipulation.dll
  00007FFB5A4A0000-00007FFB5A4D5FFF    36000 C:\WINDOWS\system32\xmllite.dll
  00007FFB41F40000-00007FFB41F49FFF     A000 C:\WINDOWS\SYSTEM32\atlthunk.dll
  00007FFB3EEC0000-00007FFB3F196FFF   2D7000 C:\WINDOWS\SYSTEM32\D3D10Warp.dll

qq 20170227095420

terrywh picture terrywh on 23 Feb 2017
  1. Do you see the problem in the latest 170227 build?
  2. Try to run ConEmu64.exe -basic -run {cmd}. Any changes?
Maximus5 picture Maximus5 on 3 Mar 2017

@miniksa Seems like it's a compatibility issue. Haven't checked it thoroughly yet on 15031, but on stable Win10 build 14393 cursor coordinates are reported from the console properly.

Maximus5 picture Maximus5 on 3 Mar 2017

Filed internally as MSFT:11150442. We'll triage and investigate.

miniksa picture miniksa on 3 Mar 2017

Just checked on Win10 build 15031 (GetSystemMetrics(SM_DBCSENABLED) == 0). The problem exists. The ReadConsoleOutputW does not return COMMON_LVB_***_BYTE flags, but GetConsoleScreenBufferInfo reports wrong X position in dwCursorPosition as if cells are doubled with COMMON_LVB_***_BYTE.

ConEmu can't do anything with that. @terrywh you shall wait for next insider build with a fix.

2017-03-04_17-19-00

lsbi    {dwSize={X=0x006e Y=0x03e8 } dwCursorPosition={X=0x0004 Y=0x0007 } wAttributes=0x0007 ...}  _CONSOLE_SCREEN_BUFFER_INFO
pLine,10    0x0000023622d30cc0 {{Char={UnicodeChar=0x003e '>' AsciiChar=0x3e '>' } Attributes=0x0008 }, {Char={UnicodeChar=...} ...}, ...}  _CHAR_INFO[0x0000000a]
    [0x00000000]    {Char={UnicodeChar=0x003e '>' AsciiChar=0x3e '>' } Attributes=0x0008 }  _CHAR_INFO
    [0x00000001]    {Char={UnicodeChar=0x0020 ' ' AsciiChar=0x20 ' ' } Attributes=0x0008 }  _CHAR_INFO
    [0x00000002]    {Char={UnicodeChar=0x63a5 '接' AsciiChar=0xa5 'Ґ' } Attributes=0x0007 }  _CHAR_INFO
    [0x00000003]    {Char={UnicodeChar=0x53e3 '口' AsciiChar=0xe3 'г' } Attributes=0x0007 }  _CHAR_INFO
    [0x00000004]    {Char={UnicodeChar=0x8fde '连' AsciiChar=0xde 'Ю' } Attributes=0x0007 }  _CHAR_INFO
    [0x00000005]    {Char={UnicodeChar=0x63a5 '接' AsciiChar=0xa5 'Ґ' } Attributes=0x0007 }  _CHAR_INFO
    [0x00000006]    {Char={UnicodeChar=0x0020 ' ' AsciiChar=0x20 ' ' } Attributes=0x0007 }  _CHAR_INFO
Maximus5 picture Maximus5 on 4 Mar 2017

Thanks for the answer. i'll see if future insider build fix this problem and close this issue.

terrywh picture terrywh on 6 Mar 2017

Win10 build 15055 still have this bugs.

takekazuomi picture takekazuomi on 11 Mar 2017
👍1

This bug exists in 15063. Seems may related with
https://technet.microsoft.com/en-us/library/mt427362.aspx

Chinese characters are not showing correctly in Bash when using Windows Console, but CMD works good.
I am using en_US.UTF8 in Bash On Windows.

Seems like Unicode or ANSI related issue.

fanthos picture fanthos on 27 Mar 2017

windows build 16170 seems have this bug fixed
default

Austaras picture Austaras on 8 Apr 2017
👍1

@miniksa It is fixed in build 16170. Please also fix it in Creates Update build 15063.

takekazuomi picture takekazuomi on 8 Apr 2017

I think its better to keep this issue open as this issue not fixed in Windows 10 RTM 1703.
Not sure if Microsoft will fix it in 1703 or not.

fanthos picture fanthos on 12 Apr 2017
👍2

I have same issue with Windows 10 build 15063.
This workaround fix solves my issue.

Use ReadConsoleOutputCharacterA and IsDBCSLeadByte instead of ReadConsoleOutputAttribute.

~~~patch
diff --git a/src/ConEmuCD/ConsoleMain.cpp b/src/ConEmuCD/ConsoleMain.cpp
index 1b277324a..89ebdafe2 100644
--- a/src/ConEmuCD/ConsoleMain.cpp
+++ b/src/ConEmuCD/ConsoleMain.cpp
@@ -5609,38 +5609,38 @@ void static CorrectDBCSCursorPosition(HANDLE ahConOut, CONSOLE_SCREEN_BUFFER_INF
// -- GetConsoleScreenBufferInfo возвращает координаты курсора в DBCS, а нам нужен wchar_t !!!
if (IsConsoleDoubleCellCP())
{

  • WORD Attrs[200];
  • LONG cchMax = countof(Attrs);
  • WORD* pAttrs = (csbi.dwCursorPosition.X <= cchMax) ? Attrs : (WORD)calloc(csbi.dwCursorPosition.X, sizeof(pAttrs));
  • if (pAttrs)
  • CHAR Chars[200];
  • LONG cchMax = countof(Chars);
  • LPSTR pChars = (csbi.dwCursorPosition.X <= cchMax) ? Chars : (LPSTR)calloc(csbi.dwCursorPosition.X, sizeof(*pChars));
  • if (pChars)
    cchMax = csbi.dwCursorPosition.X;
    else
  • pAttrs = Attrs; // memory allocation fail? try part of line?
  • pChars = Chars; // memory allocation fail? try part of line?
    COORD crRead = {0, csbi.dwCursorPosition.Y};
    //120830 - DBCS uses 2 cells per hieroglyph
    DWORD nRead = 0;
    // TODO: Optimize
  • if (ReadConsoleOutputAttribute(ahConOut, pAttrs, cchMax, crRead, &nRead) && nRead)
  • if (ReadConsoleOutputCharacterA(ahConOut, pChars, cchMax, crRead, &nRead) && nRead)
    {
    _ASSERTE(nRead==cchMax);
    int nXShift = 0;
  • LPWORD p = pAttrs, pEnd = pAttrs+nRead;
  • LPSTR p = pChars, pEnd = pChars+nRead;
    while (p < pEnd)
    {
  • if ((*p) & COMMON_LVB_LEADING_BYTE)
  • if (IsDBCSLeadByte(*p))
    {
    nXShift++;
    p++;
  • _ASSERTE((p < pEnd) && ((*p) & COMMON_LVB_TRAILING_BYTE));
  • _ASSERTE(p < pEnd);
    }
    p++;
    }
    _ASSERTE(nXShift <= csbi.dwCursorPosition.X);
    csbi.dwCursorPosition.X = max(0,(csbi.dwCursorPosition.X - nXShift));
    }
  • if (pAttrs && (pAttrs != Attrs))
  • if (pChars && (pChars != Chars))
    {
  • free(pAttrs);
  • free(pChars);
    }
    }
    }
shirosaki picture shirosaki on 12 Apr 2017
👍2

Ｔhe issue described here is still present in 170402 version, but other CJK font alignment problems appeared to have been fixed, in case anyone is wondering.

cursorposbug

(170402 version is on the left, 161206 on the right)

yi78 picture yi78 on 11 May 2017

The workaround mentioned in #1111 works for me.

cy18 picture cy18 on 12 May 2017
👍1

I tried the patch from @shirosaki , it fixes cmd.exe with Morden Console, but does not fix WSL bash.

fanthos picture fanthos on 13 May 2017

for me 170517 got it fixed

terrywh picture terrywh on 26 May 2017
👍2

fixed in KB4020102
https://support.microsoft.com/en-us/help/4020102/windows-10-update-kb4020102

Addressed issue where non-English characters showed incorrectly in apps that attempt to wrap/replace the in-box console.

takekazuomi picture takekazuomi on 4 Jun 2017
Was this page helpful?
0 / 5 - 0 ratings

Related issues

joanis picture joanis  ·  3Comments
DrMerlin picture DrMerlin  ·  4Comments
Daniel15 picture Daniel15  ·  3Comments
altbdoor picture altbdoor  ·  4Comments
phazei picture phazei  ·  6Comments