Conemu: Delay when running WSL bash shell

[BladeMF]

Versions

ConEmu build: 171117 x64
OS version: Windows 1709 x64
Used shell version (Far Manager, git-bash, cmd, powershell, cygwin, whatever): bash

Problem description

When starting the Bash task, there is a 4-5 seconds delay before the command prompt being shown. This started after the Windows update to 1709.

Task definition is:
set "PATH=%ConEmuBaseDirShort%\wsl;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --wsl -cur_console:pm:/mnt

Steps to reproduce

1. Run Bash task

Actual results

Delay in showing the command prompt.

Expected results

No delay.

[Maximus5]

Delay before each prompt?
If so, check your PS1.

[BladeMF]

No, the delay is before the first prompt only.

[BladeMF]

Cmd and PowerShell do not have the same delays

[Maximus5]

When exactly the diary occurs? Video?
What if you run wsl.exe instead of task with connector?

[BladeMF]

On the status bar, the delay is before wslbridge.exe[65]:18816 is displayed. The text says conemu-cyg-64.exe. This is when you open a new tab. The delay is before you see the first prompt.

Running wsl produces an instant prompt.

[BladeMF]

I am not sure how to record a video.

[Maximus5]

Try to add --verbose switch before the --wsl

[Habitats]

Can confirm that this is happening. Although it has been happening for me for as long as I can remember.

[BladeMF]

Here is the log:

{PID:17100} declaring TERM: `xterm-256color` (was: `cygwin`)
{PID:17100} current $HOME is `/home/VladislavKosev`
{PID:17100} 0: isatty()=1; pgrp=17100; ttyname()=`/dev/cons3`
{PID:17100} 1: isatty()=1; pgrp=17100; ttyname()=`/dev/cons3`
{PID:17100} 2: isatty()=1; pgrp=17100; ttyname()=`/dev/cons3`
{PID:17100} calling fork (pgid=17100)
{PID:17100} child pid=20128 pgid=20128 was created (ourpgid=17100)
{PID:17100} PTY was created: `/dev/pty3`; Child PID:20128
{PID:17100} 0: isatty()=1; pgrp=17100; ttyname()=`/dev/cons3`
{PID:17100} 1: isatty()=1; pgrp=17100; ttyname()=`/dev/cons3`
{PID:17100} 2: isatty()=1; pgrp=17100; ttyname()=`/dev/cons3`
{PID:17100} raising SIGUSR1 in pid=20128
{PID:17100} SIGUSR1 received
{PID:20128} child created (pgid=20128){PID:17100} ioctl(3,TIOCSWINSZ,(240,63)) succeeded (0)

{PID:20128} child process wating for SIGUSR1 (pgid=20128)
{PID:17100} ioctl(3,TIOCSWINSZ,(240,63)) succeeded (0)
{PID:20128} SIGUSR1 received
{PID:17100} ioctl(3,TIOCSWINSZ,(240,63)) succeeded (0){PID:20128} child process continues after 16 ms (SIGUSR1=1)

{PID:20128} ttyname(1)=`/dev/pty3`
{PID:17100} ioctl(3,TIOCSWINSZ,(240,63)) succeeded (0){PID:20128} ttyname(1)=`/dev/pty3`

{PID:20128} 0: isatty()=1; pgrp=20128; ttyname()=`/dev/pty3`
{PID:20128} 1: isatty()=1; pgrp=20128; ttyname()=`/dev/pty3`
{PID:20128} 2: isatty()=1; pgrp=20128; ttyname()=`/dev/pty3`
{PID:20128} raising SIGUSR1 in pid=17100
{PID:20128} argv[0]: `/conemu-cyg-64`
{PID:20128} shell: `/wsl/wslbridge.exe` `-eConEmuBuild` `-eConEmuPID` `-t` `bash` `-l` `-i`
{PID:20128}   dir: `/cygdrive/c/Users/VladislavKosev`

The delay is occurring AFTER the last line is visible on the screen.

[BladeMF]

@Habitats I cannot be certain that the Windows update caused this. It may have been an update to ConEmu or some other change. But it started at the same time.

[Maximus5]

Try to monitor process creation. The lag occurs on certain step, but outside of ConEmu code.
Try ProcessMonitor or ProcessExplorer.
Also, perhaps logs from ConEmu64.exe -basic -log -run {bash} may show something.

[BladeMF]

I tried ProcessMonitor, but the hole in the timings appears random :( There is way too much data for me to see anything without something to look for. I have the feeling that wslbridge is struggling with something, but I cant see anything that can help.

I ran the command you gave and I didn't see anything out of the ordinary. Does it write a log somewhere? I tried looking here C:\Program Files\ConEmu, but didn't find anything. The only interesting thing is that the bash produced by this command runs considerably slower. It is noticeable especially when running mc and exiting - there is a second or so where the screen is blank.

[Maximus5]

Try to filter ProcessMonitor logs by "operation contains process". That is the start point.

LogFiles on the desktop.

Did you tried official wslbridge without ConEmu? AFAIK there is a bundle with mintty.

[BladeMF]

Process monitor: I used contains conemu and contains wsl. Didn't help me. If you point me to some specific events to look for, that would be great.

Official wslbridge: back when I enabled it, I downloaded it manually and followed the instructions on your website. I could try downloading it again and modify the task's parameters to use that. The delay may have started appearing when you bundled the files. I install every alpha release that came as an update.

Logs going to do that now and post back.

[BladeMF]

ConEmu-gui-12108.log
ConEmu-srv-17728.log
ConEmu-con-17728.log

There are the logs. I used -log4. I closed it after I saw the prompt.

[BladeMF]

I downloaded wslbridge from here https://github.com/rprichard/wslbridge/releases and the installed wsltty, and then compared the two files, they seem identical. Then copied the bundled cygwin1.dll to wsl2 folder (along with wslbridge.dll and wslbridge-backend) and changed the task as follows:
set "PATH=%ConEmuBaseDirShort%\wsl2;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --wsl -cur_console:pm:/mnt
It has the same delay. Can I take it that you don't have such delay?

[BladeMF]

I managed to record a video of the delay:

It's a crappy gif, but you can see the delay between opening the new tab and the appearing of the blue and green prompt.

[Maximus5]

Yes, I don't have any delay.

What about wsltty, did you check it as I asked?

BTW, do you have git or cygwin installed on your computer?

[BladeMF]

I installed wsltty and there is also a delay there, but not as much (maybe 1-2 seconds vs 3-4 with ConEmu).

I have Visual Studio 2017 and git installed. Not sure about C++ tools, but I could install them if needed. I haven't installed Cygwin, I am not sure what to do with it.

What do you want me to do?

[Maximus5]

What about wsl.exe started without ConEmu? Delay?

It looks like the delay occurs after connector runs exec on wslbridge.exe (from your log). ProcessMonitor shows when the exe is started, and you may see from its log what process contains the lag.
If it's in wslbridge, you shall report this to wslbridge author.

VisualStudio is not useful to debug cygwin applications.

I've asked about cygwin because it's interesting to compare startup time of wsl and bash from cygwin.

[obhasqr]

@BladeMF Did you install the Windows Store version of Ubuntu?

[BladeMF]

No, I used windows Add/Remove Features (before it was in the store) and upgraded it.

[BladeMF]

@Maximus5 wsl.exe is instant.
I am not sure how to setup Cygwin to test that.

[Maximus5]

@rprichard Do you have any idea what to test further?

[BladeMF]

@Maximus5, I just noticed something. When a new console is opened, it ends up waiting with a text like "wslbridge.exe[64]:some number". The moment the prompt appears, the "some number" changes to some other number. Is that a port or a process id?

[Maximus5]

Process id

[BladeMF]

So why does it change?

[Maximus5]

Because the process is changed?

[BladeMF]

I said it because I thought it might have something to do with the delay - two wslbridge processes.

[iplux]

I started having a similar issue recently.
I have always a long delay, in extreme circumstances I get the following error:

wslbridge warning: process didn't exit after 10 seconds: "C:\PROGRA~1\ConEmu\ConEmu\wslwslbridge.exe" --press-return 496

However just after the error message, wsl opens correctly.
I am using the standard task from Conemu

[Maximus5]

The message from wslbridge itself (wslbridge.cc:845)

[Maximus5]

Antivirus/defender?

[BladeMF]

I have defender. I will try without it and post back.

[BladeMF]

nope, no change.

[Maximus5]

Perhaps it's related to https://github.com/Microsoft/WSL/issues/981
Anyway, it's out of ConEmu responsibilities

[BladeMF]

If it was related, wouldn’t we all have delays?

[Maximus5]

I don't know the reason of the problem.

[Maximus5]

BTW, adding conemu-cyg-64.exe and wslbridge.exe to Windows Defender exceptions may significantly decrease delays.
I don't think ConEmu may do anything with them

[BladeMF]

@Maximus5 Adding the whole ConEmu folder as an exception did reduce the delay, but it didn't remove all of it. :(

[elijahgagne]

I've noticed that bash is slower to start and slower to run things when using the wslbridge. If I run bash without wslbridge in ConEmu, I don't see that slowness (but there are other issues of course). I put an exception in Windows Defender and that cut down my delay by about half, but it's still enough that it's noticable. Should this get opened as an issue at https://github.com/rprichard/wslbridge?

[Maximus5]

@elijahgagne Have you tested wslbridge without ConEmu?

[Maximus5]

Anyway, I hope in the nearest future, when new PTY interface become public, ConEmu will run WSL directly, without wslbridge.

[elijahgagne]

I just tried wslbridge without ConEmu. It opens fast and I see no delays running things. So that is a good data point.

[Maximus5]

Have you tried wsltty or started wslbridge from conhost?

[elijahgagne]

I just tried both of them, neither show delays.

[InfiniteLee]

I recently nuked and reinstalled WSL and I am experiencing this issue as well, with the addition that certain things like opening/closing vim or changing directories have the same delay. I have also noticed that running wslbridge by itself (or wsl.exe for that matter) does not have the delay.

[ghost]

I see bash doesn't have delay when typing command or do other stuff but run vim inside ubuntu wsl/bash.exe/wsl delayed somhow, probably because of colors slow it down and vim's plugins.

[meatuses]

Same here. Worked fine with no delays on my old laptop with Ubuntu 14.04 WSL and not sure what version (i guess June release) of ConEmu. Now, on 16.04 or 18.04 and latest ConEmu even something like nano takes 2 redraws and a second or two to exit, initiating ssh connections is slower and so on.
Plain ubuntu.exe without wslbridge inside ConEmu works just fine. Task config:
set "PATH=C:\Program Files\WindowsApps\CanonicalGroupLimited.UbuntuonWindows_1804.2018.817.0_x64__79rhkp1fndgsc;%PATH%" & C:\Program Files\WindowsApps\CanonicalGroupLimited.UbuntuonWindows_1804.2018.817.0_x64__79rhkp1fndgsc\ubuntu.exe

Maybe there's a way to install Ubuntu 14.04 on WSL?

[ghost]

@meatuses : What's differece between System32\wsl.exe and System32/bash.exe? I don't see any differences.

[Maximus5]

Wsl has different supported switches

[ghost]

@Maximus5 : Can you elaborate? I'm using bash.exe and looking for better one to use as main terminal.

[Maximus5]

Better for what?
The only proper way to run WSL is described in BashOnWindows in ConEmu

[ghost]

I see, I have only one Ubuntu installed so wsl will default to it. No difference when run wsl.exe and bash.exe, also see we can change the default via wsl.conf. okie then.

p/s: I use bash.exe

[wkuranowski]

@Maximus5 It seems that the latest update of Windows 10 includes a public ConPTY API. Is it enough to replace wslbridge? Are you going to use this API in the near future?

[uhriap]

It seems delays on vim/htop/less/etc. are proportional to buffer height in conemu settings. Reducing buffer height lowers delays.

[B0073D]

I can confirm that lowering the buffer height does indeed lower the delay but it's still quite noticeable even with the buffer as low as one hundred.

Edit: This is still present in the latest build 190108

[Ma-Ba]

I want to say that I have the same issue. mc start times:
WSL bash.exe (no ConEmu): 0.3s
WSL wslbridge.exe (no ConEmu): 0.3s
WSL in ConEmu: 0.3s
Cygwin in ConEmu: 1s
ConEmu (conemu-cyg-64.exe --wsl): 7s (this is the problem)
ConEmu (wslbridge.exe): 0.3s (could this be the solution? need to test this)

[Maximus5]

May be this issue is related to gh-1841

[kbirger]

I'm also having this problem. Some additional notes:

1. If I reset the settings on my ConEmu, the issue persists
2. If I create a CMD tab and start "wsl.exe" from it, there is no slow-down (but there is no 256-color support, etc because connector/wslbridge is not running)
3. If I install the latest Cmder (based on same build of ConEmu), I do not experience the problem.
4. If I then import my ConEmu.xml settings to Cmder, I do not experience the problem.

This leads me to believe that the problem might be in some sort of application state/history, which is not stored in ConEmu.xml. @Maximus5 - does anything come to mind?

For me, the delay is most noticeable when starting/exiting less or vim. Can you explain gh-1841? I'm not familiar enough with the architecture of ConEmu, or these apis to know what alternate and primary scrollback buffers are, but given that vim and less both enter what seem to me like different scrolling/history modes (sorry, I'm a laymen when it comes to terminal programming :)), it seems like you're right - it could be related...

[Maximus5]

@kbirger Are you definite about same command lines for wsl task in Cmder and ConEmu instances? Just to be sure that they both run wsl via connector? Have they both same height of the buffer?

I hope so, than there are some Windows options that ConEmu doesn't control.
First thing that comes to mind - RealConsole settings.
Please open conhost settings in both instances (either from ConEmu SystemMenu/Debug or from shown conhost window) and compare them.

[kbirger]

@Maximus5 wow. interesting. I loaded Cmder back up, and now I am experiencing the same issue, regardless of whether I reset the settings on it.

I did as you said, and...

• both bring up the same properties dialog: "ConEmu" Properties (in fact, I can't open both at once)
• Buffer width is 240 and height is 9999

It seems that this buffer height has some bearing on the delay. If I set it to a smaller value, the delay disappears!

I suspect that this value is tied to Settings > General > Size & Pos > [Console buffer height]
[X] Long console output [32766]

[Maximus5]

If you are experiencing the same delay in both instances - no sense in comparing properties.
You may try to copy whole ConEmu folder to new location and run ConEmu64.exe -basic -run {bash}

[kbirger]

I get what you're saying. I was initially not experiencing the same issue, so I decided to eliminate any potential variables.

I tried what you suggested above, and I get the same issue in the new folder.

What's the downside to decreasing the buffer size? Inability to scroll as far back?

The issue seems to be a concert of wslbridge/connector + switching buffers + buffer size

edit: value of 1000 seems to be OK.

[Maximus5]

Please try 190310.
If the delay is still noticable, please make logs from both ConEmu and connector.

• ConEmu: run it with switch -log
• connector: modify {bash} task adding --log before --wsl.

[elijahgagne]

I'm seeing no noticeable delay with minimal testing.

[Ma-Ba]

I tested 190310 and the issue seems gone. Thank you!

[kbirger]

Seems better on the new build, but not in all cases. My original test of entering/exiting vim and less seems good, but I'm noticing a lot of lag on shell completion (using zsh personally, with the multi-choice completion) compared to setting the buffer size / Long console output setting to a lower value.

[Maximus5]

@kbirger Shell completion is not related to alternative buffer, it operates in primary buffer only. Another issue.

[grymmjack]

For me oh-my-zsh was the biggest culprit.

For some reason the Console buffer height under General > Size & Pos > Console buffer height was set for 32767 which is egregious, so I lowered that to 1000.

Lowering the console buffer height size made it a bit faster, but what really sped it up for me was using bash instead of oh-my-zsh. The theme I was using in oh-my-zsh was 'muse' which is a pretty minimal one, but I know most of these themes mess with the $PS1 variable to do fancy prompt things.

Kinda bummed I can't run zsh but not bummed enough to endure the lag it induces. Hope this helps someone.