Oh-my-posh3: Performance: Command execution slow on Windows

[amoldeshpande]

Prerequisites

• [ ] I have read and understand the CONTRIBUTING guide
• [x ] I looked for duplicate issues before submitting this one

Description

Slowness in rendering prompt. It's noticeably slow in Windows compared to WSL2, even in the same Terminal.

Environment

• Oh my Posh version: 3.64.2
• Theme:
• Operating System: Windows 10 2004
• Shell: powershell
• Terminal: default console

Steps to Reproduce

I was trying to figure out why composing the prompt is so slow on windows. I unfortunately don't know much go, but I muddled around with it and found a couple of things.

Firstly, running and parsing git commands seems quite slow.

PS C:\Users\amol> Measure-Command {posh-windows-amd64.exe --shell zsh --config c:\users\amol\.poshthemes\amol.omp.json}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 89
Ticks             : 895987
TotalDays         : 1.03702199074074E-06
TotalHours        : 2.48885277777778E-05
TotalMinutes      : 0.00149331166666667
TotalSeconds      : 0.0895987
TotalMilliseconds : 89.5987



PS C:\Users\amol> Measure-Command {c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe rev-parse --is-inside-work-tree }
fatal: Not a git repository (or any of the parent directories): .git


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 37
Ticks             : 376365
TotalDays         : 4.35607638888889E-07
TotalHours        : 1.04545833333333E-05
TotalMinutes      : 0.000627275
TotalSeconds      : 0.0376365
TotalMilliseconds : 37.6365

As you can see here, running the raw git command takes less than half the time it takes to run a posh command where my only config is a git segment (the default one from your theme file)

There is no git repo in that directory (I was trying to get the fastest execution for the git segment, but interestingly the overall execution time of posh does not change even within a git repo)

I tried to do some profiling in go, and came up with this for execution:

I'm always running

posh-windows-amd64.exe --shell zsh --config c:\users\amol\.poshthemes\amol.omp.json

where the theme only contains a only git prompt segment exactly like the jandedobbeleer profile

H:\github\oh-my-posh3\src>go tool pprof c:\Users\amol\foo
Type: cpu
Time: Jan 1, 2021 at 2:20pm (PST)
Duration: 201.11ms, Total samples = 60ms (29.83%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top10
Showing nodes accounting for 60ms, 100% of 60ms total
Showing top 10 nodes out of 30
      flat  flat%   sum%        cum   cum%
      50ms 83.33% 83.33%       50ms 83.33%  runtime.cgocall
      10ms 16.67%   100%       10ms 16.67%  os/exec.(*Cmd).Start
         0     0%   100%       30ms 50.00%  bufio.(*Reader).ReadLine
         0     0%   100%       30ms 50.00%  bufio.(*Reader).ReadSlice
         0     0%   100%       30ms 50.00%  bufio.(*Reader).fill
         0     0%   100%       30ms 50.00%  internal/poll.(*FD).Read
         0     0%   100%       60ms   100%  main.(*Segment).enabled (inline)
         0     0%   100%       60ms   100%  main.(*Segment).setStringValue
         0     0%   100%       60ms   100%  main.(*engine).setStringValues.func1
         0     0%   100%       10ms 16.67%  main.(*environment).hasCommand
(pprof) quit

In a repo with git, this is the profile

H:\github\oh-my-posh3\src>go tool pprof foo
Type: cpu
Time: Jan 1, 2021 at 2:08pm (PST)
Duration: 412.02ms, Total samples = 220ms (53.40%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top10
Showing nodes accounting for 220ms, 100% of 220ms total
Showing top 10 nodes out of 48
      flat  flat%   sum%        cum   cum%
     210ms 95.45% 95.45%      210ms 95.45%  runtime.cgocall
      10ms  4.55%   100%       10ms  4.55%  runtime.slicerunetostring
         0     0%   100%      150ms 68.18%  bufio.(*Reader).ReadLine
         0     0%   100%      150ms 68.18%  bufio.(*Reader).ReadSlice
         0     0%   100%      150ms 68.18%  bufio.(*Reader).fill
         0     0%   100%       30ms 13.64%  fmt.Fprintf
         0     0%   100%       30ms 13.64%  fmt.Printf
         0     0%   100%      150ms 68.18%  internal/poll.(*FD).Read
         0     0%   100%       30ms 13.64%  internal/poll.(*FD).Write
         0     0%   100%       30ms 13.64%  internal/poll.(*FD).writeConsole
(pprof)

So, it seems that running multiple git commands slows things down even more , as you'd expect.

With my less-than-zero knowledge of go, I can only theorize that

1. Perhaps parallelizing all the git commands will help responsiveness, instead of executing them sequentially. I saw a closed issue where you did this for segments, but I wasn't able to figure out if you tried just the git commands.
2. Maybe there is an optimization to be made where instead of reading line by line, you read the entire buffer and then parse it line by line. This should help in situations where there is a lot of git output, I would think.

Love the tool, thanks for the great work !

[amoldeshpande]

There also seem to be a couple of go libraries for git. Those might be better options than spawning processes. e.g., https://github.com/go-git/go-git/

[TylerLeonhardt]

+1 to this. Here's a video of before and after (uploading as a zip because GitHub said I had to):

vid.zip

I don't expect it to be as fast as the before... but certainly not as slow as it is unfortunately.

[amoldeshpande]

I played around with go-git a little. For a simple case of trying to open a repo where it does not exist it takes 24ms instead of 80+.

I couldn't figure out how to get intellisense working in Sublime Text so that I could understand the module better, so I was not able to replace the other functionality (detecting status, branch, etc.)

In any case, I think it looks like a promising avenue to investigate.

[JanDeDobbeleer]

I tried go-git before, there was a huge delay when running the status command at the time. Maybe that has been resolved, I even still have the branch with those changes.

Did you try the latest version? We did some improvements in term of performance, curious to see if that resolves anything.

@amoldeshpande I also have a working implementation of a rust library for doing the command calls, which everyone keeps saying is faster (the issue is the implementation of go on Windows, as on MacOS or unix there's no issue at all). I could provide a test build to you to see of that changes things for the better? It does for me, but I need some input from others.

[lnu]

We already went from +-350ms to 200ms for the git segment rendering(on windows). In wsl2 it takes 10ms at most.
Clearly, if we could get the same performance on windows as on linux, that would be fantastic.

[JanDeDobbeleer]

@amoldeshpande I did a quick restore of the changes I did for go-git, the slow status is still present so that's a no go (it's a few seconds, not even ms). On top of that, it can't handle folders inside a repo, it always reports as not in a repo which isn't what I expect.

That said, here's a version of oh-my-posh that has an experimental lib to do cmd executions, I'm looking forward to feedback on this one.

posh-windows-experimental-amd64.zip

[royvou]

This is an interesting issue, performance for executing all "Language segments" is also not optimal (Takes ~100MS per segment when it is "in context".
The latest release has improved it a lot (Mainly for git, but also for the .NET language segment, less for the other segments)

@JanDeDobbeleer i tried your experimental build, but i could not get it to work.

When i download the .zip and extract it, powershell/cmd complains it is missing posh3.dll

I tried to build the branch "https://github.com/JanDeDobbeleer/oh-my-posh3/tree/validate-lib" myself, and after installing GCC i get the warning the .h file could not be found (Which could be explained by the gitignore which excludes .h files)

btw: Loving oh-my-posh, gets better every day :)

[TravisTX]

Same error as royvou here

[JanDeDobbeleer]

i tried your experimental build, but i could not get it to work

@royvou aha. That's interesting as I assumed the binary would have packed it. I'll upload a new zip with that included 👍🏻

[royvou]

i tried your experimental build, but i could not get it to work

@royvou aha. That's interas I assumed the binary would have packed it. I'll upload a new zip with that included 👍🏻

As the binary increased from ~10MB to ~17MB i would have assumed as well! But looks like it hasn't :(

[lnu]

Don’t forget that the segments processing is done in //. So the prompt is always as slow as the slowest segment(more or less).

[JanDeDobbeleer]

@royvou @TravisTX I updated the original comment with a new zip file. Now tbh, if I can't pack these dependencies into the exe, I'm not inclined to go down this path.

[TravisTX]

1 is from main
2 is your experimental build

[JanDeDobbeleer]

@TravisTX that's not enough to say we're about to have a party. Confirms what I'm seeing. Better, but not by major margins.

[royvou]

I think you only need the .DLL in the zip if you were to distribute it.

A check with the --debug parameter does not show significant improvements, am i doing something wrong (As multiple people are mentioning it should be a lot faster?)?

(Screenshoht is from the PowerToys Repo, Oh-my-posh3 repo takes about ~120-140MS)

Experimental
VS
Latest

Also tried replacing the current exe to looks if it "feels" faster, but i would not say so.
@lnu Are you sure, the timings above seem to assume it is not processed in parallel.

Ah looks like TravisTX beat me to it ;)

Seems to be ~5% faster.

[amoldeshpande]

ok, so I won't try the experimental build since others have already reported back.

It's disappointing to hear about go-git as well :(

Doesn't go have C interop ? maybe a wrapper around just the git commands we need ? using the C libgit.

[JanDeDobbeleer]

I have a proposal. I just forked go and removed the timeout that's present on Windows. It's stated that removing it does work, meaning we can give it a "go" and see what the effect is of oh-my-posh built with that fork. Managing that should be doable if it provides to be a major improvement without side effects.

[JanDeDobbeleer]

Very, very fast in between everything. I could have made some mistakes here 😓
posh-windows-amd64-experimental.exe.zip

[amoldeshpande]

It's a LOT better ! Overall still a bit slower than Linux, but maybe parallelizing git commands will help with that ?

[JanDeDobbeleer]

maybe parallelizing git commands will help with that

Depends, most information is fetched by context, so not all are needed. But worthwhile to investigate! Can you tell me what the timing are now? I get 100ms on my machine which previously has 180ms. Looking at @TravisTX's machine that one's a bit faster than most...

[royvou]

Sounds like a great idea Jan 👍

I'm wondering if the change to a remote exe also has (some) effect on performance? Since now the exe is created/destroyed after each command is created. I assume this would prevent some forms of caching as well?

[amoldeshpande]

I got about 40ms with my tests above. So that I don't sound ungrateful, this version by itself makes it eminently usable for me now. Anything else on top of that is just gravy.

[lnu]

Now we have to wonder if the margin is enough to maintain an own fork of go?

[JanDeDobbeleer]

I like to go for gravy. But 40ms is blazing fast compared to where we came from. If more people can confirm this, I'll do the necessary changes to ensure we can build with that fork on Windows.

[lnu]

Also tried replacing the current exe to looks if it "feels" faster, but i would not say so.
@lnu Are you sure, the timings above seem to assume it is not processed in parallel.

With --debug each segment is processed one by one. But you can assume the most expensive segment will be the bottleneck when processed in //.

[lnu]

Am i missing something:

[amoldeshpande]

S H:\github\oh-my-posh3> C:\bin\posh-windows-amd64-experimental.exe --shell zsh --config C:\users\amol\.poshthemes\jandedobbeleer.omp.json --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - %{%}%{%}%{%}█%{%}%{%}amol%{%}%{%}@%{%}%{%}amolhomevista%{%}%{%}█%{%}%{%}%{%}
spotify(false) -  12 ms -
path(true)     -   0 ms - %{%}%{%}%{%}  oh-my-posh3%{%}%{%}█%{%}%{%}%{%}
git(false)     -  15 ms -
battery(true)  -   2 ms - %{%}%{%}%{%}█%{%}%{%}100 %{%}%{%}%{%}
node(false)    -   0 ms -
shell(true)    -   0 ms - %{%}%{%}%{%} ﲵ zsh%{%}%{%}█%{%}%{%}%{%}
root(false)    -   0 ms -
exit(true)     -   0 ms - %{%}%{%}%{%} %{%}%{%}█%{%}%{%}%{%}
PS H:\github\oh-my-posh3>

I haven't set my powershell up to render unicode correctly, but the segment timings seem in line with what I'm seeing.

Edit: as an aside, it's interesting that it thinks my desktop has a battery :) No need to derail the issue though.

Edit2: With powershell so I'm running the same command

PS H:\github\oh-my-posh3> C:\bin\posh-windows-amd64-experimental.exe --config C:\users\amol\.poshthemes\jandedobbeleer.omp.json --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █amol@amolhomevista█
spotify(false) -  10 ms -
path(true)     -   0 ms -   oh-my-posh3█
git(false)     -  16 ms -
battery(true)  -   3 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ powershell█
root(false)    -   0 ms -
exit(true)     -   0 ms -  █
PS H:\github\oh-my-posh3>

[royvou]

So far i can't say its any faster.

[lnu]

S H:\github\oh-my-posh3> C:\bin\posh-windows-amd64-experimental.exe --shell zsh --config C:\users\amol\.poshthemes\jandedobbeleer.omp.json --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - %{%}%{%}%{%}█%{%}%{%}amol%{%}%{%}@%{%}%{%}amolhomevista%{%}%{%}█%{%}%{%}%{%}
spotify(false) -  12 ms -
path(true)     -   0 ms - %{%}%{%}%{%}  oh-my-posh3%{%}%{%}█%{%}%{%}%{%}
git(false)     -  15 ms -
battery(true)  -   2 ms - %{%}%{%}%{%}█%{%}%{%}100 %{%}%{%}%{%}
node(false)    -   0 ms -
shell(true)    -   0 ms - %{%}%{%}%{%} ﲵ zsh%{%}%{%}█%{%}%{%}%{%}
root(false)    -   0 ms -
exit(true)     -   0 ms - %{%}%{%}%{%} %{%}%{%}█%{%}%{%}%{%}
PS H:\github\oh-my-posh3>

I haven't set my powershell up to render unicode correctly, but the segment timings seem in line with what I'm seeing.

Edit: as an aside, it's interesting that it thinks my desktop has a battery :) No need to derail the issue though.

Edit2: With powershell so I'm running the same command

PS H:\github\oh-my-posh3> C:\bin\posh-windows-amd64-experimental.exe --config C:\users\amol\.poshthemes\jandedobbeleer.omp.json --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █amol@amolhomevista█
spotify(false) -  10 ms -
path(true)     -   0 ms -   oh-my-posh3█
git(false)     -  16 ms -
battery(true)  -   3 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ powershell█
root(false)    -   0 ms -
exit(true)     -   0 ms -  █
PS H:\github\oh-my-posh3>

You should use --shell universal or --shell pwsh

[amoldeshpande]

hmm, measure-command in powershell shows significantly higher times for the same commands. Maybe there is an additional startup time factor ?

It's definitely 1/2 the time as measured by powershell and I can feel the difference interactively.

[amoldeshpande]

Can't get screenshots to paste

S C:\Users\amol> measure-command {C:\bin\posh-windows-amd64-experimental.exe --config C:\users\amol\.poshthemes\amol.omp.json --debug --shell pwsh}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 41
Ticks             : 410704
TotalDays         : 4.75351851851852E-07
TotalHours        : 1.14084444444444E-05
TotalMinutes      : 0.000684506666666667
TotalSeconds      : 0.0410704
TotalMilliseconds : 41.0704



PS C:\Users\amol> C:\bin\posh-windows-amd64-experimental.exe --config C:\users\amol\.poshthemes\amol.omp.json --debug --shell pwsh

Here are the timings of segments in your prompt:

text(true)  -   0 ms - █%c03█
git(false)  -  17 ms -
text(true)  -   0 ms - █%T█
exit(true)  -   0 ms -  █
PS C:\Users\amol> C:\bin\posh-windows-amd64.exe --config C:\users\amol\.poshthemes\amol.omp.json --debug --shell pwsh
█%c03█(text:0s)██(git:17.0074ms)█%T█(text:0s) █(exit:0s)
PS C:\Users\amol> measure-command {C:\bin\posh-windows-amd64.exe --config C:\users\amol\.poshthemes\amol.omp.json --debug --shell pwsh}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 80
Ticks             : 807655
TotalDays         : 9.3478587962963E-07
TotalHours        : 2.24348611111111E-05
TotalMinutes      : 0.00134609166666667
TotalSeconds      : 0.0807655
TotalMilliseconds : 80.7655

So the times measured by posh are almost the same for both versions.

[royvou]

If i'm correct changing to a custom Go build, should remove the 5 MS delay when it's cleaning up the processes, when you are not inside a git repo this should be once (The check "Is in git repo").

I replaced the original exe in my modules, and yes, this does remove that 5 MS delay.

 18:02:07   ███Roy██E:█ ﮫ 1ms█C:\Users\Roy\Documents\PowerShell\Modules\oh-my-posh\3.68.0\bin\posh-windows-amd64.exe --debug --shell universal

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █Roy@HOMEPC-ROY█
path(true)     -   0 ms -   \█
git(false)     -  41 ms -
battery(true)  -   2 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ universal█
root(true)     -   0 ms - ██
text(true)     -   0 ms - █NO CONFIG█
exit(true)     -   0 ms -  █
 18:02:10   ███Roy██E:█ ﮫ 67ms█C:\Users\Roy\Documents\PowerShell\Modules\oh-my-posh\3.68.0\bin\posh-windows-amd64-2.exe --debug --shell universal

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █Roy@HOMEPC-ROY█
path(true)     -   0 ms -   \█
git(false)     -  47 ms -
battery(true)  -   3 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ universal█
root(true)     -   0 ms - ██
text(true)     -   0 ms - █NO CONFIG█
exit(true)     -   0 ms -  █

When running it in the PowerToys Repo, it is within margin of error (and more dependant on the git performance itself).

[amoldeshpande]

ugh, ok I'm so dumb. I was running the powershell measurements in a shell that did not have git in the path. (Sidebar: maybe --debug should print if git was not found)

With git in the path, the measurements for both versions are almost identical at ~170ms.

I am still hallucinating that it feels faster in the new version though.

Edit: With Measure-command there is definitely a 40ms difference in the two, so that goes back to the startup/cleanup delays

[amoldeshpande]

Here is output with measure-command and --debug. It's interesting to see the 60-70ms overhead

PS H:\github\oh-my-posh3> measure-command { C:\bin\posh-windows-amd64.exe --config C:\users\amol\.poshthemes\amol.omp.json --debug --shell pwsh | Out-Default}
█%c03█(text:0s)█ main ≡  ~5 ?5█(git:164.0001ms)█%T█(text:0s) █(exit:0s)


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 231
Ticks             : 2317607
TotalDays         : 2.68241550925926E-06
TotalHours        : 6.43779722222222E-05
TotalMinutes      : 0.00386267833333333
TotalSeconds      : 0.2317607
TotalMilliseconds : 231.7607



PS H:\github\oh-my-posh3> measure-command { C:\bin\posh-windows-amd64-experimental.exe --config C:\users\amol\.poshthemes\amol.omp.json --debug --shell pwsh | Out-Default}

Here are the timings of segments in your prompt:

text(true)  -   0 ms - Γûê%c03Γûê
git(true)   - 173 ms - █ main ≡  ~5 ?5█
text(true)  -   0 ms - █%T█
exit(true)  -   0 ms -  █



Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 217
Ticks             : 2170004
TotalDays         : 2.5115787037037E-06
TotalHours        : 6.02778888888889E-05
TotalMinutes      : 0.00361667333333333
TotalSeconds      : 0.2170004
TotalMilliseconds : 217.0004

[JanDeDobbeleer]

@amoldeshpande that "old" version of OMP is also a bit outdated looking at the debug output. You might want to update that one to get a more up-to-date pov.

[amoldeshpande]

done. no idea where I picked up the old version since I only started using posh about a week ago :-) Anyway, no change in timings probably as expected.

[lnu]

done. no idea where I picked up the old version since I only started using posh about a week ago :-) Anyway, no change in timings probably as expected.

How did you install omp? scoop? pwsh?

[amoldeshpande]

done. no idea where I picked up the old version since I only started using posh about a week ago :-) Anyway, no change in timings probably as expected.

How did you install omp? scoop? pwsh?

Nope. just downloaded with webrequest from powershell. I have a fairly strong recollection of using 'releases/latest/download/...', but I could be wrong.

[JanDeDobbeleer]

I just realized I also need to revert back to exec.Command("command").Output() rather than the override to avoid calling process.Wait(). Ill try to do that and compile with the fork. See if that makes a difference.

[JanDeDobbeleer]

This build is with everything originally, but compiled with a go fork.
posh-windows-amd64-experimental.exe.zip

[amoldeshpande]

PS H:\github\tcsh\win32> C:\bin\posh-windows-amd64-experimental.exe --debug --config C:\Users\amol\.poshthemes\amol.omp.json

Here are the timings of segments in your prompt:

text(true)  -   0 ms - █%c03█
git(true)   - 173 ms - █ unicode ≡█
text(true)  -   0 ms - █%T█
exit(true)  -   0 ms -  █
PS H:\github\tcsh\win32> measure-command {C:\bin\posh-windows-amd64-experimental.exe --debug --config C:\Users\amol\.poshthemes\amol.omp.json}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 225
Ticks             : 2251493
TotalDays         : 2.60589467592593E-06
TotalHours        : 6.25414722222222E-05
TotalMinutes      : 0.00375248833333333
TotalSeconds      : 0.2251493
TotalMilliseconds : 225.1493

Seems about the same as the last version.

[JanDeDobbeleer]

Yup. That's what I have as well. That does imply we can shave off a couple of MS but not a lot. Command execution is simply slower on Windows. That said, below 300ms to render a prompt is still smooth.

[TylerLeonhardt]

Spinning up processes is simply more expensive on Windows... I heard this is why .NET Framework introduced App Domains ages ago... Most of the benefits of process isolation, without expensive bit of being a separate process.

[amoldeshpande]

@JanDeDobbeleer have you considered what I said about parallelizing git commands ? I don't see the harm in executing all the git commands in parallel and then parsing the output based on conditional "git status" states as usual.

[lnu]

@amoldeshpande It's not so easy. A couple of commands can be executed in // but others depend on the context.
At least we can give it a try.

[JanDeDobbeleer]

@amoldeshpande I'm going to give that a try next week. See where that brings us.

@TylerLeonhardt thanks for that context, I'm off Windows for a while now so there's a lot I'm missing.

[royvou]

Would it also be an idea, where appropriate, to replace an Cmd execution, with a small go function? I would expect this to be faster on both Windows & Linux.

For example, I think the check 'is in git folder' could be replaced with a simple check if there is a .Git folder in the current dir or any parent dir. Might be possible to replace some other cmd executions with some go code.

I think the az module could be changed as well to just read two config files!

[shedric1]

As royvou mentions, another approach is removing as many external command calls as possible. Same principle as using the go-git package, but with a simple manual implementation.

For example, the first git call is in the segment enable check, and only checks if the current working directory is inside a working tree. Replicating the same behavior using system calls through the os package should be easy; just check each folder between working directory and root for a .git subfolder. As far as I can tell, this is how the go-git package handles their PlainOpen function with DetectDotGit specified.

I got a very basic proof-of-concept working here if anyone else wants to try it out/test it: https://github.com/shedric1/oh-my-posh3/tree/git-treewalk

Some pretty promising results:

...\posh-windows-amd64-git-treewalk.exe --config C:\Misc\Resources\customtheme.omp.json --debug

Here are the timings of segments in your prompt:

path(true)           -   0 ms -    C:\Misc█
git(false)           -  20 ms -
exit(false)          -   0 ms -

For reference, here are my results of running the faster-commands build without any changes:

...\posh-windows-amd64-faster-commands.exe --config C:\Misc\Resources\customtheme.omp.json --debug

Here are the timings of segments in your prompt:

path(true)           -   0 ms -    C:\Misc█
git(false)           -  66 ms -
exit(false)          -   0 ms -

The same ~40-60ms savings are present for both repo folders and non-repo folders alike. Might be too small to warrant the potential edge-cases, but the fewer external command executions the better. Comparing against the experimental build, it's a smaller ~20-40ms savings, but still (slightly) faster consistently.

Not sure if it will cause performance issues on other OSes where external commands aren't so slow, so any Linux/macOS comparisons would be appreciated!

EDIT: looking closer, the process of walking parent dirs for a git repo by definition would also return the repo root directory. The second external git call only returns this path, so we could kill two external function calls with one stone here fairly easily.

[royvou]

This sounds great! 2x ~40ms would be a huge bonus. And would get us closer to under the 100ms.

I also did a little expiriment with go git, and as Jan already mentioned the git status equivalent, is slow with big repositories with a lot of excluded files.

Parrelize the other few git commands (or check if some could be merged) would be a huge bonus.

For the Linux check I did (on my own branch) it didnt really matter as it was always 4-6ms.

[shedric1]

These 3 seemed like the simplest candidates for native Go refactoring in the git segment:

• Remove external call by natively searching for git repo.
• Remove external call by saving repo root during enable check.
• Remove external call by count the lines in {ROOT_REPO}/.git/logs/refs/stash (if it exists)

I pushed a new proof-of-concept branch trying these changes here: https://github.com/shedric1/oh-my-posh3/tree/native-git

Using all three gave the following results on my machine inside a git repo directory:

...\posh-windows-amd64-native-git.exe --config C:\Misc\Resources\customtheme.omp.json --debug

path(true)           -   0 ms -    C:\Misc\oh-my-posh3█
git(true)            -  71 ms -  :faster-commands ≡  ?2  2
exit(false)          -   0 ms -

For reference, here's the experimental build from above:

...\posh-windows-amd64-experimental.exe --config C:\Misc\Resources\customtheme.omp.json --debug

path(true)           -   0 ms -    C:\Misc\oh-my-posh3█
git(true)            - 208 ms -  :faster-commands ≡  ?2  2
exit(false)          -   0 ms -

Still needs polishing, but in general it's a big performance improvement.

[amoldeshpande]

so, unfortunately, not seeing much improvement with this change either

S J:\github\UnrealEngine> posh-windows-amd64.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █amol@amolhomevista█
spotify(false) -  11 ms -
path(true)     -   0 ms -   UnrealEngine█
git(true)      - 1359 ms - █ 4.25 ↓15█
battery(true)  -   3 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ pwsh█
root(false)    -   0 ms -
exit(true)     -   0 ms -  █
PS J:\github\UnrealEngine> j:..\oh-my-posh3\src\oh-my-posh3.exe --config C:\Users\amol\.poshthemes\amol.omp.json --shell pwsh --debug

Here are the timings of segments in your prompt:

text(true)  -   0 ms - █%c03█
git(true)   - 1253 ms - █ 4.25 ↓15█
text(true)  -   0 ms - █%T█
exit(true)  -   0 ms -  █
PS J:\github\UnrealEngine>
PS H:\github\tcsh>

Edit: added output from the right branch, I think. some improvement.

Edit2: git status measurement for context

PS J:\github\UnrealEngine> Measure-Command {git status}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 212
Ticks             : 12129090
TotalDays         : 1.40382986111111E-05
TotalHours        : 0.000336919166666667
TotalMinutes      : 0.02021515
TotalSeconds      : 1.212909
TotalMilliseconds : 1212.909

[lnu]

While reducing the number calls, we can add parameters to add/remove informations from the git prompt(like it is done in starship). So people who only wants a subset of the info will have better performance and those who are willing to pay the price will get everything.

[lnu]

while digging in starship I found they use this library https://github.com/libgit2/libgit2(https://libgit2.org/) which has a go version(https://github.com/libgit2/git2go).
This will introduce more complexity but external call will be avoided and all git actions done using c.
https://github.com/lnu/oh-my-posh3/tree/feature/git2go

[amoldeshpande]

I will say that the test with UnrealEngine is unfair. "git status" by itself takes more than a second in that directory. But it does bring up questions about how far you can really take the performance.

[royvou]

I think we should, for the first part, go with @shedric1's branch which reduces the amount of unneeded git calls.

For performance, we should always include a git status alongside it, we can't beat that performance without some magic. (and git status is slower when there are submodules in the git repo).

Libgit2 might be an option, do we know how the git status command compares to git2s status? Also, this extra library won't prevent single exe deployment right?

[amoldeshpande]

Sounds reasonable to me. I added "git status" measurements to my comment about @shedric1's change.

My ideal goal would take into account the fact that a simple external command takes ~30-40ms on Windows. We should not take much more than that for most commands.

Someone up there made a suggestion of picking specific git states to print in the prompt. That would be perfect because the main thing I care about usually is the branch. Filters by path for each git part would be even better, but I realize that's just being greedy :-)

[royvou]

@lnu after checking the starship repo, please note they are removing libgit with calls to the native git executable for 1.0. So I don't think we should consider libgit at the current moment.

[JanDeDobbeleer]

Well, this thread just keeps growing :-)

@shedric1 I was looking at the same thing this morning, also found out the stash count is being done AT ALL TIMES which is a huge mistake on my part. I agree with swapping out the calls as much as we can, I do believe your changes are a candidate for this. How would you like to proceed? Do you want to give this a go, or do you want me to assist in merging this in properly? It will require a rewrite of tests, splitting functionality to environment, etc, so there's more work to be done than doing the changes alone.

About git2go, that still requires the libgit binary so we can't have a single exe in that case. I'd say avoiding doing git calls is the most straightforward approach here.

[shedric1]

@JanDeDobbeleer I won't have a chance to look into a proper version with updated tests and organization for a while, so anyone is welcome to get the ball rolling on it using that branch! I just whipped those up as a proof of concept to see if it worked and offered meaningful performance improvements, but I'm new to Go so a well formatted version might take me a lot longer than someone more familiar with the language. I'll take a crack at it next week if no one else takes up the mantle by then!

[JanDeDobbeleer]

I'll pick up the task. I can do this during lunchtime, I'll try to provide intermediate builds here for validation.

[amoldeshpande]

Another interesting datapoint. I use SourceTree and its embedded git version. Turns out it defaults to a really old one and you have to manually make it update. I went from git 2.11 to 2.20 when I forced the upgrade.

anyway, to get to the point, git status in my UnrealEngine repo is ~900 ms in 2.20 (so a good 300ms faster), and as a result posh comes in around 1.2 seconds instead of 1.5+

Just another variable to add to the matrix.

[gibwar]

I'm excited to see the progress in making the git segment faster (it's 81ms on my home computer, I'll update my work machine tomorrow with 3.86.3) but there is one thing I want to caution: finding git entries manually can be problematic if you're only looking for a .git _folder_.

I use git's worktrees pretty extensively and they only have a .git _file_, currently with a single entry of gitdir: c:/absolute/repo/.git/worktrees/name that link the two together. Currently 3.86.3 works with this setup and displays all output properly (branches, staging, push/pull distance, etc) and would hate to see it only work on full clone repositories.

[shedric1]

@gibwar Excellent point about worktrees, I'm not sure how the current proof of concept will work with those. As is, it's using the Stat function from the os package which should return true if a matching directory _or_ file is found, if I'm reading it correctly. If I'm understanding how worktrees work, this may cause the repo root it finds to be totally wrong.

Should be an easy enough fix though. An additional check can be used once any git path match is found to see if it was a folder or file and proceed as necessary from there. Of course this raises the question of what other git features we're forgetting to account for when manually implementing...

[amoldeshpande]

I put some timing around spawned command execution and came up with this:

J:\github\UnrealEngine>h:oh-my-posh3.exe --config C:\Users\amol\.poshthemes\amol.omp.json --shell pwsh --debug
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  rev-parse --is-inside-work-tree
43.0001ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false rev-parse --show-toplevel
44.0043ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false status -unormal --short --branch
850.9924ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false rev-list --walk-reflogs --count refs/stash
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false remote get-url upstream
37.9849ms

I think it shows what we already knew, that status is the slowest command. Figured it would helpful to have concrete numbers anyway.

Editing to add more timing investigation results:

If I change the status command to -uno instead of -unormal, that cuts the time down to 280-300-ish ms.

J:\github\UnrealEngine>h:oh-my-posh3.exe --config C:\Users\amol\.poshthemes\amol.omp.json --shell pwsh --debug
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  rev-parse --is-inside-work-tree
39.9979ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false rev-parse --show-toplevel
45.001ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false status -uno --short --branch
290.9995ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false rev-list --walk-reflogs --count refs/stash
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe  -c core.quotepath=false -c color.status=false remote get-url upstream
44.996ms

This command skips untracked files. However, to get the untracked files alone, one can do

git ls-files --others --exclude-standard --directory

this seems to give almost the same information as -unormal I am not enough of a git expert to understand the differences. It also takes 1000-ish ms to run.

All of this suggests a way to break up git status into smaller sub-commands that can presumably be parallelized.

A config option to skip untracked file counting might be a good short-term fix.

[JanDeDobbeleer]

If all goes well I'll have a working version tomorrow that uses paths to validate if we're in a git repo, including support for worktrees.

[JanDeDobbeleer]

As promised a build based on the changes made with @shedric1 's proposal. There are two executables in the zip, fork is build with the go fork that removes the wait in process. In my preliminary tests, this already has some benefits (don't mind the font not being set correctly).

Archive.zip

[royvou]

Not inside GIT repo

Inside git Repo

Git status takes ~256MS

Looks like an nice improvement! Seems to be an overhead of 20MS + the time it longest time it takes for 1 segment. The Fork would also be nice to keep in sync with GO, 40MS improvement is nice!

[JanDeDobbeleer]

Keeping the fork synced isn't going to be the challenge so I'm definitely in favor of keeping that. Cool that we're now seeing actual differences, that's clear progress!

[TravisTX]

Here' the relevant parts from my timings. Nice work!

❯ oh-my-posh --config "c:/users/travis.collins/.poshthemes/travis.omp.json" --debug

git(true)            - 138 ms - master

❯ posh-windows-amd64-experimental --config "c:/users/travis.collins/.poshthemes/travis.omp.json" --debug

git(true)            -  61 ms - master

❯ posh-windows-amd64-experimental-fork --config "c:/users/travis.collins/.poshthemes/travis.omp.json" --debug

git(true)            -  53 ms - master

[amoldeshpande]

not seeing much difference between fork and non-fork, but there's a 40ms improvement between the released version.
I had to run each version a few times to get somewhat stable measurements to account for the OS fs cache.

x> H:\Archive\posh-windows-amd64-experimental.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █amol@amolhomevista█
spotify(false) -  14 ms -
path(true)     -   0 ms -   ForgottenArmies_cpp█
git(true)      - 163 ms - █ htn ≡█
battery(true)  -   1 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ pwsh█
root(false)    -   0 ms -
exit(true)     -   0 ms -  █
x> H:\Archive\posh-windows-amd64-experimental-fork.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █amol@amolhomevista█
spotify(false) -  13 ms -
path(true)     -   0 ms -   ForgottenArmies_cpp█
git(true)      - 164 ms - █ htn ≡█
battery(true)  -   2 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ pwsh█
root(false)    -   0 ms -
exit(true)     -   0 ms -  █
PS x> C:\bin\posh-windows-amd64.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug

Here are the timings of segments in your prompt:

session(true)  -   0 ms - █amol@amolhomevista█
spotify(false) -   9 ms -
path(true)     -   0 ms -   ForgottenArmies_cpp█
git(true)      - 213 ms - █ htn ≡█
battery(true)  -   2 ms - █100 
node(false)    -   0 ms -
shell(true)    -   0 ms -  ﲵ pwsh█
root(false)    -   0 ms -
exit(true)     -   0 ms -  █
x>

[JanDeDobbeleer]

I'm testing all the different cases, so far so good. I'll already publish this part.

[JanDeDobbeleer]

@TravisTX @lnu @amoldeshpande @royvou @shedric1 @gibwar looking at the latest release, how do we feel about the speed? Improved enough to consider this solved, or do we need follow-up actions to go for the extra smile?

[TravisTX]

It's good for me (but it's always been pretty good on my computer). I'm at about 63ms in a git repo, and 7ms outside of one. Which is a notable improvement!

[amoldeshpande]

Latest release is perceptibly faster than the old one, so I'm fine with calling it good. It's about 100-120ms in a repo like posh3 for me.

thanks !

[Kudostoy0u]

Decent performance increase from 1s to 500ms, but I'm not getting speeds increases everyone else is having :/.

[JanDeDobbeleer]

@Kudostoy0u can you validate if there's an update of git for your system? That might also have an impact.

[Kudostoy0u]

Here are some versions of things I have:


I updated git to 2.30 (should have done that earlier) but still no difference :(

[amoldeshpande]

up in the thread I posted a comment with a few typical git commands that posh runs. measure those by commands themselves using the Measure-Command cmdlet. posh no longer runs all of them, but it will give you an idea of how slow git by itself is in your repo.

Add another ~40-80ms for go spawning subprocesses on Windows and you'll find your lower bound.

For example, my Unreal Engine repo went from 1200ms to 850-900ms. It's a big improvement, but still completely useless so I have a filter to turn posh off in that repo. This is not the fault of posh but just that git status takes a long time to run in that particular repo.

[Kudostoy0u]

BTW I have this function in my pwsh profile:

Hm... This is very erratic behavior. This is the result for an empty git repo, just a folder which I ran git init:

As soon as I create a file and add it, the performance increases

And that wasn't a one-off thing, I ran the command again and it reported 301 ms

[JanDeDobbeleer]

@Kudostoy0u is that the latest release or still one of the artifacts above?

[Kudostoy0u]

It is the experimental fork. I switched to using it since it does in fact induce quicker prompt response times when in a git repository. Here is my full PowerShell profile:

[JanDeDobbeleer]

@Kudostoy0u I would actually use the latest release as a lot of git calls were removed since then. It contains more optimizations than the fork right now.