Asdf: Completion changes in v0.77 causing slow startup and invalid zcompdump file location

[andrewthauer]

Overview

Changes in v0.77 to asdf.sh now call autoload -Uz compinit & compinit automatically when using zsh. This has the following issues:

• The compinit command should only be called once per shell startup in zsh
• compinit is often called with different arguments. Sometimes this is to use a cached version or to relocation the .zcompdump file. Making assumptions on how it's called undermines any customization.
• This can drastically slow down the shell startup time as it could already be called somewhere else by a framework or customized setup. On my machine this adds on average an additional 1 second delay.

See https://github.com/asdf-vm/asdf/blob/master/asdf.sh#L37

Steps to reproduce

• Make sure your using zsh
• Run /usr/bin/time zsh -i -l -c exit to get a baseline
• Comment out lines 37 & 38 from asdf.sh
• Run /usr/bin/time zsh -i -l -c exit again and compare

NOTE: This could vary depending on a bunch of factors.

Expected behavior

There should be no noticeable difference in startup time. It also shouldn't affect the behaviour of any custom zsh completion initialization.

Ideally only the fpath is set and asdf assumes zsh completions will be initialized later, either by a zsh framework or manually in the .zshrc (or equivalent). Therefore documenting this instead of automatically calling it will be less intrusive.

Actual behavior

Shell startup time is significantly slower then in v0.76. It can also affect the behaviour of the zsh completion initialization in unwanted ways. For instance, creating a .zcompdump file when the location has been moved from the default location using compinit -i -d "$_zcompdump".

Environment

macOS

asdf version:

0.77

[jthegedus]

Hi @andrewthauer , thanks for the thorough explanation.

Looking at the diff between https://github.com/asdf-vm/asdf/compare/v0.7.6...v0.7.7 the only changes to this file were

- if [ -n "$ZSH_VERSION" ]; then
-   autoload -U bashcompinit    
-   bashcompinit
+ if [ -n "${ZSH_VERSION:-}" ]; then
+   fpath=(${ASDF_DIR}/completions $fpath)
+   autoload -Uz compinit
+   compinit
+ fi

the bulk of which came from https://github.com/asdf-vm/asdf/pull/544

The reason I made the change from bashcompinit to compinit was because in the latest documentation we recommended to ZSH users to use this should their terminal have issues finding it.

Our old docs:

echo -e '\n. $HOME/.asdf/asdf.sh' >> ~/.zshrc
echo -e '\n. $HOME/.asdf/completions/asdf.bash' >> ~/.zshrc

If you are not using a framework, or if on starting your shell you get an error message
like 'command not found: compinit', then add this line before the ones above.

>autoload -Uz compinit && compinit

However, we have for a very long time been using one of these two. I think perf comparisons here should be made to using bashcompinit as opposed to nothing as it would better reflect the state of v0.7.6

My comment in the culprit PR covers the history of using bashcompinit/compinit in an automated fashion for ZSH users

Use compinit as per latest instructions in README.
in #69 we changed to bashcompinit for zsh users in this script, but then in #270 instructed users to use compinit in the README. We will go with the zsh communities latest suggestion unless someone raises an issue with this change? - https://github.com/asdf-vm/asdf/pull/544#discussion_r379262375

These were my test results:

asdf: v0.7.7
macOS: 10.15.3

Average of 3 runs each & by no means scientific:

compinit:
        0.43 real         0.25 user         0.19 sys
bashcompinit:
        0.40 real         0.23 user         0.19 sys
nothing (asdf.sh L37-38 commented out):
        0.39 real         0.23 user         0.18 sys

Were there other changes in your system during this time? Technically a slowdown, but not one I have perceived across these two versions of asdf.

compinit is often called with different arguments. Sometimes this is to use a cached version or to relocation the .zcompdump file. Making assumptions on how it's called undermines any customization.

Would it be correct to assume you are using a custom compinit configuration and were not changing the default bashcompinit, hence why you've not seen this issue before?

A solution which doesn't alter the current behaviour for the existing users but that accommodates custom configurations like yours is preferred in this case. Perhaps we can guard the autoload of compinit? I am open to suggestions, though I don't believe we should remove this entirely as it has been common usage of asdf for quite some time.

[andrewthauer]

@jthegedus - Thanks for the quick reply. I can't speak so much as per the previous changes and decisions that led to this state, so I'll try and outline my situation and thoughts on it instead.

• I have a hand rolled zsh setup and do not use any zsh frameworks or plugin loaders. This has allowed me to leverage many advanced features but keep the startup time very minimal (avg 10-15ms)
• I haven't changed much in my setup recently. I did do a brew upgrade which pulled in a new version of zsh v 5.8 though. I was running into issues right away and originally thought it was related to zsh. Further investigation lead me to this change in v0.77 which was pulled in at the same time.
• I do use use a custom compinit call which is called later in my .zshrc initialization. It both caches zcompdump files to improve loading (borrowed from prezto) as well as redirects the default file location to be XDG compliant. This optimization in my setup no longer works.
• I'm doing nothing with bashcompinit as it makes no sense in this case as I'm leveraging zsh proper completions. I'm not sure why this was chosen in the first place when using zsh. To my knowledge this just makes it possible to use bash completions in zsh (which have different formats).
• I leverage zsh-completions and other completions heavily which is why it's really costly to run (hence the optimization). I'm not sure on your setup, but if you do not have many zsh completions in your fpath then it might not be noticeable.
• From what I've read about zsh completions, etc. is that you should only call compinit once per a zsh shell session. Frameworks like oh-my-zsh call compinit automatically for you. It needs to be called after modifying the fpath. This is illustrated in the oh-my-zsh. Completions won't work if the fpath is modified after calling compinit. However, I strongly believe this should be the responsibility of the zshrc setup (or framework) and not an library (as is the case here).
• The homebrew zsh docs talk about this fpath sequence order. However, I believe the intention is that this is done by the user of homebrew and not libraries / packages themselves.

So imo, the original suggestion in the documentation to have a user call autoload -Uz compinit && compinit makes sense. Calling compinit here is likely changing the behaviour for everyone if they know it or not.

I think it's safe to say I have issues with #544 (as per above).

In the short term, I'd be happy with an env flag to override this behaviour. However, longer term (or now) I think it would be best to avoid calling compinit and rely on user's zsh setup to do this. This seems much more inline with behaviour that I've seen for other zsh scripts, tools, etc. NOTE: This implies that asdf is initialized first and/or the custom fpath is appended before compinit is called by the users zshrc.

[Strayer]

This also causes issues with zinits turbo mode. When sourcing asdf in .zshrc, this error message now appears: $HOME/.asdf/completions/asdf.bash:68: command not found: complete

I do have to admit that I don't fully understand how zinits turbo mode works though, I just wanted to mention another side-effect of this forced loading of completions. Previously I only sourced the asdf.sh and then told zinit to source the completions script after initializing the completions.

https://github.com/psprint/zinit#calling-compinit-with-turbo-mode

[jthegedus]

Thanks again for the thorough feedback @andrewthauer And thanks for letting us know @Strayer that this has touched more of the community.

After thinking this over I am leaning more towards removing the automatic calls in favour of clearer setup instructions. I will raise a PR soon with the proposed changes.

[jthegedus]

EDIT: v0.7.8 was released https://github.com/asdf-vm/asdf/pull/678#issuecomment-602593454

Apologies again. Keep your eyes open for a new version release, or try asdf update --head to get the latest master branch

[Boubalou]

Updated to latest v0.7.8 to no avail. Still getting the error:

/Users/boubalou/.asdf/completions/asdf.bash:68: command not found: complete

[andrewthauer]

Updated to latest v0.7.8 to no avail. Still getting the error:

/Users/boubalou/.asdf/completions/asdf.bash:68: command not found: complete

My guess is you are souring the bash completions but using zsh. Just remove the line where you are manually sourcing the bash completions as it makes no sense with zsh.

I believe the updated docs talk about how to properly setup completions moving forward for both zsh and bash.

[jthegedus]

@Boubalou The oh-my-zsh asdf plugin still uses the bash completions if that's how you're sourcing asdf?

[Boubalou]

Right guys! For the records, here is what I had to remove in my .zshrc:

...
alias zshconfig="sublime ~/.zshrc"
alias ohmyzsh="sublime ~/.oh-my-zsh"

DEFAULT_USER=$USER

. $HOME/.asdf/asdf.sh
. $HOME/.asdf/completions/asdf.bash
...

The last line: . $HOME/.asdf/completions/asdf.bash is the one in fault here. Happy sourcing, boys and girls. 🎉

[3fernandez]

Thanks guys, I've followed the tips here and I was able to get rid of .asdf/completions/asdf.bash:68: command not found: complete while loading my .zshrc.

However, asdf completions working, is that right? 😞

[jthegedus]

@3fernandez Now with a clean slate you should read the 🆕 instructions at https://asdf-vm.com/#/core-manage-asdf-vm under the ZSH tab as setting up the completions is a manual process

[Boubalou]

@jthegedus Interesting! Your post made me dig a little further and I noticed I could use the Oh My Zsh plugin, which led me to get the issue once again!

Turns out I had to manually upgrade Oh My Zsh, which updated the plugin and now everything works wonderfully. (my Oh My Zsh version was so old, it didn't have auto-update flags)

N.B.: If you start using the plugin from Oh My Zsh, make sure to also remove this line in your .zshrc:

. $HOME/.asdf/asdf.sh