Nixpkgs: bash-completion package doesn't work on (non-NixOS) Linuxes

[x80486]

Issue description

I installed bash-completion-2.8 via Nix Packages in Arch Linux. I've noticed that, even though that's installed, Git auto-completion (not gitflow-autocompletion) doesn't work. Moreover, I also installed gradle-completion-1.3.1 and it doesn't work either; so it looks like there is an issue with the integration of bash-completion into ~/.nix-profile – at least on non-NixOS linuxes.

I tried that also on Ubuntu 18.04.1 LTS. Refer to the discussion for #42799.

Steps to reproduce

On Linux (not NixOS), install bash-completion and gradle-completion via nix-env --install --prebuilt-package bash-completion gradle-completion.

Go to any Git project, try to auto-complete any command. Additionally, go to any Java/Kotlin project that uses Gradle as a build tool...and it won't auto-complete any Gradle task either.

Technical details

[gorre@uplink kotlin-grpc]$ nix-shell -p nix-info --run "nix-info -m"
these paths will be fetched (59.35 MiB download, 372.91 MiB unpacked):
  /nix/store/18ll3cxv888pmyhhkq68a8cd1d06kn42-bash-interactive-4.4-p23-info
  /nix/store/1hi76hr87bd1y1q1qjk0lv8nmcjip1c8-binutils-2.30
  /nix/store/4z04ym7w3vhk16h11a2fss25mb4ymg0j-bash-interactive-4.4-p23-doc
  /nix/store/517mx0hmmmhcl2f20g3lw96azixc6dl5-patchelf-0.9
  /nix/store/5d2njc16vdwq5ivc8y53cdlr63vmbzpf-gcc-7.3.0
  /nix/store/5yl80a6mvrvs7qjpacv2w8hfkvi5y7f6-findutils-4.6.0
  /nix/store/7s1s2brv9jkwy2m5crw1q9405dmp267g-gnumake-4.2.1
  /nix/store/7x63bxivrk58xwqxh3yzxa257dy1r769-gnugrep-3.1
  /nix/store/81q180k8ajw13fw22vglbvga3l9wblhj-xz-5.2.4
  /nix/store/a1zsyfbci72sa7j1jhqwjxnhzv99gppf-readline-7.0p5
  /nix/store/aa16jcadyam5r1zjy7fs4zi86vgm56lf-linux-headers-4.15
  /nix/store/ai2z7yrsy9h13ns3nhqzcx0010kvvvdg-pcre-8.42
  /nix/store/cbj1vh1b9yk7h724ync6ps7g7aiijiwb-ed-1.14.2
  /nix/store/d2r0ha0m1sk9bsh29snnbprfyhgmwllv-binutils-wrapper-2.30
  /nix/store/dbcn3vixwb892gzg0i8snpdqbnxfqd23-diffutils-3.6
  /nix/store/dxjnkd3rd244ipkc05xdpk6wgjzhlvzl-paxctl-0.9
  /nix/store/dzdm3n08sbb3g2ih2p3758ap68zd100v-stdenv-linux
  /nix/store/fz0s16k0lwb07fzh7j0rwwa0qpmnffqj-gawk-4.2.1
  /nix/store/g9mbwrsn10llm968hlaqvwnlb4frs984-gnused-4.5
  /nix/store/igrs1ns8i2vc1lifapyjxpn6xc53hk1k-gnutar-1.30
  /nix/store/ixsxalncig84f4157g2cykb8gp66dap0-expand-response-params
  /nix/store/m2mwfr63zh28wxy6zzmgns4vhjmyw0p1-nix-info
  /nix/store/mb1wy0rc4x8gj9lvdypfid6xil799wyz-patch-2.7.6
  /nix/store/qicx06y64b0mjwf60wwl3qz8py1ify01-gzip-1.9
  /nix/store/rzfwir2wavpf474jgixzm3jdyxnksf86-xz-5.2.4-bin
  /nix/store/slm6059ll30pawd22wwlj013cnkncifb-bzip2-1.0.6.0.1-bin
  /nix/store/wfq1z9jm9pdkmn43z6z893h1yylgpbmr-bash-interactive-4.4-p23
  /nix/store/wkfafgwgs1na5f808wybs8l6v8yjrvqz-gcc-wrapper-7.3.0
  /nix/store/y75amg1fc859ppgmvdg2klp0bi38v7x4-glibc-2.27-bin
  /nix/store/yr516apbvjbl5r5xiwzj9z72v77smw4y-bash-interactive-4.4-p23-dev
  /nix/store/z10kkr081zb02kyjw8ybf2s9zll49cvx-glibc-2.27-dev
  /nix/store/zb7zl93fr0k6gcz09lyv46war6xyjn1n-bash-interactive-4.4-p23-man
copying path '/nix/store/4z04ym7w3vhk16h11a2fss25mb4ymg0j-bash-interactive-4.4-p23-doc' from 'https://cache.nixos.org'...
copying path '/nix/store/18ll3cxv888pmyhhkq68a8cd1d06kn42-bash-interactive-4.4-p23-info' from 'https://cache.nixos.org'...
copying path '/nix/store/zb7zl93fr0k6gcz09lyv46war6xyjn1n-bash-interactive-4.4-p23-man' from 'https://cache.nixos.org'...
copying path '/nix/store/1hi76hr87bd1y1q1qjk0lv8nmcjip1c8-binutils-2.30' from 'https://cache.nixos.org'...
copying path '/nix/store/slm6059ll30pawd22wwlj013cnkncifb-bzip2-1.0.6.0.1-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/dbcn3vixwb892gzg0i8snpdqbnxfqd23-diffutils-3.6' from 'https://cache.nixos.org'...
copying path '/nix/store/cbj1vh1b9yk7h724ync6ps7g7aiijiwb-ed-1.14.2' from 'https://cache.nixos.org'...
copying path '/nix/store/ixsxalncig84f4157g2cykb8gp66dap0-expand-response-params' from 'https://cache.nixos.org'...
copying path '/nix/store/5yl80a6mvrvs7qjpacv2w8hfkvi5y7f6-findutils-4.6.0' from 'https://cache.nixos.org'...
copying path '/nix/store/fz0s16k0lwb07fzh7j0rwwa0qpmnffqj-gawk-4.2.1' from 'https://cache.nixos.org'...
copying path '/nix/store/y75amg1fc859ppgmvdg2klp0bi38v7x4-glibc-2.27-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/7s1s2brv9jkwy2m5crw1q9405dmp267g-gnumake-4.2.1' from 'https://cache.nixos.org'...
copying path '/nix/store/g9mbwrsn10llm968hlaqvwnlb4frs984-gnused-4.5' from 'https://cache.nixos.org'...
copying path '/nix/store/igrs1ns8i2vc1lifapyjxpn6xc53hk1k-gnutar-1.30' from 'https://cache.nixos.org'...
copying path '/nix/store/qicx06y64b0mjwf60wwl3qz8py1ify01-gzip-1.9' from 'https://cache.nixos.org'...
copying path '/nix/store/aa16jcadyam5r1zjy7fs4zi86vgm56lf-linux-headers-4.15' from 'https://cache.nixos.org'...
copying path '/nix/store/mb1wy0rc4x8gj9lvdypfid6xil799wyz-patch-2.7.6' from 'https://cache.nixos.org'...
copying path '/nix/store/z10kkr081zb02kyjw8ybf2s9zll49cvx-glibc-2.27-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/517mx0hmmmhcl2f20g3lw96azixc6dl5-patchelf-0.9' from 'https://cache.nixos.org'...
copying path '/nix/store/d2r0ha0m1sk9bsh29snnbprfyhgmwllv-binutils-wrapper-2.30' from 'https://cache.nixos.org'...
copying path '/nix/store/5d2njc16vdwq5ivc8y53cdlr63vmbzpf-gcc-7.3.0' from 'https://cache.nixos.org'...
copying path '/nix/store/dxjnkd3rd244ipkc05xdpk6wgjzhlvzl-paxctl-0.9' from 'https://cache.nixos.org'...
copying path '/nix/store/ai2z7yrsy9h13ns3nhqzcx0010kvvvdg-pcre-8.42' from 'https://cache.nixos.org'...
copying path '/nix/store/a1zsyfbci72sa7j1jhqwjxnhzv99gppf-readline-7.0p5' from 'https://cache.nixos.org'...
copying path '/nix/store/7x63bxivrk58xwqxh3yzxa257dy1r769-gnugrep-3.1' from 'https://cache.nixos.org'...
copying path '/nix/store/wfq1z9jm9pdkmn43z6z893h1yylgpbmr-bash-interactive-4.4-p23' from 'https://cache.nixos.org'...
copying path '/nix/store/wkfafgwgs1na5f808wybs8l6v8yjrvqz-gcc-wrapper-7.3.0' from 'https://cache.nixos.org'...
copying path '/nix/store/yr516apbvjbl5r5xiwzj9z72v77smw4y-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/m2mwfr63zh28wxy6zzmgns4vhjmyw0p1-nix-info' from 'https://cache.nixos.org'...
copying path '/nix/store/81q180k8ajw13fw22vglbvga3l9wblhj-xz-5.2.4' from 'https://cache.nixos.org'...
copying path '/nix/store/rzfwir2wavpf474jgixzm3jdyxnksf86-xz-5.2.4-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/dzdm3n08sbb3g2ih2p3758ap68zd100v-stdenv-linux' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 4.14.56-1-lts, Arch Linux, noversion`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.0.4`
 - channels(gorre): `"nixpkgs-18.09pre147772.d1ae60cbad7"`
 - nixpkgs: `/home/gorre/.nix-defexpr/channels/nixpkgs`

[hedning]

bash-completion require you to source etc/profile.d/bash_completion.sh in eg. .bashrc. It will then look for completion scripts in $XDG_DATA_DIRS/share/bash-completion/completions/.

Installing bash-completion with the native package manager (it's most likely already installed though) and adding ~/.nix-profile/share to XDG_DATA_DIRS should make things work.

I'm guessing that the installer should really add the profile's share/ to XDG_DATA_DIRS by default (might be a reason why it's not doing it, but I'd suspect it's just an oversight). Automatically sourcing the bash-completion setup might be trickier and is probably unnecessary as it's installed by default on most distros.

[x80486]

I did try adding ~/.nix-profile/share to XDG_DATA_DIRS before posting, but that doesn't quite work.

[gorre@uplink kotlin-grpc]$ echo $XDG_DATA_DIRS
/home/gorre/.nix-profile/share/:/home/gorre/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/

This is an excerpt of my .bashrc:

# Nix Package Manager
if [ -e $HOME/.nix-profile/etc/profile.d/nix.sh ]; then source $HOME/.nix-profile/etc/profile.d/nix.sh; fi
# Completions
if [ -e $HOME/.nix-profile/etc/profile.d/bash_completion.sh ]; then source $HOME/.nix-profile/etc/profile.d/bash_completion.sh; fi
if [ -e $HOME/.nix-profile/etc/bash_completion.d/git-completion.bash ]; then source $HOME/.nix-profile/etc/bash_completion.d/git-completion.bash; fi
if [ -e $HOME/.nix-profile/share/bash-completion/completions/gradle ]; then source $HOME/.nix-profile/share/bash-completion/completions/gradle; fi
#export XDG_DATA_DIRS="$HOME/.nix-profile/share/:$XDG_DATA_DIRS"

In that way it works as it should be – with export XDG_DATA_DIRS or not.

On the other hand (which is what I was expecting to work as such), if I do this:

# Nix Package Manager
if [ -e $HOME/.nix-profile/etc/profile.d/nix.sh ]; then source $HOME/.nix-profile/etc/profile.d/nix.sh; fi
# Completions
#if [ -e $HOME/.nix-profile/etc/profile.d/bash_completion.sh ]; then source $HOME/.nix-profile/etc/profile.d/bash_completion.sh; fi
export XDG_DATA_DIRS="$HOME/.nix-profile/share/:$XDG_DATA_DIRS"

...nothing works. All the completion scripts inside $HOME/.nix-profile/share/ are not applied – even if I uncomment the line with bash_completion.sh.

[hedning]

Hmm, that's really strange. Does this work (relevant code in bash-completion)?

export BASH_COMPLETION_USER_DIR=$HOME/.nix-profile/share/bash-completion/completions

You're getting completion from eg. /usr/share?

You could check complete -p | grep -- -D which should output something like this complete -F _completion_loader -D. I guess there's a chance there's something else hogging the default completion (not sure why, but worth checking I guess). Then you can check type _completion_loader and type __load_completion to see if you're getting code that looks sane.

[x80486]

There is not output for any of the previous if I don't first source $HOME/.nix-profile/etc/profile.d/bash_completion.sh – and I'm also correctly sourcing the current shell whenever I make any change(s) in $HOME/.bashrc.

I don't have a $HOME/.local/share/bash-completion/../../, but strangely enough (for me) I do have a /usr/share/bash-completion/completions/ – I didn't noticed it before. I'm sure I've never installed (directly) bash-completion-x.y-z using Pacman and I don't have that package currently installed:

[gorre@uplink ~]$ sudo pacman -Q bash-completion
error: package 'bash-completion' was not found

[hedning]

A lot of packages come with bash completion bundled, which in Arch's case will be installed in /usr/share/bash-completion/completions, regardless of bash-completion being installed or not.

No idea what the problem is really, but here's some troubleshooting steps to try with $HOME/.nix-profile/etc/profile.d/bash_completion.sh sourced and xdg_data_dirs set:

1. Check if you're getting completion for eg. wget. If so bash-completion is at least in somewhat working order
2. Check if any commands listed in /usr/share/bash-completion/completions gives you proper completion.
3. Check type _gradle after having attempted completion with gradle once. If you're getting some bash code as output it means bash-completion sets things up properly, but there's something wrong with the gradle completion
4. Install some other bash completion script (eg. nix-bash-completions) with nix-env and check if that works

If nothing of the above works you can try setting up bash-completion from pacman (if you haven't already tried that).

If things doesn't work then I would take look at .bashrc and see if you're eg. unsetting some option bash-completion expects to be on. A quick way to troubleshot this is sourcing bash_completion.sh at the bottom of your bashrc.

[x80486]

I installed nix-bash-completions-0.6.6; modified my .bashrc shell script as:

# Nix Package Manager
if [ -e $HOME/.nix-profile/etc/profile.d/nix.sh ]; then source $HOME/.nix-profile/etc/profile.d/nix.sh; fi
if [ -e $HOME/.nix-profile/etc/profile.d/bash_completion.sh ]; then source $HOME/.nix-profile/etc/profile.d/bash_completion.sh; fi
export XDG_DATA_DIRS="$HOME/.nix-profile/share/:$XDG_DATA_DIRS"

...started a completely new shell, and now things work! It's funny because I've tried it previously and I usually keep track of all changes to (try to) revert things later on to a previous (known) state...but right now I don't know what changed. In any case, I would say everything should be fine, except for Gradle:

[gorre@uplink completions]$ pwd
/home/gorre/.nix-profile/share/bash-completion/completions
[gorre@uplink ~]$ echo $XDG_DATA_DIRS
/home/gorre/.nix-profile/share/:/home/gorre/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/
[gorre@uplink completions]$ ll | grep gradle
lrwxrwxrwx 1 gorre 108 Dec 31  1969 gradle -> /nix/store/7ylhrp638k8rfa7ry3k9mnr22rlyv9pa-gradle-completion-1.3.1/share/bash-completion/completions/gradle
[gorre@uplink completions]$ type _gradle
bash: type: _gradle: not found

Something I don't fully understand is: from where bash_completion.sh is reading/sourcing the completions' list? I'm saying that because when I saw that Gradle wasn't working I thought that it was reading those from /usr/share/bash-completion/completions/; so I backup and deleted the content of that sub-directory, but still, everything works. I also commented out export XDG_DATA_DIRS="$HOME/.nix-profile/share/:$XDG_DATA_DIRS"...and still, wget, grep, nix-env, etc., behaves as expected.

---

Also, looks like for Git, it's necessary to source manually: $HOME/.nix-profile/etc/bash_completion.d/git-completion.bash.

[hedning]

Glad its mostly working :)

bash_completion.sh simply sources this file bash_completion. That will set _completion_loader as the default completion handler.

The _completion_loader will run __load_completion with the current command as the first argument. That will look for files in $BASH_COMPLETION_USER_DIR, in $XDG_DATA_DIRS and the package's own completions/ directory (so things like wget will work no matter what XDG_DATA_DIRS is).

If it can find a file matching the command name (possibly with a .bash suffix, or _ prefix) it will source that file, which should set up proper completion for the command in question.

Changing XDG_DATA_DIRS after using completion on eg. nix-env won't have any effect on nix-env completion as the necessary functions are already loaded, and complete -F _nix nix-env have told bash that nix-env should use _nix as its completion function. You'll need to start a new shell to see any effect here.

/etc/bash_completion.d isn't lazy loaded and meant to be taken care of here, so you might want to set export BASH_COMPLETION_COMPAT_DIR=$HOME/.nix-profile/etc/bash_completion.d.

Hope that clarifies things.

[hedning]

Would you mind renaming or closing the issue, as the problem seems to be with gradle/gradle-completion.

[x80486]

Closing this one, since I already have another one for Gradle O:)

Thanks again for the tips!