Nixpkgs: Nonexistent directories in PATH

[mrobbetts]

Issue description

My PATH appears to have several directory entries that do not really exist. Specifically, I have:

/home/[USER]/bin
/etc/per-user-pkgs/[USER]/bin
/home/[USER]/.nix-profile/bin
/nix/var/nix/profiles/default/bin

Where do these come from? Are they needed in some instances or are they truly erroneous?

It wouldn't matter much, but I use the Fish shell, and receive:

set: Warning: $PATH entry "/home/[USER]/bin" is not valid (No such file or directory)
set: Warning: $PATH entry "/etc/per-user-pkgs/[USER]/bin" is not valid (No such file or directory)
set: Warning: $PATH entry "/home/[USER]/.nix-profile/bin" is not valid (No such file or directory)
set: Warning: $PATH entry "/nix/var/nix/profiles/default/bin" is not valid (No such file or directory)

each time I log in :)

Technical details

• System: (NixOS: nixos-version, Ubuntu/Fedora: lsb_release -a, ...)
  17.09.git.3f8f8bf (Hummingbird)

• Nix version: (run nix-env --version)
  nix-env (Nix) 1.11.11

• Nixpkgs version: (run nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion)
  "17.09.git.3f8f8bf5f6"

• Sandboxing enabled: (run grep build-use-sandbox /etc/nix/nix.conf)
  build-use-sandbox = true

[matthewbauer]

see #25712 for reference on this

[vcunat]

Hmm, that could be worked around by using symlinks to an empty directory in nix store (shared fixed-output build). Or one could try to silence fish somehow :-)

[rnhmjoj]

The warning was only recently added into fish and I'm not aware of any way to hide it.
The motivation probably was when you manually do set PATH <not-existing-path> $PATH you can save yourself some time finding a typo but having non-existing directories in PATH should cause no problem at all.

[Mic92]

The following seems to work in fish:

set PATH /i/do/not/exists $PATH ^/dev/null

[faho]

The warning was only recently added into fish and I'm not aware of any way to hide it.

Nope, unless your definition of "recently" stretches to 2006 :smile:. (Though they were tweaked later)

So, I'm trying to understand this. Which script is responsible for getting the paths into fish? Why does it set them unconditionally? Why wasn't this noticed before?

Because it would be easy to write:

for c in $components
    test -d "$c"; and set PATH $PATH $c
end

Or is the logic more complicated (some components pre- and some appended)?

It would be possible for us at fish-shell upstream to remove the warning. Personally I've never been a massive fan, but I've also never really thought about it too much, so I'd like to hear some arguments.

[rnhmjoj]

Nope, unless your definition of "recently" stretches to 2006

That's strange, I believe it started to appear since fish 2.3.0.

Which script is responsible for getting the paths into fish?
Why does it set them unconditionally?

I believe the source is nixos/modules/programs/environment.nix.
The file itself says the options should be moved in more appropriate places.

The environment file is then sourced using foreign-environment in nixos/modules/programs/fish.nix
(fenv source ${config.system.build.setEnvironment})

[vcunat]

It's probably some recent change in environment variable setup on NixOS.

[therealpxc]

This is due to our use of the foreign-env plugin for Fish initialization on NixOS. NixOS generates POSIX shell scripts for setting up the environment, and the fish foreign environment plugin simply runs those scripts, monitors the environment changes, and then reproduces them in fish. It doesn't suppress error messages, and it does not check to see whether files exist in the PATH variable and similar, so effectively neither do we.

We used to suppress all messages during the NixOS portion of fish initialization (an oversight) and so all this was invisible for a time.

[rnhmjoj]

What about something like this?

diff --git a/functions/fenv.apply.fish b/functions/fenv.apply.fish
index 34a25e3..6a68185 100644
--- a/functions/fenv.apply.fish
+++ b/functions/fenv.apply.fish
@@ -29,7 +29,10 @@ function fenv.apply
         set value (echo $variable | sed 's/[^=]*=//')

         if test "$key" = 'PATH'
-          set value (echo $value | tr ':' '\n')
+          for i in (echo $value | tr ':' '\n')
+            test -d "$i"; and set path $path "$i"
+          end
+          set value $path
         end

         set -g -x $key $value

[therealpxc]

I think that's good for our purposes, but I also think it deviates from expected behavior for fenv in general because bash doesn't do such checks and will happily add such paths, and it also means that NixOS users using fish will have additional environment differences to NixOS users using bash. Do we care about that?

Maybe as an alternative (I don't love this) , we can check for a non-existent path and in that case redirect stderr for that ´set´ call to a log file or /dev/null?

This way we'll get identical environments to those of bash users but still suppress those warnings.

What do you think?

[rnhmjoj]

Right, one expects fenv to produce the exact same environment for fish as any other shell. Let's just add a ^ /dev/null.

[Mic92]

Don't adding them by default leads to unexpected behavior. If you install your first package to ~/.nix-profile/bin, you expect to be able to immediately use it also the directory might not have been there before.

[0xABAB]

We already have three attempts now and they are all wrong. :-1:

[therealpxc]

I'm surprised at how strongly you seem to feel about this, @0xABAB. To be frank, all we needed here was a quick hack anyway.

Right now, because it was easy to implement and it works, we are using an oh-my-fish plugin to run that script with bash, monitor environment changes, and then reproduce them in fish. Why is that not your major concern, if you're so worried about fish initialization being fast and sensible?

So the fact that we use fenv for this purpose is already a hack. And this matter of unappealing warning messages which appear once per login shell (and not for child shells) is entirely cosmetic. If you want to make an improvement to fish initialization which replaces all this ugliness with something much more sane, I suggest you look into NixOS' config.system.build.setEnvironment, the POSIX (or at least non-fishy; it may contain bashisms, whatever) shell script which sets up NixOS environments during shell initialization, and create a fish-based companion script which is written to avoid these expected error messages and add that to NixOS. If you do that, we can drop our dependency on fenv entirely for setting up the basic Nix environment (we may still want it so that the global NixOS options relating to shell initialization can (ideally optionally, imo) affect fish users (and there we might not want to suppress these warnings by default).

Your advice could use some perspective as well as some tact, my friend.

[0xABAB]

@therealpxc I don't see the point of adding code that doesn't completely solve a problem, because it means a project will never be done.

I have never approved of any solution, in case you didn't notice. I rejected your solution, remember? I only said it was superior to what was merged now. That doesn't mean it's good.

[rnhmjoj]

Can the issue be considered solved?

[Mic92]

I probably should also backport your pull request.

[Mic92]

also in 17.03 now: 94941cb0455bfc50b1bf63186cfad7136d629f78