Nixpkgs: sshd fails to start on 16.09: /var/emtpy must be owned by root and not group or world-writable

[domenkozar]

Possible cause https://github.com/NixOS/nixpkgs/commit/4f8f1c30cbfb3e5e346a7ad1a699d3c8b3a46605 /cc @rickynils

[domenkozar]

Following didn't help:

• chmod 400 /var/emtpy

Following did help:

• rm -f /var/empty & mkdir /var/empty

[domenkozar]

Deploying fresh 16.03 and 16.09 on top with sshd running doesn't yield this problem.

Maybe something created files in /var/emtpy?

[domenkozar]

Aha, here's what I have on my system:

$ ls /var/empty/ -la
total 4
dr-xr-xr-x 1 root root    8 Oct 31  2015 .
drwxr-xr-x 1 root root  136 Sep  4 18:58 ..
-rw------- 1 root root 1024 Oct 31  2015 .rnd

[domenkozar]

http://manpages.ubuntu.com/manpages/precise/en/man1/rand.1ssl.html:

 The rand command outputs num pseudo-random bytes after seeding the
       random number generator once.  As in other openssl command line tools,
       PRNG seeding uses the file $HOME/.rnd or .rnd in addition to the files
       given in the -rand option.  A new $HOME/.rnd or .rnd file will be
       written back if enough seeding was obtained from these sources.

Maybe some service has HOME=/var/empty set when it runs rand?

[edolstra]

We could make /var/empty immutable to prevent this kind of problem.

[grahamc]

Making it immutable seems like a good long-term permanent solution.

[domenkozar]

Not sure it was /var/empty/.rnd fault, I've tested on my machine and sshd starts regardless if I have /var/empty/a file.

[obadz]

Can't repro with

nix-build -E 'with import <nixpkgs/nixos> {
    configuration = {
      services.openssh.enable = true;
      users.extraUsers.root.initialHashedPassword = "";
      virtualisation.graphics = false;
    }
  }; vm' &&
./result/bin/run-nixos-vm

sshd starts fine in there. Am on 7c377f8.

[obadz]

Also starting the VM on release-16.03 on a clean filesystem, and then starting it again on 7c377f8 on the same filesystem doesn't seem to be a problem either.

[domenkozar]

I assume some service changed /var/empty, since it's the default directory for users.

This machine has been upgraded for a few NixOS stable releases, so it must be that it was writable at one time or changed by root.

[domenkozar]

I think making /var/empty immutable sounds correct, since this will pop up again.

[domenkozar]

Hmm, could this be a duplicate of https://github.com/NixOS/nixpkgs/issues/14910 ?

[obadz]

Only if createHome = true I guess?

[domenkozar]

Writting an assertion now to get cases when createHome = true and home = /var/empty.

[domenkozar]

Nothing sets that as of today, but let's make /var/empty immutable so this won't be able to happen in future.

[domenkozar]

https://github.com/NixOS/nixpkgs/pull/18365

[kvz]

There may very well be a million things I'm doing wrong, but this is just a friendly hint to let you know I had this break on me in the Riak package. Riak wants a user which does not specify a homedir, so /var/empty is used from what I understand. Riak's Erlang backend sets a ~/.erlang.cookie file, which location is impossible to change (I googled hard).

Rolling back won't fix my issue, as the old version does not contain code to revert this (e.g. chattr -i /var/empty). So instead I now added some Bash to revert this on my servers until I can think of a better way to fix this.

As said, just to let you know. I'm a newcomer to this, and very thankful for NixOS' existence and the hard work you're putting in. I'm also not contesting this particular change, it seems sounds.