Nixpkgs: "Bad file descriptor" and "Too many open files"

[FRidh]

Describe the bug
Switching NixOS to cfed29bfcb28259376713005d176a6f82951014a fails with

# nixos-rebuild switch
building the system configuration...
error (ignored): error: --- SysError ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
opening directory '/tmp/nix-build-users-groups.json.drv-2': Too many open files
error (ignored): error: --- SysError ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
opening directory '/nix/store/vyynl4q8f5ap34x1i3amfrh9wwh5j8m6-users-groups.json.drv.chroot': Bad file descriptor
error: --- SysError ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
opening directory '/nix/store/74asavs7xp33immp3yxa9kbjgsmq67z3-texlive-robustindex-49877': Too many open files

Not including texlive.combined.scheme-full worked around the issue.

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

Probably related https://github.com/NixOS/nixpkgs/issues/100281.

[andir]

Just to mention the workaround until we get to actually tackle the problem:

$ sudo mount -o remount,nr_inodes=1M /tmp

We are running out of inodes on /tmp, since systemd v246 the default limit there it 400k which isn't really enough for mixing some of our larger builds (chromium, firefox, …).

The line that is causing the problem is https://github.com/systemd/systemd/commit/7d85383edbab73274dc81cc888d884bb01070bc2#diff-ff9de8d0d78a6434e92fcd42eca4801d682efdc8c0539f901022b4374bd94a13R25.

One of the solutions that I'd like to see is a) reporting upstream & b) creating our own (customizable) mount unit instead of just importing this. The limit of 400k seems rather arbitrary and probably varies in usefulness between environments.

EDIT: the too many open files errors is actually a bit different from the error that I've been observing but might be the same source? @FRidh can you try with a stable version of Nix? Is nix maybe leaking FDs?

[FRidh]

Using a configurable mount unit is https://github.com/NixOS/nixpkgs/issues/14777. I recall there was some discussion on that topic, where some said we should use upstream unit files to keep up with changes they make.

[xaverdh]

For me just building texlive in isolation (with nix build) worked. But that's more of a workaround really.

[aaronjanse]

I also have this issue. Possible related; I switched to flakes today. I also had the issue while building texlive. The exact texlive package that fails appears to be random.

[jtojnar]

For me, building texlive on Ubuntu also caused this error, and I was able to fix it by increasing the soft limit on number of open file descriptors using ulimit -n 500000 (1024 by default). But I am using single-user install on my Ubuntu box so the fix will likely not apply to NixOS.

[FRidh]

See https://github.com/NixOS/nix/issues/4046. regarding leaking file descriptors.

[smaret]

I get a similar error:

error: --- SysError ------------------------------------------------------------------- nix
opening directory '/nix/store/d8sbz0scgbq1mj281cf2nvlwkqhc6h7q-ketcindy.r54074.tar.xz': Too many open files

on macOS 11.0 when installing TeXLive with flakes. Is this related?