Nix: Darwin sandbox build fails with "fork: Operation not permitted"

[edolstra]

E.g. with Nix 9be037448ce353d932249e91466c219670d334df, Nixpkgs 15.09:

$ nix-build --option build-use-chroot true -E 'with import <nixpkgs> {}; runCommand "foo" { } "touch $out"'
building path(s) ‘/nix/store/hnzwbm53jds3kzv4ic360kpjib397x45-foo’
/nix/store/vzk0677wwysshiffn05cfjaq19sl8qfp-stdenv-darwin/setup: fork: Operation not permitted
builder for ‘/nix/store/hy4qcbcapvscwi1clp32hdqva2d9v4ma-foo.drv’ failed due to signal 11 (Segmentation fault: 11)
error: build of ‘/nix/store/hy4qcbcapvscwi1clp32hdqva2d9v4ma-foo.drv’ failed

This is caused by the fact that a bunch of required sandbox permissions (in particular, allow process-fork) have been moved from Nix to pkgs/stdenv/pure-darwin/standard-sandbox.sb in Nixpkgs. These permissions should be provided unconditionally, not left to the Nix expression to get right.

[copumpkin]

I might try to revamp the Nix sandboxing mechanism in the next few weeks, but your point makes sense to me.

[datakurre]

This is my error with the same command. (Built Nix from https://github.com/NixOS/nix/commit/9be037448ce353d932249e91466c219670d334df and used nixpkgs master https://github.com/NixOS/nixpkgs/commit/fde89b6762de859a738264750bd184205b9d18a5 .)

these derivations will be built:
  /nix/store/29l1n6p1j948prnipr32b5ci2bl69chb-foo.drv
building path(s) ‘/nix/store/9pfhjiljxgkjd75kpwzfpjm475d3w2ix-foo’
killing process 25740
killing process 25740: Operation not permitted
error: while setting up the build environment: changing into ‘/private/var/folders/y6/2zdynm9x3nv6sq5p6780dwdc0000gn/T/nix-build-foo.drv-0’: No such file or directory

[pikajude]

Sandbox profiles were in nix until I realized that keeping them there was unmaintainable due to how rapidly I needed to iterate on them. @shlevy signed off on and merged the PR that moved profiles out of nix and into nixpkgs. Keeping nix master up to date with nixpkgs master added an extra layer of headaches to the ordeal, which is another reason for the switch.

I'm fine with moving _some_ profiles back into nix, but we need to work out an appropriate minimum level of permissions. If we are willing to source profiles from both nix and nixpkgs, I recommend just copyiing standard-sandbox.sb into nix, but leaving the other profiles in nixpkgs where they are.

[domenkozar]

@copumpkin is this still relevant?

[svend]

When I I set build-use-chroot = true, I still get the following error with nix 1.11.2, built from 437d3cdc7ac2f75d51b1c9980f6720e071d785e2, and nixpkgs built from NixOS/nixpkgs@77f8f35d57618c1ba456d968524f2fb2c3448295.

installing ‘screen-4.3.1’
building path(s) ‘/nix/store/s73v65lychncadgxv2l60lwk29gapjj8-user-environment’
killing process 1465
killing process 1465: Operation not permitted
error: while setting up the build environment: changing into ‘/private/var/folders/8k/bl8cjsgx2hn19vwkwg467s0h000_9s/T/nix-build-user-environment.drv-0’: No such file or directory

[domenkozar]

So it's still broken?

[svend]

It's still broken for me using the versions of nix and nixpkgs I listed above. Let me know if you need more information.

[benley]

With nix 1.11.2 and nixpkgs 16.09pre83147.df89584 I'm no longer able to reproduce this failure. Is this bug still relevant?

[svend]

I'm still getting the same error:

$ nix-env -i redis
installing ‘redis-3.0.7’
building path(s) ‘/nix/store/brw0k04pci4ra111w8m70g8r47flavx8-user-environment’
killing process 75913
killing process 75913: Operation not permitted
error: while setting up the build environment: changing into ‘/private/var/folders/8k/bl8cjsgx2hn19vwkwg467s0h000_9s/T/nix-build-user-environment.drv-0’: No such file or directory

• System: Mac OS X 10.11.5
• Nix version: nix-env (Nix) 1.11.2
• Nixpkgs version: "16.09pre86545.65ac26e"

/etc/nix/nix.conf:

# https://github.com/NixOS/nix/issues/759
build-use-chroot = true
build-use-sandbox = relaxed
darwin-log-sandbox-violations = true

[svend]

Same error with the master branch of nixpkgs:

nixpkgs $ git show | grep ^commit
commit 54a15fb2e4272ce80dc30b153b77fe8089c8a7f3
nixpkgs $ nix-env -f `pwd` -i hello
installing ‘hello-2.10’
building path(s) ‘/nix/store/h5xawhywbii6n1rpmgamnqbb1m5nl1mm-user-environment’
killing process 77419
killing process 77419: Operation not permitted
error: while setting up the build environment: changing into ‘/private/var/folders/8k/bl8cjsgx2hn19vwkwg467s0h000_9s/T/nix-build-user-environment.drv-0’: No such file or directory

Let me know what other information I can provide.

[benley]

I bet I screwed up and was testing with sandboxing disabled.

[fixbits]

Hi, i think mine is also related...

nix-env -i python2.7-mechanize-0.2.5
...
…
...
Finding dependencies for /usr/lib/libncurses.5.4.dylib...
killing process 4884
killing process 4884: Operation not permitted
error: while setting up the build environment: changing into ‘/private/var/folders/bv/fbm2bjqn4cd8mvyk4kfj1ph40000gp/T/nix-build-python2.7-clientform-0.2.10.drv-0’: No such file or directory

nix.conf:
gc-keep-outputs = true
build-use-chroot = true
binary-caches = https://cache.nixos.org https://hydra.nixos.org
use-binary-caches = true
build-use-sandbox = relaxed

[copumpkin]

@edolstra @domenkozar @shlevy can one of you assign me to this? I'm going to be taking a look at it soon (but it looks like nixpkgs staging is going to break everything on darwin so I'm looking at that first)

[shlevy]

@copumpkin Seems I can't assign people not associated with the repo somehow

[copumpkin]

Oh okay. Well, all y'all lining up to fix this, wait in line, cause I'm doing it.

[LnL7]

@copumpkin Anything I can do to help with this?

[yurrriq]

I've got a weird situation wherein nix-build gets a segfault, but nix-shell then make works fine... Any thoughts/updates?

[domenkozar]

This is definitely fixed in Nix master with darwin sandboxing.