Nix: Install application with Nix from own binary cache?
I am working on an application called Studio that is installed with Nix and also extensively uses Nix at runtime. The Nix code representing the application is an overlay that adds our own relevant new packages to a pinned version of nixpkgs. These new packages are not upstream in nixpkgs and not built on NixOS Hydra.
I want to streamline the installation of this application and I especially want to download packages from a binary cache rather than building them locally. I have a Hydra instance building all the relevant packages already.
The problem is: How do I make nix-shell download binaries from the cache? It seems like in a standard nix installation only cache.nixos.org will be trusted and there will be no effect in asking nix-shell to use additional caches too?
The only way that I can think of is to ask the user to become root and edit the global nix.conf but that seems like more hassle than waiting on a build.
Tips would be welcome!
lukego
All 24 comments
Changing cache from which someone installs is an operation only allowed to trusted users. The point is that you may squat the hash of some common package that doesn't exist locally yet and install malware in there – other users might e.g. on update use that path...
vcunat
on 23 Jan 2018
I do understand that. I am hoping to find a workable solution anyway.
I have already asked users to install Nix themselves using curl https://nixos.org/nix/install | sh. So in some sense they must be "trusted." I suppose that script is quietly using sudo behind the scenes? I could do the same I suppose.
lukego
on 23 Jan 2018
Just an idea: Could I make an additional installation of Nix with its own store and configuration? i.e. to embed Nix as a component of the application? This could potentially be a way to avoid conflicts with other applications and the need to do things as root.
lukego
on 23 Jan 2018
Yes, you may use single-user installation – the store is then owned by that single user. Of course, changing location from /nix/store makes you rebuild everything; some have been working around that by making binaries _think_ they're on /nix/store without actually being there – via proot.
vcunat
on 23 Jan 2018
Just now I'm thinking the simplest compromise might be to make a custom script bin/install-nix-for-studio that downloads and installs Nix in the usual way and then also updates the global configuration to make our binary cache trusted.
Could also consider a more extreme approach like wrapping everything inside a Docker container. This would also solve some issues I have had with nix-shell not providing much isolation e.g. when naughty applications go looking for things at hard-coded locations. However, Docker is a pretty heavy dependency...
lukego
on 23 Jan 2018
Doesn't really solve your problem, but FWIW you could fetch fixed-output paths from an "untrusted" source... similar to behavior of a binary cache. Only more difficult to maintain, less automagic, and decidedly less elegant. Useful for bootstrapping things, though, and might help get people started using tagged releases or something?
Hmmm... goes off and thinks
dtzWill
on 24 Jan 2018
In Nix 1.12, content-addressable paths (such as fixed-output paths) don't need a signature, so they can be substituted from any (untrusted) binary cache. The eventual goal is to have a command to rewrite arbitrary closures into content-addressable form. The only thing lacking at the moment is that content-addressable paths cannot have self-references.
edolstra
on 24 Jan 2018
Also in Nix 1.12, on Linux, you can change the physical location of the Nix store:
$ nix run --store /home/eelco/my-nix nixpkgs.hello -c hello
[2 copied (20.0 MiB), 5.3 MiB DL]
Hello, world!
This uses a mount namespace to bind-mount /home/eelco/my-nix/nix/store on /nix/store, so it can still use substitutes from cache.nixos.org.
edolstra
on 24 Jan 2018
That's amazing!
(Is it linux-only because other platforms (such as OSX?) don't have an appropriate equivalent for bind mounts?)
@lukego so you might want to try something like this if you're okay with using 1.12:
$ nix run --store $HOME/studio-nix --option binary-caches "https://hydra.snabb.co https://cache.nixos.org" --option require-sigs false -f . studio-gui -c studio-x11
Which seems to mostly work (using studio commit 106100c04bcc6ba9494681a957588c84ceebfc7d)!
Looks like "coreutils" should be added as a dependency too, or as a temporary workaround:
$ nix run --store $HOME/studio-nix --option binary-caches "https://hydra.snabb.co https://cache.nixos.org" --option require-sigs false -f . studio-gui coreutils -c studio-x11
This sounds pretty awesome!
Currently I ask people to install Nix with curl https://nixos.org/nix/install | sh. How should I revise that for installing Nix 1.12?
lukego
on 24 Jan 2018
@lukego just install nixUnstable package in existing nix and you have nix 1.12 available.
disassembler
on 25 Jan 2018
@disassembler I am starting the application using nix-shell. I wonder how I would apply nixUnstable here? Perhaps I would run an "outer" nix-shell that installs nixUnstable and then calls into an "inner" nix-shell that starts the application using Nix 1.12? Or is there a simpler way?
lukego
on 26 Jan 2018
How can I bootstrap this?
Currently I am trying to use nix to install nixUnstable but I don't see that this will reliably give me the right version. Just in testing now it seems like I do get a prerelease of nix 1.12 but it doesn't seem to include the nix run command:
$ nix-shell -p nixUnstable --run "nix run --store /home/eelco/my-nix nixpkgs.hello -c hello"
error: ‘run’ is not a recognised command
Try ‘nix --help’ for more information.
$ nix-shell -p nixUnstable --run "nix --version"
nix (Nix) 1.12pre4523_3b81b26
Is there something that I can write instead of nixUnstable there to "pin" this to a suitable prerelease version of Nix 1.12 that will be cached on Hydra?
lukego
on 1 Feb 2018
3b81b26 is almost 2 years old! Try a newer Nixpkgs/NixOS channel.
edolstra
on 1 Feb 2018
@edolstra I have a bootstrapping problem. I am not controlling the version of nix/nixpkgs/nixos that's installed on the host: it could be old, bad, random. I'm hoping that I can still somehow use it to access a known-good version of Nix 1.12 to run my application with. Is this possible? Intuitively it seems like it should be straightforward to ask nix-shell to fetch a specific build of nix-1.12 from Hydra and run that?
lukego
on 1 Feb 2018
@lukego Try nix-shell -I nixpkgs=https://nixos.org/channels/nixos-17.09/nixexprs.tar.xz ... (or in a newer version of Nix, nix-shell -I nixpkgs=channel:nixos-17.09 ...).
edolstra
on 1 Feb 2018
Thanks @edolstra. I am able to install a newer version of nixUnstable like this. I am having an error when I run the --store option though:
error: don't know how to open Nix store '/home/luke/nix-store'
Could it be related to the nix daemon being an older version? (I suppose that when I am running with a private nix store I might want to run daemonless somehow?)
lukego
on 13 Feb 2018
Yeah, the Nix daemon version shouldn't matter. The error message suggests that you still have an older version though. What does nix-env --version say?
edolstra
on 13 Feb 2018
[luke@interlaken:~/git/studio-installer]$ nix-shell -I nixpkgs=https://nixos.org/channels/nixos-17.09/nixexprs.tar.xz -p nixUnstable
[nix-shell:~/git/studio-installer]$ nix-shell --version
nix-shell (Nix) 1.12pre5873_b76e282d
I have already defined an overlay over nixpkgs. I suppose that I could use this to use a specific version of nix. Just have to be careful and pick one that is in the standard binary cache so that it doesn't have to build locally. Any recommendation?
lukego
on 13 Feb 2018
Since I started running nixUnstable with nix-shell I have also started seeing error messages like this:
error: imported archive of ‘/nix/store/y2fq4kdnzrhy0jj2layg3ngpf0h5ls8f-source’ lacks a signature
Your original question is the reason why we tell users to install via a channel, because when you do nix-channel --add ..., Nix will add the channel's specified binary cache.
mbrock
on 8 Mar 2018
Eh, what I said isn't very useful because (as I just learned) the binary cache's public key is not added as trusted when you add a channel, so users still have to edit a configuration file.
mbrock
on 9 Mar 2018
I believe this is partially addressed by https://cachix.org and nix-shell part still needs investigation at https://github.com/cachix/cachix/issues/52
domenkozar
on 13 Jun 2018
Related issues
edolstra
·
49Comments
fzakaria
·
39Comments
edolstra
·
96Comments
taktoa
·
35Comments
Profpatsch
·
73Comments
Most helpful comment
Also in Nix 1.12, on Linux, you can change the physical location of the Nix store:
This uses a mount namespace to bind-mount /home/eelco/my-nix/nix/store on /nix/store, so it can still use substitutes from cache.nixos.org.