Nix: Support private files in the Nix store

[edolstra]

Sometimes it's desirable to have files in the Nix store that are not world-readable, such as configuration files containing passwords. This could be implemented as follows:

• Private files are marked as such by giving them a certain magic prefix to the name, e.g. p:. When these are registered, Nix should make them readable only to root (permission 0700) and the calling user (using an ACL). (The Nix daemon knows the uid of the caller.)
• The permissions on a derivation should be propagated to the output paths. Also, private derivations should be run with a umask of 0077.
• If another user attempts to register an already valid private file, then its uid should be added to the ACL of the file. If it's a derivation, then the uid should also be added to the ACLs of any valid outputs.
• Store operations that read files from the store (such as exportPath()) should check that the caller is in the ACL of the file.

So, for instance, if the root user builds a NixOS configuration containing some private files, then those files are only readable as root. If Alice builds the same configuration, then those files will be readable by root and Alice. If Alice then uses Charon to copy the configuration to a remote machine (i.e. nix-copy-closure --to root@remote ...), then the files on the remote machine will only be readable by root.

This model only allows files that are private to the instantiating user and root. So it doesn't support specifying a file that is readable only to the sshd user. But that's probably not necessary.

[shlevy]

A few questions:

• Does this potentially break nixos-rebuild build as non-root?
• Is there any way to tell as a user that I've attempted to register an already-valid file? For example, if root and I both use the same password file for a particular program, will I be able to tell when I add that file to the store that root has the same one?
• Would there be a reasonable way for a NixOS-managed service to know how to find per-user private files?

[edolstra]

• It shouldn't break non-root nixos-rebuilds since any private files would be owned by (or accessible to) the calling user.
• If a user attempts to register an already-valid file, he would be added to the ACL to the file. So he could conclude from this that root needed the same file.
• What kind of service, and why would it need to do that?

[viric]

What about the build users? Should they be in the ACLs?

[chaoflow]

Being able to tell that somebody else generated the same private file sounds like a no-go for me.

What do you think about including the user the file is private to, i.e. the one who created it, to be included in the hashing (for example an empty file with the user's name in the output)? So even if root, Alice and Charon all want a file with the same content, they would end up with 3 different files.

[shlevy]

The problem with including the user in the hashing is you may eventually share stores, but it's better than nothing. Maybe a wrapper function in nixpkgs that hashed in a GUID read from some personal file? So I have ~/.nixpkgs/guid and when I run pkgs.lib.privateFile 'mypass' "passw0rd" it creates a file whose first line is passw0rd and the second line is the guid?

[edolstra]

Why is this a problem? If the secret stored in the file is sufficiently secure (i.e. random), then this won't happen. I mean, if your login password is "foobar", then other people may be able to find out. So just don't do that.

[shlevy]

Eh, it's not really a problem, at least not for nix itself. The security-paranoid folk can just make sure all of their private files have some unique identifier included.

[mornfall]

Unless I am mistaken, private files in the store are sensitive to offline dictionary attacks, since you can learn the hash and you have a procedure to check your guesses. It is also conceivable, that a rainbow attack could be mounted against a particular version of nixpkgs. It would be advisable that any such private files contain significant salting, regardless of the "learn a secret by random collision" problem. A hypothetical lib.privateFile could then enforce that salting by taking a template string and substituting in the salt (which makes it possible to hide that salt in comments or other such convenient places).

[viric]

Yes, the sysadmin could care storing the salt somewhere only readable by the root user.

[domenkozar]

There is a use case for pkgs.bacula that bacula user needs access to a configuration file with passwords (so non-root).

[MarcWeber]

Alternative quick & dirty solution which may be useful for some use cases: https://github.com/MarcWeber/nix/blob/experimental/write-file-hashed/.topmsg

[domenkozar]

Better link to see @MarcWeber diff: https://github.com/MarcWeber/nix/compare/experimental;write-file-hashed

What's the current workaround for this if a config file includes a password?

[MarcWeber]

My version allows storing passwords in config, but instead of writing it to the store a path to a file containing it will be written to the store. The containing file can be read only by root. The complicated piece is that you have to write code which has to be run as root (eg when preparing config files for services) - but its very easy to review (eg in contrast to patching nix store allowing paths to be readable by some users only requires to think about many issues such as copying store paths - preserving privileges thereby etc). State is "I don't use my patch but it should be working".

[domenkozar]

What about a function with signature:

pkgs.lib.privateFile "my.conf" "user" "secret content";

And only user would have read permissions to that file? That solves the use cases I've been seeing on daily basis. Many services require to specify an user and thus we can pass that user to privateFile function.

No idea how that would work with nix-copy-closure, but we could just preserve the uid/gid.

[edolstra]

One problem with that is that user may not exist yet, since it might be created at activation time.

However, for the most common case (root), it would be fine. And if the service needs to be readable by a non-root service, you could always copy it to some private location in /run in the service's preStart script.

[domenkozar]

Now that our users require an uid, we could use that instead of the username. That would work then right?

[nbp]

@edolstra , with the test cases I made in #329, I am confident that the current proposal is secure enough to handle secret files for the owner of the nix store. When would you be able to review it?

This branch does not use ACL, it only remove the read/execute access for the group and others. Still, the implementation is made such as we can extend it with ACL later. I think it would be good to have a first iteration of NixOS with this branch before adding ACL.

[wmertens]

I just thought of another use-case of @MarcWeber's patch: setuid binaries.

As a reminder, https://github.com/MarcWeber/nix/compare/master...experimental/write-file-hashed shows how nix can write content with special permissions outside of the store. It's a very minimal patch compared to https://github.com/NixOS/nix/pull/329. (what it is missing is that the hash needs to include the permissions and ownership as well for security)

Using this, you can either sideload secret files, generated them from an expression or have them be copied on package install from a specific directory under /nix-support.

Files can be secrets or simply regular files that need special permissions, like setuid binaries.

@edolstra @nbp?

[domenkozar]

How can I help make this happen? NixOS is getting used more and more and this may lead to bad PR in the future since it's indeed a big issue (the biggest?).

[lethalman]

Personally I overcome this problem by providing files as strings instead of a nix store path. Yes that means the configuration is not deployable/reproducible, but perhaps I don't want my passwords to be deployable/reproducible?

[wmertens]

@lethalman can you give an example? I don't understand what you're doing when providing a private file as a string.

[lethalman]

That nix will not store the file in the nix store, instead will use the absolute path on your system. E.g. somepath = "/foo"; instead of somepath = /foo; .

[wmertens]

Oh very cool, I didn't realize that! That's basically the solution I
proposed above minus the hashing and permissions checking, but it works now.

On Wed, Mar 11, 2015 at 11:56 AM lethalman [email protected] wrote:

That nix will not store the file in the nix store, instead will use the
absolute path on your system. E.g. somepath = "/foo"; instead of somepath
= /foo; .

—
Reply to this email directly or view it on GitHub
https://github.com/NixOS/nix/issues/8#issuecomment-78242352.

[nbp]

Using double quote as a source of security is extremely weak, and it is easy to miss that when writing a configuration. I suggested to @edolstra that we should just add a different mode to the nix evaluation loop which prevents reading any file which is not already world readable.

Another solution, is to enforce that the system cannot be build by root but only as a user, and then make the system use this configuration. This way, the user which is building the configuration can only read the public files and if it attempts to load a private file, then the process will crash. We could add an extra option to allow root execution of nixos-rebuild.

Ideally we should be able to specify this with the type of the option. For example, we would have a scoped evaluation which prevent us from reading a private file, in a similar way as the addErrorContext builtin function. Thus a privateFile type will use merge function to do a strict evaluation of the merged values within a forbidPrivateFilesContext.

[chaoflow]

@nbp +1

[copumpkin]

This is probably too radical to really be practical, but I was just ruminating on a Nix security model based on "you don't get access to a store path unless you have a Nix expression that [produces a derivation that] hashes to that path".

In this model, your /nix/store would be inaccessible by default. To gain access to a store path, you would run your program through a "store broker" that would stick your process in a filesystem namespace that has access to the things you've provided (mechanism TBD) suitable Nix expressions for. nix-env and so on would handle this for you, and wrapper script generators would have to do something similar.

Then as long as whatever process generated my private files and put them into the store is itself kept private, the goal of this ticket is achieved. And if the process isn't private, then protecting the store output doesn't seem very useful anyway.

Any thoughts?

[proger]

As a data point, around a year ago I had a similar idea to @copumpkin's to use nix expressions that are evaluated in the "ephemeral" nix store without stdenv and subsequently stored in the big store in the encrypted form.

Implementing this only needs some language primitive that allows referencing paths from a different nix store.

See the gist at https://gist.github.com/proger/3563bc77184a12654fee

[nbp]

As this is mostly a NixOS issue, I am currently working on a prototype for NixOS which basically consist into building private files in a "trusted" nix store (ephemeral or not), and rely on one unique impurity to decode the files.

[spacekitteh]

Is there any progress on this?

[spacekitteh]

This is probably too radical to really be practical, but I was just ruminating on a Nix security model based on "you don't get access to a store path unless you have a Nix expression that [produces a derivation that] hashes to that path".

@copumpkin The most straightforward method would be with LXC I think.

[edolstra]

Actually the most straightforward way would be to remove read (but not execute) permission from the Nix store, i.e. rwx--x--x. That way, unless you already know a store path, you can't access it. However, this would also require that nothing inadvertently "leaks" store paths (e.g. via nix-store -qR /run/current-system).

[shlevy]

@edolstra Surely that will break a significant portion of builders (for dumb reasons, but still)

[edolstra]

I guess it would need to be rwx-wx--x to let nixbld add paths to the store. Or use a sandbox. However, I don't think it's a feasible option due to the path leakage risk.

[kevincox]

@edolstra I don't think removing r from the store is sufficent. For example on NixOS you know where the bootloader is which knows where stage one is...

Basically you are going to want to know where services are from a public place otherwise you can't do much with your system unless you boot it by typing the path to the kernel.

One not terrible solution would add a file in each derivation that contains the ACLs that should be applied to it when it is installed. I think we already have a .nix-something directory in each derivation that could be used for something like this.

[Azulinho]

I have been tracking this issue. Assuming the problem this is attempting to solve is about providing secrets to a service at runtime, wouldn't using something like keywhiz be a much better fit?

[kevincox]

I think the problem is exactly the opposite. On NixOS since everything is in the store and the store is public there is no safe place to store secrets. So currently if you don't want any process running on your machine to read something you have to get those secrets at runtime.

I think this thread is about a way to have secrets that "permanently" live on the server but are only accessible to the entities (probably users or groups) that you specify. For example I can reference my SSL key in my nginx config and it will "just work" without giving all users access to my private key.

[spacekitteh]

likewise, i don't want any process to even see what i have installed other than the process and it's direct requirements, like a chroot.

MACs are basically what is required, I think. SELinux is perfect for this.

[edolstra]

I implemented an encryptString builtin function a while ago to encrypt secrets at evaluation time: https://github.com/edolstra/nix/commit/6b7003672ff8d73d2a1f3b6709b216b3d0a7ad9c. The resulting encrypted strings can be decrypted at runtime (e.g. by the preStart script of a systemd service) by calling nix-store --decrypt (which decrypts the encrypted parts of a file read from stdin). Since the encryption happens at evaluation time, no keys end up in the Nix store.

Example usage for the generation/use of a wpa_supplicant.conf in NixOS: https://github.com/edolstra/nixpkgs/commit/4c8212069429bf9fb959e00ce8d9345ac7cb7ff0. Note that only the passphrases in wpa_supplicant.conf are encrypted, the rest of the file is not.

[wmertens]

On Wed, Nov 9, 2016 at 4:11 PM Eelco Dolstra [email protected]
wrote:

I implemented an encryptString builtin function a while ago to encrypt
secrets at evaluation time: edolstra@6b70036
https://github.com/edolstra/nix/commit/6b7003672ff8d73d2a1f3b6709b216b3d0a7ad9c.
The resulting encrypted strings can be decrypted at runtime (e.g. by the
preStart script of a systemd service) by calling nix-store --decrypt (which
decrypts the encrypted parts of a file read from stdin). Since the
encryption happens at evaluation time, no keys end up in the Nix store.

Finally something that seems like a good basis for a proper solution!
Bigger files could just have their contents encrypted with a key that is
encrypted by this.

I wonder if this also needs something like the os x keychain, so that you
can change the store key and re-encrypt the encrypted keys. Probably not,
since you could just regenerate the configuration with the new store key?

And to limit access to keys for daemon users, you could use sudo with
forced commands so they can only decrypt their own files.

[Ekleog]

This may be a stupid question, but is there any reason these commits are not pushed yet? Even if it's not perfect it still seems way better than what there is currently (except that we'll have to keep track of exactly which services are safe to give passwords to and which are not until all migrate), and the commits dating back to Feb. 2015 make me think it would be useful to have a starting point for other people to build up on it?

[kevincox]

FWIW I'm not totally convinced that this is the best solution.

Pros:

• Nix store is still just data.
• Access can be controlled by access to the keyfile.
• Punts on the access control problem.

Cons:

• Performance overhead (decrypted file is buffered in memory as well as the overhead of calling the decryption in the first place)
• More complex for apps to work with.
• Apps become more nix-specific (or you have to write a decryption-to-tmp wrapper which is spreading the security responsibilities around)
• Punts on the access control problem.

I don't know if there is a good solution. What @edolstra has implemented is a trade off between simplicity and effectiveness which is a pretty reasonable approach. However I think it is worth debating this more before shipping something.

I am still in favour of shipping an ACL with each store path. This has much higher complexity and makes store paths less portable across machines (what if that user doesn't exist) however that seems like a minor issue to me. Of course ACLs have other problems for example they either require changing the nix archive format (which might be a non-starter although it could be done) or standardizing a directory inside a NAR that will be read when it is "installed" and the permissions set up appropriately.

[wmertens]

On Tue, Dec 6, 2016 at 8:58 PM Kevin Cox [e-mail] wrote:

FWIW I'm not totally convinced that this is the best solution.

Cons:

• Performance overhead (decrypted file is buffered in memory as well
  as the overhead of calling the decryption in the first place)

That is a red herring, you will never notice this overhead. Very large
files should not be stored in this way to begin with.

>

• More complex for apps to work with.

This is not for apps, this is for NixOS configuration. The apps will never
notice anything, their wrappers will give them what they need.

>

• Apps become more nix-specific (or you have to write a
  decryption-to-tmp wrapper which is spreading the security responsibilities
  around)

Same argument, same response.

>

• Punts on the access control problem.

Do you mean authorization? There are already many excellent ways to handle
that, so the less nix does, the better, since it is then easier to
integrate.
Authentication is definitely not nix's responsibility.

>

I am still in favour of shipping an ACL with each store path. This has
much higher complexity and makes store paths less portable across machines
(what if that user doesn't exist) however that seems like a minor issue to
me.

Uhm, you complain about the tiny overhead of encryption on some files, and
then you propose adding a serious overhead on all accesses to the store? :)

Not to mention, how to implement these ACLs on non-NixOS…

>

[kevincox]

Maybe I would take it farther then most but I would make many of my store paths private to a specific user and I would like to protect everything in them by default. It's not critical, you are right, but I like to keep my processes as separate as possible.

Uhm, you complain about the tiny overhead of encryption on some files, and then you propose adding a serious overhead on all accesses to the store? :)

Not to mention, how to implement these ACLs on non-NixOS…

I think you understand what I mean by this. I am proposing that posix ACLs get set on the store paths when they get unpacked. Not some sort of proxy that does ACL checking. The overhead of posix ACLs is incredibly low. As an alternative if you are worried about this is that user and group permission control could be use, which is of course less flexible but almost zero overhead. (as this check is already being done)

[wmertens]

On Wed, Dec 7, 2016 at 11:47 AM Kevin Cox [e-mail] wrote:

I am proposing that posix ACLs get set on the store paths when they get
unpacked. Not some sort of proxy that does ACL checking. The overhead of
posix ACLs is incredibly low. As an alternative if you are worried about
this is that user and group permission control could be use, which is of
course less flexible but almost zero overhead. (as this check is already
being done)

Well, if you have a 500-user shared host, it probably is a bit problematic,
but indeed this is feasible. It does not work on single-user installs and
NFS-mounted stores though, and adds code to nix-store for managing the ACLs
(not sure how much).

With encryption, it works everywhere, all the time.

[domenkozar]

Bottom of line, someone should collect use cases (there are many already in nixpkgs) and see what method solves them or not.

[kevincox]

Well, if you have a 500-user shared host, it probably is a bit problematic,

I don't see how this is problematic. If you want to share with a lot of people you are going to have a long ACL (or add them to a group) but you would have the same problem with other methods. Somehow you have to give people access.

adds code to nix-store for managing the ACLs (not sure how much).

Actually setting the ACL is a couple of lines. The parsing would be more and there would need to be code to support it in nixpkgs.

Bottom of line, someone should collect use cases (there are many already in nixpkgs) and see what method solves them or not.

+:100:

[spacekitteh]

Honestly, both ACLs and encryption should be used.

[wmertens]

On Wed, Dec 7, 2016 at 12:14 PM Kevin Cox [e-mail] wrote:

Well, if you have a 500-user shared host, it probably is a bit problematic,

I don't see how this is problematic. If you want to share with a lot of
people you are going to have a long ACL (or add them to a group) but you
would have the same problem with other methods. Somehow you have to give
people access.

D'oh, forgot about groups, of course.

Bottom of line, someone should collect use cases (there are many already
in nixpkgs) and see what method solves them or not.

+💯

The only use cases I saw so far involved key files or passwords that needed
to be provided to daemons at runtime. For that, I would prefer having the
data encrypted at rest.

In any case, both approaches are orthogonal, so they can both be
implemented and even used simultaneously.

[abbradar]

FWIW there's another use case that I'd like to be supported. Currently we can enable SSH in initrd, which allows one to e.g. enter LUKS passwords remotely. However, for this private SSH server keys need to be written to initrd. We can read-protect /boot so that they cannot be read from there but they are still in /nix/store, globally accessible.

tl;dr: a way to protect all intermediate build products, and for bootloader installation scripts (i.e. some things ran as part of nixos-rebuild with root access rights) to have protected access to those products.

[Ericson2314]

I'm thinking a content-addressed store could cause similar breakage, so it might make sense to stage them together.

[CMCDragonkai]

Here's another usecase: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/prey.nix It requires API keys specified as text options in the service.

[offlinehacker]

Nice, i didn't know about encryptString

[domenkozar]

RFC for this issue is up: https://github.com/NixOS/rfcs/pull/5

[ghost]

I'm no expert, but it seems to me that the store needs to be broken up between a global store and several user stores. There must be tons of problems with this approach; what are they?
Edit. I dont think this is the right solution, which probably lays at the filesystem layer. Nix is a bit ahead of its time. A different approach to ownership/permission perhaps. I'v seen the mention of plan9 namespace. I'll delve into it at some point, hopefully.

[jgillich]

@edolstra That doesn't seem right?

[Azulinho]

Certainly a mistake, but probably worth closing this and follow up on the RFC ?

[edolstra]

Doh. This was a side-effect of merging the nix-repl repo.

[offlinehacker]

Ohh, and I was so happy for a moment :)

On Tue, 25 Apr 2017, 19:21 Eelco Dolstra, [e-mail] wrote:

Doh. This was a side-effect of merging the nix-repl repo.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nix/issues/8#issuecomment-297103184, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjvS8jWZ4PQQ0fIUnbZi_YOSM5IuWC2ks5rziuBgaJpZM4AA4a7
.

[ip1981]

Nix store is the code, secrets are the data. Let's do not make it more complicated :)
Code needs less protection if needs at all.

[catern]

Has anyone looked seriously at a model based on can-only-read-if-you-have-the-hash? As said above, that could be trivially implemented by removing read but not execute permissions from the store. This seems like the most principled way to do it, and would immediately open up a lot of interesting applications.

What would be required to fix the hash leakage? We would need to prevent there from being any globally-readable references to store objects which might transitively mention secrets. That doesn't seem absolutely impossible. It kind of parallels similar efforts to prevent address leakage from the Linux kernel, and other information leakage from trusted code to untrusted code.

Do any tools themselves leak paths? nix-daemon?

[Ekleog]

Trying to erase all leakage is just like trying to erase all security flaws in a program: it's doomed to failure, and is way better to add hardening (and erase all security flaws we encounter, also, of course).

For example of path leaks out of the top of my head, the grub command line leaks paths, the kernel, the initrd, /etc/systemd/services... basically the entire NixOS system has not been thought so as to erase all references to hashes. The only case I could think of for avoiding hash leaks is in the context of containers, where the entire filesystem is restricted to a specific host-independent configuration, which allows to be “sure” (as much as one can be) that no hash will leak. But that doesn't solve the broader problem of hash leaks.

Basically, all the nix and nixpkgs ecosystem would have to be fixed to account for this new “no hash leak” rule, so that isn't really possible in my opinion.

(Then, I wonder where the encryption implementation currently lies, I seem to remember it was (almost?) ready in nix1.12?)

[ip1981]

can-only-read-if-you-have-the-hash

It seems like the setuid bit. There is at least one problem: interpreters like Python or Bash (who owns a hash: a script or an interpreter?).

[catern]

Trying to erase all leakage is just like trying to erase all security flaws in a program: it's doomed to failure, and is way better to add hardening (and erase all security flaws we encounter, also, of course).

That is probably true. Even if the whole ecosystem was fixed to avoid leaking hashes, we would still need some kind of hardening to avoid problems resulting from hash leaks.

Is it possible to do such hardening? Maybe if there's some clever and elegant way to harden against path leaks, we can have a solution for secrets that is fundamentally based on paths-as-capabilities, plus hardening.

Maybe the desired property for such hardening would be that even if someone is able to get access to a hash, they won't necessarily be able to access the underlying data, because of additional checks which validate their access. Not sure if that's a correct formulation of the problem, or how to approach solving it in an elegant way.

Maybe we want revocability, in some sense? Lots of revocability techniques out there, maybe something could work without losing our existing advantages.

[copumpkin]

The way I was proposing a while ago was a "can-only-read-if-you-have-the-hash-preimage", but you'd likely need a fancy protocol for gaining access to the filesystem entry that way.

[7c6f434c]

I wanted to experiment with a non-world-enmuratable Nix store and see if there are any drawbacks/unexpected problems, but apparently the local store constructor enforces the 0o1775 mode (and I wanted to use mainline Nix). Is there a specific well-known reason for that?