habitat shouldn't require sudo

[szymonpk]

Is there any particular (and well motivated) reason why habitat should be running using sudo? It requires access to /hab/pkgs/, can user change path for pkgs? or keep it somewhere in home while uid!=0?

Applications shouldn't require root to start (for example hab start core/redis). While I can interact with docker without beeing root, hab tries to force this on me.

[mickfeech]

I believe it creates chroots which is why escalated privileges are required.

[szymonpk]

So it isn't only pkg path issue, it's too bad that I've to use sudo for 'app' development. Can someone from devs confirm this? I think it also would be good to put message in doc, I might not be only person wondering.

Thanks @mickfeech,

[adamhjk]

Habitat has two pieces that require root. The studio requires it, as @mickfeech points out, because we create chroot environments and use bind mounts. We think there is an alternative path using different tooling to create the chroot (https://github.com/proot-me/PRoot/blob/master/doc/proot/manual.txt) that would not need root, but we haven't tested it yet. Docker avoids the impression of needing root because it routes everything through the Docker daemon - which _is_ running as root.

The supervisor is a package manager and a process supervisor. In both cases, it helps to be root - the first to get the packages installed in the first place (either its root, or you chown the filesystem to a particular user, a-la homebrew), and the second to run services itself. Habitat drops privileges for the supervised process to be run as the hab user by default, and sets permissions on the svc directory to make sure it can only write to approved locations.

That's the situation as it stands. Happy to talk about what the right combination of behavior would be to allow safe package building and safe service supervision without elevated privileges - but I suspect it would involve a bunch more manual setup work, at least for supervision.

[szymonpk]

Thanks Adam for detailed explanation. Now I can understand why running hab might require root (supervisor), app building/development shouldn't at some point for sure.

Maybe docker model (with dockerd) isn't such bad idea?, because proot solves only half of your issues. And it isn't about trust (well, maybe a little bit, but! chef lover from early 0.10 here! and it's running as root), rather about separation and clean dev environment. Using sudo at every step is also a little disturbing and inconvenient and I don't even have sudo on my dev workstation because I don't need it.

For know I'll probably go for dedicated VM, running in background, for sake of clean workspace.

[lilianmoraru]

@adamhjk I don't know if it is directly applicable but vagga(unshare) does this without needing escalated privileges.
But it uses uidmap. I'll ping @tailhook in any case.

[lilianmoraru]

Flatpak uses Bubblewrap/user namespaces, in order to sandbox unprivileged.

[adamhjk]

I think we should accept investigating this further, especially for the studio use case. I'm not convinced it's better to avoid the supervisor wanting to start as root, given that in most cases what we do is drop privileges, and we're also a package manager.

[juliandunn]

Alright, so let's talk about the experience of the studio use case itself, because we're already making a few changes to the sup permissions model under #755.

I agree that the current situation of having to sudo everything isn't ideal. I see two independent options for the UX on Linux:

1. You can run any hab command as yourself; but only hab pkg build and hab studio enter require sudo.
2. You don't need sudo at all even for hab pkg build or hab studio enter.

What are people's expectations? Can you react with ❤️ if you are ok with option 1, or 🎉 if you would rather option 2?

[miketheman]

To an earlier comment, starting services _may_ require root/elevated privs, particularly in the case of opening ports under 1024 (so if you're running a web server on ports 80/443, going to need some extra privs).
As I have yet to actually experience hab on Linux, I cannot say definitively what I expect to happen, but it definitely feels like entering a studio to experiment and build packages should not require escalation of privilege.

[fujin]

@miketheman I don't mean to "well actually" but you can set capability on the binary to bind to a port <1024 as non-root. I typically do this during omnibus-pipeline style builds on binaries before they are finally packaged up.

[miketheman]

@fujin Certainly true. But not a default deployment method for many existing things, so violates the principles of convention and simplicity, in my mind.

[fnichol]

I'll chime in here with some extra context.

The option 1 above is the most doable in the short term--the key is adding a bit of sudo knowledge to the key-related subcommands (i.e. making an additional check for SUDO_USER to determine the key cache path). It's one of those things that doesn't become totally clear without more users using the software :)

Using Docker as a bypass for running a Studio under root isn't the ultimate solution and quadrupally so if you are using a Linux workstation and running the Docker engine locally. A combination of capabilities are currently required for Studio including chroot, bind mounting, mounting a Docker socket if present, and dev/tmpfs mounting. The only workable solution I've found so far running in a Docker container is with the --privileged flag which essentially gives the container root access to your entire system anyway. I've taken 2 quick passes at trying to determine the correct system capabilities to pass to the container and still haven't had success. That isn't to say it may not be possible, but I'd love some help here--it would help every Mac workstation user which is great!

As mentioned above we might have an alternate solution in proot and/or fakeroot-ng, both of which exploit ptrace(2) in the Linux kernel. I started some exploratory work there the week after launch and would like to spend a bit more time seeing what is possible.

Finally, remember that Habitat is _not_ Chef (i.e. configuration management) meaning that there may be edges with respect the Supervisor's responsibility. For example, we're closing out a feature for the Supervisor where it can determine if a non-root user is required for a service and check to see if that user exists. If it doesn't, the Supervisor would exit, reporting its missing requirement. Now, should the Supervisor attempt to create this user? Perhaps. Should that be a platform-portable strategy? I don't know. We're still working through how far into a running system Habitat should venture (i.e. user management, etc.). Given these challenges, it becomes extra complicated to determine which non-root user should launch a Supervisor and which set of security constraints are in effect (i.e SELinux, etc.).

Food for thought, nice discussion!

[tailhook]

A combination of capabilities are currently required for Studio including chroot, bind mounting, mounting a Docker socket if present, and dev/tmpfs mounting. The only workable solution I've found so far running in a Docker container is with the --privileged flag which essentially gives the container root access to your entire system anyway.

As already pointed out you can achieve all of that with vagga. Or you can reimplement it using unshare crate that vagga uses (ping me if you need any help). This requires user_namespaces to be enabled. They are enabled in ubuntu at least starting with 14.04, and in previous versions, if you update kernel version. Some distributions like Arch Linux don't enable CONFIG_USER_NS in their kernel config, so it may not universally be available, but I think it's worth trying.

But the thing is:

mounting a Docker socket if present

If you mount docker socket, you can ignore other security issues because having access to the docker socket is equivalent to having passwordless sudo.

[tailhook]

Ah, by the way, you can use old good unshare binary. Which is in ubuntu's util-linux since xenial, to try it without installing anything, like this:

$ unshare -r chroot /path/to/chroot /bin/bash
root@hostname:/#

Just make sure that /path/to/chroot contains /bin/bash (your new shell user is not real root, just a root in a namespace).

(And yes this doesn't help with ports < 1024 if you ask)

[fnichol]

@tailhook Thanks for the context and for vagga. I ran into the project multiple times criss-crossing Rust crate issues and PRs--it looks solid!

[skalsi-atlassian]

When running hab pkg build, does the package actually build as root? If so, aren't you trusting all the build tools and so on to run as root?

[smith]

@skalsi-atlassian hab pkg build creates a chroot, as does hab studio enter, which requires root to create. In this chroot, all the build tools are running as root, but it's in the chroot which keeps it separate from the rest of the system.

[skalsi-atlassian]

IIUC if you are root and inside a chroot, it is trivial to exit the chroot. https://filippo.io/escaping-a-chroot-jail-slash-1/

I don't believe it is the intent of the chroot to offer a "sandbox" when the user is root.
EDIT: More here: https://lwn.net/Articles/252794/

[juliandunn]

I am going to close this issue for now. On Habitat 0.11.x and later, the following behaviors are implemented:

• hab on Linux will ask you for a sudo password transparently if it needs to perform a privileged operation. For example, creating a studio on Linux is, by definition, a privileged activity since it needs root to bind mount appropriate sockets, etc. into the chroot.
• On Windows and Mac since the studio environment is actually a Docker container, there is no need for sudo and thus Habitat won't ask you for this.
• All other operations do not require root on any platform (e.g. setup, key generation, etc.).

I believe this closes out the current issue.

[tailhook]

For example, creating a studio on Linux is, by definition, a privileged activity since it needs root to bind mount appropriate sockets, etc. into the chroot.

While I'm not using habitat, so don't object to closing the issue :) The explanation ignores this discussion at all. I've demonstrated how you can chroot as unprivileged user here and bind mounts work after unsharing user namespace too.

Sorry, if that sounds offensive. I just want to increase awareness that unprivileged chroots often work and should be used if possible.

[fnichol]

@tailhook You're right that our recent features that @juliandunn alluded to don't change the underlying requirement for root permissions. There were a few UX issues we addressed first that deal with the location of signing keys, etc. when running as root, a non-root user, or a non-root user using sudo. For the moment we aren't directly using unshare or ptrace(3) but we may very well adopt that tack in the medium to longer term (we still have similar questions to answer on *BSD, macOS, and possibly Solaris, all of which could work with a massive hammer such as chroot). You're points are valid to me, I'm looking forward to spending some more time shaving this one down a bit. Thanks!