Buildah: rootless builds using buildah

[lukasheinrich]

we're looking to build images in an environment that restricts user's permissions (shared cluster at CERN) and are investigating building images using tools like skopeo / umoci / runc (i.e. a similar to what https://github.com/cyphar/orca-build is doing). From previous issues, (https://github.com/projectatomic/buildah/issues/287) it seems like rootless builds are not possible yet -- or has the situation changed in the meantime?

[rhatdan]

I believe you might be able to do things with usernamespace, but Buildah can not do it yet. I have no idea of umoci can do it yet with a normal workflow. This big issue is the inability to mount even in a User Namespace environment.

[rhatdan]

@cyphars can you build container images in a rootless environment using umoci?

[lukasheinrich]

Thanks for the quick reply. From @cyphar's work it seems like it is possible, but I wasn't sure what orca-build's long-term plans are -- being a hackweek project and all -- and buildah seemed like a natural home for such a feature if it's implementable.

[cyphar]

Yes, umoci has rootless support out-of-the-box (and has for quite a long time). It doesn't use user namespaces, so it works on effectively any kernel (though with the new capabilities v3 bits in recent kernels, it might be necessary to figure out whether we can make setcap work properly). LXC also now uses umoci for their OCI support because it works with --rootless as well.

@lukasheinrich As for orca-build, at the moment it is on my backlog to update it but my hands are quite full with a few other projects. I do plan to actually maintain it as a properly-supported project, but I only have so many hours to work on things. New maintainers are welcome. :smile_cat:

[rhatdan]

@lukasheinrich @cyphar I would love to see if we could merge some of the ideas together between the two.

[lukasheinrich]

unfortunately I have not much bandwidth to take on new responsibilities -- but I'm definitely happy to be involved in the discussions. I also noticed that some try to do "rootless build based on runc" also with buildkit (@AkihiroSuda -- https://github.com/moby/buildkit/issues/210) -- though I must admit, I'm lacking some basics on how these different approaches are different / fit together

[AkihiroSuda]

My WIP rootless runtime is here : https://github.com/AkihiroSuda/runrootless

[lukasheinrich]

Hi @AkihiroSuda -- my understanding was that runc already supports running rootless. What I thought was thinking about was the ability to build new images (e.g. from formats like Dockerfiles) in rootless mode -- which is what orca-build does)

[cyphar]

@lukasheinrich runc does already work with rootless, but @AkihiroSuda's project basically combines runc with PRoot. This works around the "apt-get doesn't work in rootless containers" problem that I mention in my rootless talks. I also have a similar project called remainroot which you could use with rootless containers. But PRoot is probably a safer bet at the moment (it's properly maintained AFAIK, and remainroot still has some odd bugs that stop it from working properly with apt-get).

@AkihiroSuda Have you considered making runrootless a tool that will modify an OCI bundle rather than a whole new wrapper script? In principle you would only need to touch the config.json since you also bind-mount PRoot into the container.

[AkihiroSuda]

@AkihiroSuda Have you considered making runrootless a tool that will modify an OCI bundle rather than a whole new wrapper script? In principle you would only need to touch the config.json since you also bind-mount PRoot into the container.

Yes, https://github.com/AkihiroSuda/runrootless/issues/1

[lukasheinrich]

@rhatdan just do get more understanding how you see the projects fitting together: are you mostly aiming for buildah to be an alternative syntax to Dockerfiles (that could possibly compile to LLB, and then be built with a (rootless) buildkit) or a OCI image building tool that's/should stay independent of buildkit?

[rhatdan]

My goal is that our tools support to formats for building container images. Dockerfile and straight Bash. I don't want to support other formats. I would not say buildah is an alternate syntax other then to say it is just a Bash script with a few primatives used to create the images. I would want higher level tools like ansible to wrap up the functionality of buildah to support different ways of specifying what goes into a container image, IE Different syntax's. Another tool that we are looking to incorporate is OpenShift S2I, where the user does not worry about creating a container image, he just checks into his code into a git repository and the tooling creates the image.

For the bulk of future container image developers are going to use packaged software to build container images. Therefore these image building tools have to be able to support yum install and apt get, Perhaps make install as well. Achieving this in a "rootless" environment is the ultimate goal. The main problem I see with this is how to handle creating/mounting /proc and /sys into containers, since packaged software often expects this software to exist, and UserNamespace does not support mounting these file systems.

[cyphar]

The main problem I see with this is how to handle creating/mounting /proc and /sys into containers, since packaged software often expects this software to exist, and UserNamespace does not support mounting these file systems.

I'm not sure what you mean -- rootless containers have both /proc and /sys (procfs is mountable inside a user namespace, while sysfs needs to be bind-mounted but that isn't a real issue for read-only usecases).

[rhatdan]

Actually I later found out that /proc and /sys can be mounted in a usernamespace. We are working on getting overlayfs to be "semi" supported also, although there are some security concerns on it.

[rhatdan]

We are getting closer to this.

[cgwalters]

FWIW, with ostree we ship rofiles-fuse which can be used unprivileged today - but only outside of a container, since FUSE today relies on a suid binary granting CAP_SYS_ADMIN.

[frezbo]

@rhatdan any links to the docs: https://github.com/projectatomic/buildah/issues/386#issuecomment-370968776

[rhatdan]

@frezbo you want docs on /proc and /sys being mountable, or the work on Overlay? The Overlay Is conversations with Kernel developers in Email. Mainly @ebiederman.
Googling found this article on /proc and /sys
https://lwn.net/Articles/652468/

[frezbo]

I wanted the former, thanks @rhatdan

[rhatdan]

We now have support for rootless builds. Closing

[maxking]

@rhatdan Any user documentation on how one would use buildah without root privileges in a user-namespaced environment?

[TomSweeneyRedHat]

@maxking are you looking for a tutorial kind of thing? We do have documentation in the man pages. We are planning to write a blog tutorial once everything is complete. At the moment you can create a container as a non-root user, just use Buildah commands as a non-root user. That's assuming the latest bits from GitHub. The current rpm doesn't yet contain them, but should in the very near future.

[rhatdan]

@maxking Only requirement would be that UserNS is setup correctly, which means the user executing the buildah commands has an entry in /etc/subuid and /etc/subgid.

On RHEL7/Centos7 their is a sysctl that has to be set to allow usernamespace to work.

On Fedora systems you should be able to just execute the normal buildah commands as a user.

[maxking]

@TomSweeneyRedHat @rhatdan Thanks! I just wanted to know if there is any extra steps required to use buildah as an unprivileged user, preferably inside a running container with user namespaces enabled.

[judavi]

@maxking Only requirement would be that UserNS is setup correctly, which means the user executing the buildah commands has an entry in /etc/subuid and /etc/subgid.

On RHEL7/Centos7 their is a sysctl that has to be set to allow usernamespace to work.

On Fedora systems you should be able to just execute the normal buildah commands as a user.

@rhatdan the UserNS setting that you mention is the same to this?
install.md

[rhatdan]

Correct

[jchristi]

Would definitely love to see a blog or tutorial on how to get this working in RHEL7/CentOS7.

[rhatdan]

Will be doing a Fedora Classroom tomorrow on Buildah.
https://fedoramagazine.org/fedora-classroom-building-container-images-with-buildah/