Runc: SELinux and UserNamespace don't work together

[rhatdan]

The current implementation of UserNamespace breaks with SELinux enabled.

On Fedora 25:

docker run -ti alpine sh
/usr/bin/docker-current: Error response from daemon: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\"rootfs_linux.go:54: mounting \\\\"tmpfs\\\\" to rootfs \\\\"/var/lib/docker/100000.100000/overlay/16d494d8bef4421aa530c899e4f847b707f4dafd5d677e33b6e2cc2a5abb7086/merged\\\\" at \\\\"/dev\\\\" caused \\\\"permission denied\\\\"\\"\"\n".

Looks like runc is attempting to mount tmpfs on /dev, with the context="container_file_t" and the kernel blocks this. The kernel rightly blocks a confined process by user namespace, from setting arbitrary labels on a tmpfs file system.

Why can't the tmpfs file systems be mounted outside of the user namespace?

[rhatdan]

This bug was reported here

https://bugzilla.redhat.com/show_bug.cgi?id=1401537

[rhatdan]

stracing runc shows it getting an EACCESS on the mount of tmpfs at /dev with selinux context

[cyphar]

Why can't the tmpfs file systems be mounted outside of the user namespace?

Because we do all of the mounts after we've joined the namespaces. And some mounts won't work until you're in a user namespace (and you can't make the mount namespace first because the "owner" of the namespace is pinned).

[rhatdan]

I did not think most mounts can be done inside of a user namespace. I see this as an unfixable problem if the mounts have to happen inside of the user namespace.

[cyphar]

@rhatdan FS_USERNS_MOUNT specifies that certain filesystems (such as proc, sysfs, devpts, ramfs, cgroupfs) are "user namespace aware" and allow you to mount those filesystems inside a user namespace.

So there are _some_ filesystems where we can't do the mounting in another order.

Is there another way for us to tmpfs mounts that respect the SELinux way of doing things? It'd be a real shame if you can't effectively combine user namespaces and SELinux.

[rhatdan]

We could attempt to set the label after the mount, but I think this will be blocked also.

label.SetFileLabel(mountpath, mountLabel)

If you are running with a user namespace and SELinux disabled inside of the container, you could attempt to do a chcon -t container_file_t /dev To see if this is allowed.

[rhatdan]

Stephen Smalley (SELinux Kernel Maintainer points out in the bugzilla:

"Is this related at all to Linux kernel commit aad82892af261b9903cc11c55be3ecf5f0b0b4f8 ("selinux: Add support for unprivileged mounts from user namespaces") circa Linux 4.8? If so, that change does cause SELinux to reject context mounts on user namespace mounts, to prevent unprivileged processes from performing context mounts."

[mrunalp]

One option that could be used is a privileged helper like a pre-start hook but it won't work for unprivileged (rootless) containers unless it is a setuid helper in the same vein as newuidmap/newgidmap utilities.

[rhatdan]

I believe this is a kernel Bug, I have just opened a request to the SELinux mailing list to have

http://kernel.suse.com/cgit/kernel/commit/?id=aad82892af261b9903cc11c55be3ecf5f0b0b4f8

Reverted.

This kernel patch is basically blocking the setting of labels using context= switch if you are in a User Namespace. I don't believe DAC checks should be effecting MAC Operations. The SELinux Maintainer, seems to agree with me. So hopefully we will get a patch into the Kernel fairly quickly.

[euank]

@rhatdan Do you have an update on this? Presumably the thread you created is https://marc.info/?l=selinux&m=148105484111300&w=4

Reading that, I don't see whether any patch was actually created as a followup, and if you have any additional references or info that'd be helpful.

Edit: dug a little more and I think the patch resulting is https://patchwork.kernel.org/patch/9466451/

[rhatdan]

Yes this patch will go into 4.11, We are using and running it on Fedora right now.

[karthik101]

Is this issue fixed in kernel 4.14 ?

[rhatdan]

It should be, although I have not checked.

[karthik101]

This Issue is still Present,
my docker version : 17.06.1-ce,
Kernel version: 4.14.0
docker logs:
Nov 20 06:53:34 workstation dockerd: time="2017-11-20T06:53:34.470743494-05:00" level=error msg="Handler for POST /v1.30/containers/23eb74bff19085238ebbc30b4852dd37016e6341e9f6781eb77a308d99fb8603/start returned error: oci runtime error: container_linux.go:262: starting container process caused \"process_linux.go:247: running exec setns process for init caused \\\"exit status 34\\\"\"\n"

Nov 20 06:53:40 workstation kernel: audit: type=1400 audit(1511178820.136:155): avc: denied { dac_read_search } for pid=2277 comm="master" capability=2 scontext=system_u:system_r:postfix_master_t:s0 tcontext=system_u:system_r:postfix_master_t:s0 tclass=capability permissive=0

Nov 20 06:53:50 workstation kernel: audit: type=1400 audit(1511178830.140:156): avc: denied { dac_read_search } for pid=2277 comm="master" capability=2 scontext=system_u:system_r:postfix_master_t:s0 tcontext=system_u:system_r:postfix_master_t:s0 tclass=capability permissive=0

Nov 20 06:54:50 workstation kernel: audit: type=1400 audit(1511178890.200:157): avc: denied { dac_read_search } for pid=2277 comm="master" capability=2 scontext=system_u:system_r:postfix_master_t:s0 tcontext=system_u:system_r:postfix_master_t:s0 tclass=capability permissive=0

Any workarounds you guys did? I cant just disable selinux for this.

[rhatdan]

The AVC yhou are showing is postfix_master_t, which is not a container label. Are you running postfix in a container?

[karthik101]

Its actually a base centos container. I removed --selinux-enabled in the daemon, but selinux is enabled in the host, namespace work after i disable both.

[rhatdan]

If you are running a runc container on top of a labeled directory this transition could happen I suppose. If you did not tell runc to use SELinux. Then runc might still transition to the postfix_master domain.

[mcallaway]

Has there been any progress on this?

[root@localhost ~]# getenforce
Enforcing
[root@localhost ~]# docker --version
Docker version 17.12.0-ce, build c97c6d6
[root@localhost ~]# uname -r
4.15.1-1.el7.elrepo.x86_64
[root@localhost ~]# docker info | grep -A3 Security
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Security Options:
 seccomp
  Profile: default
 selinux
[root@localhost ~]# docker run hello-world
docker: Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "process_linux.go:301: running exec setns process for init caused \"exit status 40\"": unknown.

[euank]

@mcallaway I believe that's a different issue than this one. Although the title of this one is broad, it was specifically about a /dev mount failure. The audit denial logged is also helpful for figuring out these issues; please include that as well.
It may be worth trying to reproduce it with runc master and without docker if you want to open a new issue here, or perhaps it could make sense to report it against docker/for-linux.

@rhatdan Based on your comment that this is a kernel issue (https://github.com/opencontainers/runc/issues/1215#issuecomment-265261420), should we close this issue on the runc side? It seems like what you reported has been resolved kernel-side and there's no pending work on the runc side.

[rhatdan]

Yes lets close this. We are just about to start working on user namespace support in CRI-O and friends, so we will probably find any issues dealing with SELinux and UserNamespace.