Runc: The example `runc spec` file does not work with ubuntu `apt update`
Hello! I'm having some trouble getting the default OCI runtime file to perform some ordinary root-ish actions. Reading through the documentation, I'm not quite sure what to change.
I've identified a quick, basic example:
./runc-1.0.0-rc5 spec
mkdir rootfs
docker export $(docker create ubuntu) | tar -C rootfs -xf -
# Allow writes
sed -i 's/"readonly": true/"readonly": false/' config.json
# Change command to `apt update`
sed -i 's/"sh"/"apt", "update"/' config.json
./runc-1.0.0-rc5 run dummyId
I would expect this to have a relatively benign network error, since I have not removed the network namespace. Instead, I get a bunch of permission errors:
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (1: Operation not permitted)
E: seteuid 100 failed - seteuid (1: Operation not permitted)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
rm: cannot remove '/var/cache/apt/archives/partial/*.deb': Permission denied
Reading package lists... Done
W: chown to _apt:root of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chown to _apt:root of directory /var/lib/apt/lists/auxfiles failed - SetupAPTPartialDirectory (1: Operation not permitted)
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (1: Operation not permitted)
E: seteuid 100 failed - seteuid (1: Operation not permitted)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (1: Operation not permitted)
E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (1: Operation not permitted)
E: Method http has died unexpectedly!
E: Sub-process http returned an error code (112)
What do I need to change to allow setegid / etc to succeed?
Should that maybe be a flag on runc spec?
Any help would be appreciated :)
_(Obviously, I only used sed to make it trivial for someone else to try. I wouldn't use that normally.)_
dramaticostrich
All 14 comments
It looks like you're trying to run in rootless mode -- in other words you are trying to run runc as an unprivileged user. Either use sudo runc run ... or take a look at https://rootlesscontaine.rs/ to learn more about how to use rootless containers.
cyphar
on 9 Aug 2018
I am using a root shell by way of sudo bash, sorry that it wasn't clear:
$ whoami
root
Perhaps there is something wrong with my environment? I've been able to reproduce the instructions above with two different machines now, both of which are patched & up-to-date. This is using the rc5 release amd64 binary.
dramaticostrich
on 13 Aug 2018
Can anyone lend a hand? I've now been able to reproduce the steps above in a fresh vagrant:
This was using the ubuntu/artful64 vagrant box, inside a tempdir to avoid any filesystem mounting issues. Not sure what else I can try :/
dramaticostrich
on 16 Aug 2018
Can you attach a copy of your config.json?
cyphar
on 17 Aug 2018
Sure can:
https://gist.github.com/dramaticostrich/6c412423a6c912228aaaea5ddabe1205
Should be the same as anyone who runs runc spec with rc5 and adds the above sed commands.
dramaticostrich
on 17 Aug 2018
Were you able to resolve this?
I am facing multiple issues including this one when creating a runc container with the latest docker mariadb image. Unable to start the mariadb service either. Thought I'd create a debian9 slim container and directly install mariadb in it as a work around but again stuck with error you get above for apt.
I am running as root and have set read-only to false in config.json.
JanakiJoshi
on 28 Sep 2018
Were you able to resolve this?
I am facing multiple issues including this one when creating a runc container with the latest docker mariadb image. Unable to start the mariadb service either. Thought I'd create a debian9 slim container and directly install mariadb in it as a work around but again stuck with error you get above for apt.
I am running as root and have set read-only to false in config.json.
Resolved using the config below.
{
"ociVersion": "1.0.0",
"process": {
"terminal": true,
"user": {},
"args": [
"bash"
],
"env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"TERM=xterm"
],
"cwd": "/",
"capabilities": {
"bounding": [
"CAP_AUDIT_WRITE",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FSETID",
"CAP_FOWNER",
"CAP_MKNOD",
"CAP_NET_RAW",
"CAP_SETGID",
"CAP_SETUID",
"CAP_SETFCAP",
"CAP_SETPCAP",
"CAP_NET_BIND_SERVICE",
"CAP_SYS_CHROOT",
"CAP_KILL",
"CAP_AUDIT_WRITE"
],
"effective": [
"CAP_AUDIT_WRITE",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FSETID",
"CAP_FOWNER",
"CAP_MKNOD",
"CAP_NET_RAW",
"CAP_SETGID",
"CAP_SETUID",
"CAP_SETFCAP",
"CAP_SETPCAP",
"CAP_NET_BIND_SERVICE",
"CAP_SYS_CHROOT",
"CAP_KILL",
"CAP_AUDIT_WRITE"
],
"inheritable": [
"CAP_AUDIT_WRITE",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FSETID",
"CAP_FOWNER",
"CAP_MKNOD",
"CAP_NET_RAW",
"CAP_SETGID",
"CAP_SETUID",
"CAP_SETFCAP",
"CAP_SETPCAP",
"CAP_NET_BIND_SERVICE",
"CAP_SYS_CHROOT",
"CAP_KILL",
"CAP_AUDIT_WRITE"
],
"permitted": [
"CAP_AUDIT_WRITE",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FSETID",
"CAP_FOWNER",
"CAP_MKNOD",
"CAP_NET_RAW",
"CAP_SETGID",
"CAP_SETUID",
"CAP_SETFCAP",
"CAP_SETPCAP",
"CAP_NET_BIND_SERVICE",
"CAP_SYS_CHROOT",
"CAP_KILL",
"CAP_AUDIT_WRITE"
],
"ambient": [
"CAP_AUDIT_WRITE",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FSETID",
"CAP_FOWNER",
"CAP_MKNOD",
"CAP_NET_RAW",
"CAP_SETGID",
"CAP_SETUID",
"CAP_SETFCAP",
"CAP_SETPCAP",
"CAP_NET_BIND_SERVICE",
"CAP_SYS_CHROOT",
"CAP_KILL",
"CAP_AUDIT_WRITE"
]
},
"rlimits": [
{
"type": "RLIMIT_NOFILE",
"hard": 1024,
"soft": 1024
}
],
"noNewPrivileges": true
},
"root": {
"path": "rootfs",
"readonly": false
},
"hostname": "runc",
"mounts": [
{
"destination": "/proc",
"type": "proc",
"source": "proc"
},
{
"type": "bind",
"source": "/etc/resolv.conf",
"destination": "/etc/resolv.conf",
"options": [
"rbind",
"ro"
]
},
{
"destination": "/dev",
"type": "tmpfs",
"source": "tmpfs",
"options": [
"nosuid",
"strictatime",
"mode=755",
"size=65536k"
]
},
{
"destination": "/dev/pts",
"type": "devpts",
"source": "devpts",
"options": [
"nosuid",
"noexec",
"newinstance",
"ptmxmode=0666",
"mode=0620"
]
},
{
"destination": "/dev/shm",
"type": "tmpfs",
"source": "shm",
"options": [
"nosuid",
"noexec",
"nodev",
"mode=1777",
"size=65536k"
]
},
{
"destination": "/dev/mqueue",
"type": "mqueue",
"source": "mqueue",
"options": [
"nosuid",
"noexec",
"nodev"
]
},
{
"destination": "/sys",
"type": "sysfs",
"source": "sysfs",
"options": [
"nosuid",
"noexec",
"nodev"
]
},
{
"destination": "/sys/fs/cgroup",
"type": "cgroup",
"source": "cgroup",
"options": [
"nosuid",
"noexec",
"nodev",
"relatime"
]
}
],
"linux": {
"resources": {
"devices": [
{
"allow": false,
"access": "rwm"
}
]
},
"namespaces": [
{
"type": "pid"
},
{
"type": "ipc"
},
{
"type": "uts"
},
{
"type": "mount"
}
],
"maskedPaths": [
"/proc/kcore",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/sys/firmware",
"/proc/scsi"
],
"readonlyPaths": [
"/proc/asound",
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
}
}
Not sure if this is the best way of doing it.
JanakiJoshi
on 28 Sep 2018
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FSETID",
"CAP_FOWNER",
"CAP_SETGID",
"CAP_SETUID",
"CAP_SETFCAP",
These are the capabilities you were likely missing. I'm not sure when we changed the defaults, but we really need to add more capabilities to the default configuration. This is the default that umoci generates.
"bounding": [
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FSETID",
"CAP_FOWNER",
"CAP_MKNOD",
"CAP_NET_RAW",
"CAP_SETGID",
"CAP_SETUID",
"CAP_SETFCAP",
"CAP_SETPCAP",
"CAP_NET_BIND_SERVICE",
"CAP_SYS_CHROOT",
"CAP_KILL",
"CAP_AUDIT_WRITE"
],
Why do we default to CAP_AUDIT_WRITE? Do we really want containers writing audit messages?
rhatdan
on 29 Sep 2018
I just looked back into the git history and this has been the default for more than 2 years... This is really odd -- I've never noticed that runc spec was _that_ broken with regards to capabilities.
@rhatdan Yeah, CAP_AUDIT_WRITE doesn't make much sense to me (though the audit folks do have some odd things tied to CAP_AUDIT_WRITE). However it should be noted that Docker (and thus cri-o too probably) also set CAP_AUDIT_WRITE. umoci does it because that's the default that runtime-spec used, but I think we should probably change it.
I'm going to submit a patch to fix up these defaults.
cyphar
on 29 Sep 2018
We drop a lot of additional CAPS in CRI-O then Docker uses, since we don't default to doing builds.
CAP_AUDIT_WRITE, I believe was added way back so people could run tools like sshd inside of a container. Perhaps sshd did not handle the lack of this CAP well. But since this is not a common practice, I don't think is should be the default.
rhatdan
on 2 Oct 2018
Regardless, I think the runc spec command is just an example and in no way provides a secure or recommendation on that values go in your spec.
crosbymichael
on 3 Oct 2018
I just wanted to point out that the solution to this issue is not to drop rootless runc, like the config.json posted above, but rather to create _two_ uid and gid mappings. See #2517 for an example.
It is entirely possible to make seteuid and setegid etc work inside a rootless container.
haampie
on 12 Jul 2020
I've added these capabilities but still getting the same error.
"CAP_CHOWN", "CAP_DAC_OVERRIDE", "CAP_FSETID", "CAP_FOWNER", "CAP_SETGID", "CAP_SETUID", "CAP_SETFCAP",These are the capabilities you were likely missing. I'm not sure when we changed the defaults, but we really need to add more capabilities to the default configuration. This is the default that
umocigenerates."bounding": [ "CAP_CHOWN", "CAP_DAC_OVERRIDE", "CAP_FSETID", "CAP_FOWNER", "CAP_MKNOD", "CAP_NET_RAW", "CAP_SETGID", "CAP_SETUID", "CAP_SETFCAP", "CAP_SETPCAP", "CAP_NET_BIND_SERVICE", "CAP_SYS_CHROOT", "CAP_KILL", "CAP_AUDIT_WRITE" ],
ashwani29
on 12 Oct 2020
Related issues
mrunalp
路
7Comments
jessfraz
路
3Comments
AkihiroSuda
路
8Comments
imatespl
路
12Comments
lizhuqi
路
6Comments
Most helpful comment
Regardless, I think the
runc speccommand is just an example and in no way provides a secure or recommendation on that values go in your spec.