Kind: Cannot start with podman backend due to AppArmor errors

[jbarrick-mesosphere]

What happened:

I cannot start kind using the podman provider, due to an error indicating AppArmor is not enabled:

Error: Apparmor profile "unconfined" specified, but Apparmor is not enabled on this system

What you expected to happen:

The kind cluster to start.

How to reproduce it (as minimally and precisely as possible):

➜  ~ sudo aa-status
apparmor module is loaded.
48 profiles are loaded.
48 profiles are in enforce mode.
   /usr/lib/apache2/mpm-prefork/apache2
   /usr/lib/apache2/mpm-prefork/apache2//DEFAULT_URI
   /usr/lib/apache2/mpm-prefork/apache2//HANDLING_UNTRUSTED_INPUT
   /usr/lib/apache2/mpm-prefork/apache2//phpsysinfo
   /usr/lib/dovecot/anvil
   /usr/lib/dovecot/auth
   /usr/lib/dovecot/config
   /usr/lib/dovecot/deliver
   /usr/lib/dovecot/dict
   /usr/lib/dovecot/dovecot-auth
   /usr/lib/dovecot/dovecot-lda
   /usr/lib/dovecot/dovecot-lda//sendmail
   /usr/lib/dovecot/imap
   /usr/lib/dovecot/imap-login
   /usr/lib/dovecot/lmtp
   /usr/lib/dovecot/log
   /usr/lib/dovecot/managesieve
   /usr/lib/dovecot/managesieve-login
   /usr/lib/dovecot/pop3
   /usr/lib/dovecot/pop3-login
   /usr/lib/dovecot/ssl-params
   /usr/lib/dovecot/stats
   /usr/sbin/dnsmasq
   /usr/sbin/dnsmasq//libvirt_leaseshelper
   apache2
   apache2//DEFAULT_URI
   apache2//HANDLING_UNTRUSTED_INPUT
   apache2//phpsysinfo
   avahi-daemon
   docker-default
   dovecot
   identd
   klogd
   lsb_release
   mdnsd
   nmbd
   nscd
   ntpd
   nvidia_modprobe
   nvidia_modprobe//kmod
   ping
   smbd
   smbldap-useradd
   smbldap-useradd///etc/init.d/nscd
   syslog-ng
   syslogd
   traceroute
   winbindd
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
➜  ~ sudo KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster --name newcluster
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Creating cluster "newcluster" ...
 ✓ Ensuring node image (kindest/node:v1.18.2) 🖼
 ✗ Preparing nodes 📦 
ERROR: failed to create cluster: podman run error: command "podman run --hostname newcluster-control-plane --name newcluster-control-plane --label io.x-k8s.kind.role=control-plane --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run --volume /var --volume /lib/modules:/lib/modules:ro --detach --tty --label io.x-k8s.kind.cluster=newcluster --publish=127.0.0.1:40923:6443/TCP kindest/node@sha256:7b27a6d0f2517ff88ba444025beae41491b016bc6af573ba467b70c5e8e0d85f" failed with error: exit status 125
Command Output: WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
Error: Apparmor profile "unconfined" specified, but Apparmor is not enabled on this system
➜  ~ 

Environment:

• kind version: (use kind version): v0.8.1
• Kubernetes version: (use kubectl version): N/A
• Docker version: (use docker info): N/A
• OS (e.g. from /etc/os-release): Arch Linux
• podman version: 2.0.3

[BenTheElder]

podman 2.X was worked around in HEAD, recently, can you try that?

[BenTheElder]

https://github.com/kubernetes-sigs/kind/pull/1728

[mlouage]

So for Podman 2.x I cannot use a kind download from the releases? I downloaded the latest release (0.8.1) and I also have this issue.

Fedora 32 with Podman 2.0.4 (rootless). I have no App Armor installed, I believe Fedora has SELinux which should be enabled on my system.

[BenTheElder]

You can't, podman broke handling of the app armor flag relative to docker. It should be worked around if you install from the latest kind sources, but I can't confirm that myself.

[BenTheElder]

Podman 2.X has multiple breaking changes kind is working around in https://github.com/kubernetes-sigs/kind/pull/1728

Fedora 32 with Podman 2.0.4 (rootless). I have no App Armor installed, I believe Fedora has SELinux which should be enabled on my system.

Rootless podman does not work and has never worked. You will need to use it rootfully to run Kubernetes for now.

Rootless Kubernetes does not exist upstream yet, but even when it does rootless podman has it's own issues. See discussion in #1727.

[mlouage]

Ok, thanks. I'll try to build it from source. Not too familiar with Go but I'll give it a try!

[BenTheElder]

you can build from source with make build from a clone without installing go, (it will use docker/podman), the binary will be in bin/kind

[BenTheElder]

v0.9.0 is slated for tomorrow tentatively, this should be fixed already.
cgroupsv2 is still pending some other work, mostly not in the kind repo.