Rkt: question: rkt fly mounts

[monder]

Environment

rkt Version: 1.18.0
appc Version: 0.8.8
Go Version: go1.7.3
Go OS/Arch: linux/amd64
Features: -TPM +SDJOURNAL
--
Linux 4.8.6-coreos x86_64
--
NAME=CoreOS
ID=coreos
VERSION=1235.0.0
VERSION_ID=1235.0.0
BUILD_ID=2016-11-17-0416
PRETTY_NAME="CoreOS 1235.0.0 (MoreOS)"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"
--
systemd 231
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS -ACL +XZ -LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD -IDN

What did you do?

With the new wrappers, etcd-wrapper in particular, it runs etcd with --stage1-from-dir=stage1-fly.aci (BTW, I dont understand why exactly fly is required here, shouldn't it work with regular stage1? But thats not the point.)
I've noticed something that wasn't expected. I use external volumes for etcd data and mount/unmount attach/detach between machines. For etcd I was unable to unmount these volumes after container exited. Turns out its not just etcd but with any stage1-fly container.

e.g.

$ mkdir /tmp/mount-example

$ sudo mount /dev/xvdh /tmp/mount-example

$ mount | grep xvdh
/dev/xvdh on /tmp/example-mount type ext4 (rw,relatime,seclabel,data=ordered)

$ sudo rkt run \
  --insecure-options=image \
  --volume v1,kind=host,source=/tmp/example-mount \
  --mount volume=v1,target=/opt/something \
  --stage1-from-dir=stage1-fly.aci \
  docker://busybox 

$ mount | grep xvdh
/dev/xvdh on /tmp/example-mount type ext4 (rw,relatime,seclabel,data=ordered)
/dev/xvdh on /var/lib/rkt/pods/run/c9e6f646-0f1e-4f9d-bf8e-c051573a78d6/stage1/rootfs/opt/stage2/busybox/rootfs/opt/something type ext4 (rw,relatime,seclabel,data=ordered)

$ sudo umount /tmp/mount-example

# The volume still cannot be detached because its in use
$ sudo mount | grep xvdh
/dev/xvdh on /var/lib/rkt/pods/run/c9e6f646-0f1e-4f9d-bf8e-c051573a78d6/stage1/rootfs/opt/stage2/busybox/rootfs/opt/something type ext4 (rw,relatime,seclabel,data=ordered)

# It's released only after rkt rm or gc in a few hours.

Do we really need to keep these mounts alive even after container exited?

[lucab]

Do we really need to keep these mounts alive even after container exited?

stage1-fly doesn't have any pod isolation, thus all operations are performed directly on the host. This includes mounts, which are performed in the host mount namespace. When the fly app terminates, the mount is kept alive by other processes in the same namespace (ie. all the other host processes). As there is no long running daemon, the proper way to clean up leftovers (including mounts) is via rm/gc as you stated.

If you are hitting some problematic behavior, it may be worthy tweaking the service unit or the wrapper to perform auto-cleanup on exit, similarly to https://github.com/euank/coreos-overlay/commit/50db4fa37683ccd3597d0d76af16599b3b91a53c. However, the generic gc timer should be already enough for normal usecases.

[monder]

Alright, thank you for clarification.

[lucab]

@monder you welcome. If you think that some wrappers may need adjustments (eg. they mess on restart) feel free to open tickets for them against coreos/bugs.