Moby: Cannot remove /var/lib/docker after changing docker runtime root to /home/docker after a fresh install with no containers / images

[vivekdasgupta]

Description of problem:

Cannot remove /var/lib/docker just after a fresh install even when docker is stopped. And even though the docker runtime root has been changed to /home/docker and is no longer /var/lib/docker. There are no images and no containers yet.

#rmdir /var/lib/docker
rmdir: failed to remove `/var/lib/docker': Device or resource busy

lsof does not show anyone using this file/dir

Version-Release number of selected component :
RHEL 6
Kernel 2.6.32-431.el6
Docker version 0.11.1, build fb99f99/0.11.1

uname -a

Linux xxxx 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux

#docker info
Containers: 0
Images: 0
Storage Driver: devicemapper
Pool Name: docker-253:4-3801090-pool
Data file: /home/docker/devicemapper/devicemapper/data
Metadata file: /home/docker/devicemapper/devicemapper/metadata
Data Space Used: 291.5 Mb
Data Space Total: 102400.0 Mb
Metadata Space Used: 0.7 Mb
Metadata Space Total: 2048.0 Mb
Execution Driver: lxc-0.9.0
Kernel Version: 2.6.32-431.el6.x86_64

Steps to Reproduce:

1. yum -y docker-io
2. service docker start ; service docker stop
3. add ' -g="/home/docker" ' to other_args in /etc/sysconfig/docker
4. service docker start ; service docker stop
5. checked that the docker runtime env files are now in /home/docker
6. Try to remove or rename /var/lib/docker and it does not allow

Actual results:
rmdir: failed to remove `/var/lib/docker': Device or resource busy

Expected results:
Should have removed or allowed to rename the directory

[crosbymichael]

Can you check if it is still mounted?

[tianon]

I think this is technically a duplicate of #3894 (that ticket only mentions btrfs specifically, but the problem of mount points being left around is the same as btrfs subvolumes being left around, IMO).

[vivekdasgupta]

Yes it was still mounted.

cat /proc/mounts |grep docker
/dev/mapper/vg_xxxx-lv_root /var/lib/docker ext4 rw,seclabel,relatime,barrier=1,data=ordered 0 0
/dev/mapper/vg_xxxx-lv_home /home/docker ext4 rw,seclabel,relatime,barrier=1,data=ordered 0 0

Unmounting manually allows deleting the dir.

Yes this is same as #3894, so i ll close it as a dupe.

[hemano]

This helped me:
cat /proc/mounts |grep docker
sudo umount /path

[roshan3133]

# umount /var/lib/docker/aufs
# rm -fr /var/lib/docker

works for me !!!

[bamb00]

Does not work for me. The aufs remains as an empty directory. Rebooting the vm does not help with removing aufs directory.

   # cat /proc/mounts |grep docker
   # umount /var/lib/docker/aufs
      umount: /var/lib/docker/aufs: target is busy
      (In some cases useful info about processes that
      use the device is found by lsof(8) or fuser(1).)

[hieudang-agilityio]

Does not work for me. The aufs remains as an empty directory. Rebooting the vm does not help with removing aufs directory.

   # cat /proc/mounts |grep docker
   # umount /var/lib/docker/aufs
      umount: /var/lib/docker/aufs: target is busy
      (In some cases useful info about processes that
      use the device is found by lsof(8) or fuser(1).)

I got the same issue, but I reboot the host and retry @roshan3133's solution, it works for me

[ushuz]

hemano's solution above (https://github.com/moby/moby/issues/6077#issuecomment-278549989) works for me.

However, a contrib script was created in #7423 to address #3894, maybe helpful? Haven't test it myself.

https://github.com/moby/moby/blob/10866714412aea1bb587d1ad14b2ce1ba4cf4308/contrib/nuke-graph-directory.sh

[raphyphy]

This helped me:
cat /proc/mounts | grep docker
sudo umount /path

This worked for me on uninstalling docker in WSL. Thank you so much