Vagrant: Unable to mount nfs volume in docker compose (vagrant)

[abhaynahar]

Vagrant 2.2.0

Host operating system

MacOs Mojave 10.14.2

Guest operating system

Vagrantfile

Vagrant.configure(2) do |config|
config.vm.define "iceland" do |iceland|
    iceland.vm.provider "docker" do |docker|
      docker.build_dir = "/iceland/server/vagrant"
      docker.create_args = ["--add-host=iceland:127.0.0.1"]
      docker.name = "iceland"
      docker.compose=true
      docker.compose_configuration={
          "volumes" => {
              'iceland_data_volume' => {
                  "driver" => "local",
                  "driver_opts" => {
                      "type" => "nfs",
                      "o" => "addr=host.docker.internal,nolock",
                      "device" => ":/iceland/data"
                  }
              }
          },
      }
      docker.volumes =['iceland_data_volume:/temp_directory/data']
      docker.ports =
          [
              "1000:1000",
          ]
    end
  end
  end

Dockerfile in the build_dir

FROM ubuntu:18.04

Expected behavior

Running vagrant up should have generated a docker-compose.yml that looks like this

version: '2'
volumes:
  iceland_data_volume:
    driver: local
    driver_opts:
      type: nfs
      o: addr=host.docker.internal,nolock
      device: ":/iceland/data"
services:
  iceland:
    build:
      context: "/iceland/server/vagrant/"
    environment: {}
    expose: []
    ports:
    - 1000:1000
    volumes:
    - "iceland_data_volume:/temp_directory/data"
    - "/iceland/server/vagrant:/vagrant"
    links: []
    command: []

Actual behavior

What actually happened?

---
version: '2'
volumes:
  iceland_data_volume:
    driver: local
    driver_opts:
      type: nfs
      o: addr=host.docker.internal,nolock
      device: ":/iceland/data"
services:
  iceland:
    build:
      context: "/iceland/server/vagrant"
    environment: {}
    expose: []
    ports:
    - 1000:1000
    volumes:
    - "/iceland/server/vagrant/iceland_data_volume:/temp_directory/data"
    - "/iceland/server/vagrant:/vagrant"
    links: []
    command: []

Steps to reproduce

Use the above vagrant and docker (vagrant up) file to generate the following docker-compose.yml inside of .vagrant/docker-compose directory

The bug

volumes explicitly created in docker compose are appended with the current path
"/iceland/server/vagrant/iceland_data_volume:/temp_directory/data" where as in my case it should have been only "iceland_data_volume:/temp_directory/data" where iceland_data_volume is a nfs volume.
This behaviour is happening because of https://github.com/hashicorp/vagrant/blob/85acf0cac724ef4bfda593a66e0c17f7e4110da0/plugins/providers/docker/driver/compose.rb#L105 line of code I think.

[briancain]

:wave: Hello @abhaynahar - Have you tried with the latest version of Vagrant? I think maybe this was fixed in the docker provider since version 2.2.0. Thanks!

[abhaynahar]

@briancain I tried it with vagrant 2.2.4 and it gives me the same result :/

[briancain]

Ah, thanks @abhaynahar - I think the fix I was thinking of is in the regular docker driver, not the compose driver.

[abhaynahar]

thanks @briancain

[abhaynahar]

@briancain can you help me with dates of the fix. How long does it usually take to fix a bug like this?

[briancain]

@abhaynahar - the fix itself doesn't look too complicated....just not sure when I'll be able to start fixing it if I do it myself. I'm putting it in the next milestone so hopefully soon.

[briancain]

Hey @abhaynahar - what is the full path to your NFS mount? Is iceland_data_volume being mounted from else where onto your host machine, and then you just want to use regular shared folders for your container? I believe the "type" => "nfs" option is saying that your local project folder will use NFS to mount it on your container, not that your local folder is an NFS share. (I might be misunderstanding your issue though!)

[abhaynahar]

iceland_data_volume is being created from my host machine path /iceland/data I am trying to create a nfs type mount out of the local path using docker compose, so that It can be mounted in the container as nfs disk. The reason to do all this is to get better disk performance on my mac. Currently my development setup is very slow with the regular docker read and write sync.

[briancain]

@abhaynahar - in that case your path should actually be '/iceland/data. Does `iceland_data_volume exist at all? :thinking:

 docker.volumes =['/iceland/data:/temp_directory/data']

With how you configured it before, Vagrant assumed the folder you were using was in your current Vagrant working dir, and that is why it was path expanded. If your folder is at your root you need to specify that. Doing so I think will resolve your issue.

[abhaynahar]

@briancain the reason I am trying to mount iceland_data_volume is because i have made it as a nfs device in the docker compose file
docker.compose_configuration={ "volumes" => { 'iceland_data_volume' => { "driver" => "local", "driver_opts" => { "type" => "nfs", "o" => "addr=host.docker.internal,nolock", "device" => ":/iceland/data" } } }, }
so when i am able to mount it, it will be much faster than the native mount of docker on mac.

[briancain]

@abhaynahar Ahhh ok, I see now. If you remove the docker.volumes configuration and just use that docker.compose_configuration option instead, does it work? I think maybe the volumes option isn't required if you are just giving docker compose its full configuration through that hash.

[abhaynahar]

I will have to use volumes otherwise the services will have no connection to the volumes.

[briancain]

Hi @abhaynahar - I was able to get it to work with these settings:

  (1..3).each do |i|
    config.vm.define "docker-#{i}"  do |docker|
      docker.vm.provider "docker" do |d|
        d.compose = true
        d.compose_configuration={
          "volumes" => {
              'iceland_data_volume' => {
                  "driver" => "local",
                  "driver_opts" => {
                      "type" => "nfs",
                      "o" => "addr=host.docker.internal,nolock",
                      "device" => ":/iceland/data"
                  }
              }
          },
        }
        d.volumes = ["/iceland/data:/temp_dir/data"]
        d.git_repo = "https://github.com/briancain/nginx-docker-test.git"
        d.cmd = ["tail", "-f", "/dev/null"]
      end
    end
  end

vagrant@vagrant:~/test$ ls /iceland/data/test.txt
/iceland/data/test.txt
vagrant@vagrant:~/test$ cat /iceland/data/test.txt
This is a test file.
vagrant@vagrant:~/test$ vagrant up docker-1
Bringing machine 'docker-1' up with 'docker' provider...
==> docker-1: Building the container from the git repository: https://github.com/briancain/nginx-docker-test.git...
    docker-1:
    docker-1: Image:
==> docker-1: Creating the container...
    docker-1:   Name: test_docker-1_1556039384
    docker-1:  Image:
    docker-1:    Cmd: tail -f /dev/null
    docker-1: Volume: /iceland/data:/temp_dir/data
    docker-1: Volume: /home/vagrant/test:/vagrant
    docker-1:
    docker-1: Container created: 9607cab97e9046b4
==> docker-1: Starting container...
==> docker-1: Provisioners will not be run since container doesn't support SSH.
vagrant@vagrant:~/test$ docker ps
CONTAINER ID        IMAGE               COMMAND               CREATED             STATUS              PORTS               NAMES
9607cab97e90        test_docker-1       "tail -f /dev/null"   12 seconds ago      Up 11 seconds       80/tcp, 443/tcp     test_docker-1_1
vagrant@vagrant:~/test$ docker exec 9607cab97e90 ls /temp_dir
data
vagrant@vagrant:~/test$ docker exec 9607cab97e90 ls /temp_dir/data
test.txt
vagrant@vagrant:~/test$ docker exec 9607cab97e90 cat /temp_dir/data/test.txt
This is a test file.

__Edit:__ Hmm, although it doesn't seem to be an nfs mount:

vagrant@vagrant:~/test$ docker exec 9607cab97e90 stat -f -L -c %T /temp_dir
overlayfs

[briancain]

Ok I think I have a better understanding of what's going on after reading this forum post: https://forums.docker.com/t/how-to-mount-nfs-drive-in-container-simplest-way/46699/2

So essentially the name iceland_data_volume is supposed to be created by the docker compose config file, and then the volumes section later should reference that value (like you were saying originally). I'm not sure if volumes should always be expanding it regardless of the volume definition or not, or if this is just something special with how docker compose handles nfs volumes.

[briancain]

I am also not sure if your host is correct, when I just remove the path expansion docker gets upset about the address used for NFS. Just an FYI:

ERROR: for docker-1  Cannot start service docker-1: error while mounting volume '/var/lib/docker/volumes/test_iceland_data_volume/_data': error resolving passed in nfs address: lookup host.docker.internal: no such host
Encountered errors while bringing up the project.

__Edit:__ Ahh scratch that. That seems to be a special alias for Docker for Mac :)

[briancain]

So it looks like this path expansion was originally introduced because of https://github.com/hashicorp/vagrant/issues/8822 ... where the volumes option would wrongfully expect the source mount to exist where the docker compose file exists (which is at the local .vagrant dir of your project directory). This path expansion line should still exist then in the compose driver, however what is tricky is that it _doesn't_ make sense if you're trying to also use the volumes option directly inside a services config block in a docker compose config file, which I think is essentially treating that value like a variable (i.e. the --name value with docker volume) rather than an actual path. :confused:

Perhaps we could have some logic around seeing if a volumes hash was specified for more advanced configuration, and then do not path expand if the key was used as a source dir in the docker.volumes configuration for any services....since that value is now a key and not a "real path"? :thinking:

[abhaynahar]

@briancain totally agree with your last comment. I think we have to check if the given volumes in services refer to any volume in the volumes section itself and only path expand if nothing matches

[abhaynahar]

@briancain thanks for fixing it!

[arturgrabowskitb]

If anyone lands here in search for a missing volumes solution. This worked for me:

 localCfg.vm.synced_folder ".", "/vagrant", disabled: true
 localCfg.vm.provider "docker" do |d|
        d.compose = true
        d.image = vmCfg[:docker][:image]
        dockercompose = YAML.load_file('./docker-compose.yml')
        d.compose_configuration = dockercompose
        d.volumes = dockercompose['services']['yourcontainername']['volumes']
 end

Hope that helps

[OZZlE]

Seems like it only works in VirtualBox v6.0.2! (+vagrant 2.2.2 - not sure it matters though!)

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.