Compose: Cannot find Swarm volumes

[etoews]

Compose cannot find Swarm volumes when using volumes or volumes_from

Create a Swarm cluster (the best guide is Get started with multi-host networking) or use this handy gist.

Reproduce volumes

Create a docker-compose.yml file

version: "2"

services:
  test:
    image: alpine
    command: ls /
    volumes:
      - data:/data

volumes:
  data:
    external:
      name: my-data

Run the following

$ docker volume create --name my-data
my-data

$ docker-compose --verbose up
...
compose.volume.initialize: Volume data declared as external. No new volume will be created.
compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('my-data')
ERROR: compose.cli.main.main: Volume my-data declared as external, but could not be found. Please create the volume manually using `docker volume create --name=my-data` and try again.

$ docker volume inspect my-data
[]
Error: No such volume: my-data

$ docker volume inspect swarm-agent1/my-data
[
    {
        "Name": "my-data",
        "Driver": "local",
        "Mountpoint": "/mnt/sda1/var/lib/docker/volumes/my-data/_data"
    }
]

Compose and/or the volume inspect operation doesn't seem to understand the volume name will be prefixed with a node name in Swarm. And naturally we can't be encoding node names in our Compose files.

Reproduce volumes_from

Create a docker-compose.yml file

version: "2"

services:
  test:
    image: alpine
    command: ls /
    volumes_from:
      - container:swarm-agent

Run the following

$ docker-compose --verbose up
...
compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- (u'swarm-agent')
ERROR: compose.cli.main.main: Service "test" mounts volumes from "swarm-agent", which is not the name of a service or container.

$ docker inspect swarm-agent
[]
Error: No such image or container: swarm-agent

$ docker inspect swarm-agent2/swarm-agent
[ ... lots of JSON ... ]

Again, Compose and/or the container inspect operation doesn't seem to understand the container name will be prefixed with a node name in Swarm. And naturally we can't be encoding node names in our Compose files.

Versions

$ docker version
Client:
Version: 1.10.1
API version: 1.22
Go version: go1.5.3
Git commit: 9e83765
Built: Thu Feb 11 20:39:58 2016
OS/Arch: darwin/amd64

Server:
Version: swarm/1.1.3
API version: 1.22
Go version: go1.5.3
Git commit: 7e9c6bd
Built: Wed Mar 2 00:15:12 UTC 2016
OS/Arch: linux/amd64

$ docker-compose version
docker-compose version 1.6.0, build d99cad6
docker-py version: 1.7.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014

[dnephin]

Possibly a duplicate of, or some overlap with #2958

[krishamoud]

I'm having the same issue. Here is my compose file

version: '2'
services:
  interlock:
      image: khamoud/interlock
      command: -D run
      tty: true
      ports:
          - 8080
      environment:
          INTERLOCK_CONFIG: |
              ListenAddr = ":8080"
              DockerURL = "${SWARM_HOST}"
              TLSCACert = "/certs/ca.pem"
              TLSCert = "/certs/server.pem"
              TLSKey = "/certs/server-key.pem"

              [[Extensions]]
              Name = "nginx"
              ConfigPath = "/etc/conf/nginx.conf"
              PidPath = "/etc/conf/nginx.pid"
              MaxConn = 1024
              Port = 80
      volumes:
          - nginx:/etc/conf
          - /etc/docker:/certs:ro
          - /var/tmp/config.toml:/bin/config.toml
volumes:
  nginx:
    external: true

Here is the output of docker volume ls

local               aws-manager0/nginx
local               aws-node0/0cb67125e1a7873214140866aa04e75b5999363dc277b662cb0f5932d6c5714d
local               aws-node0/nginx
local               aws-node1/nginx
local               aws-node1/bd34a4852c970511dc663ad88d97b473c3f6d381c8bcc702b57a1627693bea09

[nikolajwestergaard]

+1 - I have same issue.

[nanounanue]

Same issue here

[homeles]

+1 having same issue

[aronahl]

I had this issue (with volumes not volumes_from), and I resolved it by removing same-named volumes from all swarm members other than the one where I wanted the service to run.

I also have node-affinity for each service that uses an external volume. I didn't test without the affinity.

[mithrandi]

It seems like you can now avoid this issue in the v2 compose format by using the explicit container:<container name> syntax.

(That is, blah still does not work, but container:blah now works)