Compose: Allow for variable substitution for volume names

[dpavlotzky]

Hi all,

# docker-compose -v
docker-compose version 1.8.0-rc2, build c72c966

I am trying to use one docker-compose.yml file to start mutliple multi-container services. So far, so good. The containers comprising one service share a data volume and this is where I am having some trouble.

I would like my docker-compose.yml file to look something like this:

version: '2'

services:
  php:
    image: dpavlotzky/wordpress_php:4.6
    volumes:
      - ${COMPOSE_PROJECT_NAME}:/data

  nginx:
    image: dpavlotzky/wordpress_nginx:4.6
    depends_on:
      - php
    environment:
      - SERVICE_80_NAME=wordpress
      - SERVICE_443_IGNORE=true
      - SERVICE_TAGS=${COMPOSE_PROJECT_NAME}
    volumes:
      - ${COMPOSE_PROJECT_NAME}:/data:ro

  mariadb:
    image: dpavlotzky/mariadb
    volumes:
      - ${COMPOSE_PROJECT_NAME}:/data

volumes:
  ${COMPOSE_PROJECT_NAME}:

Every service uses a data volume named after the project name. This usually is a url (e.g. www.test.com).

But this does not work unfortunately:

# COMPOSE_PROJECT_NAME=www.test.com docker-compose up
ERROR: The Compose file './docker-compose.yml' is invalid because:
volumes value Additional properties are not allowed ('${COMPOSE_PROJECT_NAME}' was unexpected)

So then I tried this:

version: '2'

services:
  php:
    image: dpavlotzky/wordpress_php:4.6
    volumes:
      - my_volume:/data

  nginx:
    image: dpavlotzky/wordpress_nginx:4.6
    depends_on:
      - php
    environment:
      - SERVICE_80_NAME=wordpress
      - SERVICE_443_IGNORE=true
      - SERVICE_TAGS=${COMPOSE_PROJECT_NAME}
    volumes:
      - my_volume:/data:ro

  mariadb:
    image: dpavlotzky/mariadb
    volumes:
      - my_volume:/data

volumes:
  my_volume:
    name: ${COMPOSE_PROJECT_NAME}

# COMPOSE_PROJECT_NAME=www.test.com docker-compose up
ERROR: The Compose file './docker-compose.yml' is invalid because:
volumes.my_volume value Additional properties are not allowed ('name' was unexpected)

The name trick seems to only work for external volumes...

I now work around this with a shell script that first creates a named volume (if it does not exist yet) with the COMPOSE_PROJECT_NAME as name and then creates a YAML file with COMPOSE_PROJECT_NAME as name:

version: '2'
volumes:
  www.test.com:
    external: true

and I include that using:

COMPOSE_FILE=docker-compose.yml:${COMPOSE_PROJECT_NAME}.yml

But I am actually hoping I could get by without the scripting...

Thank you for creating docker and docker-compose!

David

[shin-]

Hi David, what you're trying to do seems redundant to me given the fact that non-external volumes will already be namespaced using the project name. You can just call your data volume my_volume and there won't be any conflict.

Hope that helps!

[dpavlotzky]

Ahhh... that will do. Thanks for pointing that out shin-!

[brunocascio]

@shin- @dpavlotzky Same problem here.

Using different docker-compose files:

For example:

docker-compose-1.yml

...
volumes:
  api:
  mongo_data:
...

and

docker-compose-2.yml

...
volumes:
  api:
     external: true
  mongo_data:
     external: true
...

I got this error:

ERROR: Volume api declared as external, but could not be found. Please create the volume manually using docker volume create --name=api and try again.

I fixed it creating an .env file with content:

COMPOSE_PROJECT_NAME=myproject

and changing:

docker-compose-2.yml

...
volumes:
  api:
    external:
      name: "${COMPOSE_PROJECT_NAME}_api"
  mongo_data:
    external:
      name: "${COMPOSE_PROJECT_NAME}_mongo_data"
...

[predkambrij]

I'm wondering if there's any way to substitude volume names from .env

[iosven]

I would have been interested in a solution for this to support variables in volume, network, container and service names, as it makes the docker-compose.yml very reusable in the presence of one deployment per customer, but I understand that there are other ways to do it.

[matthewbloch]

This particular use case was solved in the 3.4 file format. You still can't use volume keys with environment variables, but you can specify their external names at the top-level like this:

volumes:
  dbstorage:
    name: appname_db_${RAILS_ENV:-development}

services:
  db:
    volumes:
      - dbstorage:/var/lib/postgresql/data
   # etc.

It might be more precise than using external because it will create non-existent volumes.