Compose: allow using variables from docker-compose.env in volume paths (and other configs in docker-compose.yml)

[gggeek]

I know that it is already possible to use shell environment variables which will expand in the definition of e.g. the path of a mount volume

volumes:
    - "$SOLR_HOME:/opt/solr/home"

But to set all the environment variables I need, this forces me to write some shell wrappers around the invocation of docker-compose (a one liner, admittedly, but I am a lazy bum...)

Since I already use a docker-compose.env file to define variables which are used in the construction of the containers, I wonder:
would it make sense to expand the variables in the docker-compose.env file so that they are available in docker-compose.yml ? (apart from an obvious loop problem of using them to define the value for 'env_file'...)

[dnephin]

there's some related discussion in #2380, and #745 (as well as probably a few other issues that are tagged with area/config)

[glenngillen]

As I mentioned in #2352, there is precedent for this already with existing tools reading from a .env file. I care more about a solution than the specific file it uses however it seems to me that there's little value in re-inventing yet another approach to doing this.

So long as docker-compose reads from such a file I'll be happy though :smile:

[gggeek]

Indeed seems like #2352 is requesting the exact same thing (my bad for not spotting it and opening a dupe)

[dopry]

For me it would be much more preferable to have the environment read from the shell by default instead of a file. I can always source a file before running a command. The proposal in https://github.com/docker/compose/issues/2441 seems like a much better solution as it uses a familiar shell syntax for providing default values in variable expansion and doesn't create the loop problem the op mentions.

[stuckj]

Unless I'm mistaken, this issues seems to still not be resolved specifically because environment variables can't be used in volume keys (https://github.com/docker/compose/issues/3998). I'm running into this problem now.

[shin-]

https://docs.docker.com/compose/compose-file/compose-file-v2/#name

[stuckj]

Ah, thanks. I missed that the proposal in https://github.com/docker/compose/issues/3998#issuecomment-255069785 was merged in. The reply after it made it sound like it was rejected.

[chadyred]

To just change the ending name like this, if it could be a solution to your issue...

version: "3.4"
services:
    database:
        volumes:
            - "fix-name-volume:/var/lib/postgresql/data"

volumes:
    fix-name-volume:
        name: ${REAL_NAME_OF_VOLUME} # Real ending name
        external: false

[tsu1980]

In my case, Using volume name way didn't work. but put the envvar into volume name directly, it works.

version: "3.4"
services:
    database:
        volumes:
            - ${REAL_NAME_OF_VOLUME}:/var/lib/postgresql/data

Docker version 18.03.1-ce, build 9ee9f40

[asarkar]

@tsu1980 Where is REAL_NAME_OF_VOLUME defined, in the container, docker compose or your host?

[AnEmortalKid]

@asarkar

It works if REAL_NAME_OF_VOLUME is defined in your host.

version: '3.7'
services:
  testlet:
    image: alpine:latest
    volumes:
      - ${MY_DIR}:/mounted
    command: ["cat", "/mounted/foo.txt"]

So if I had the foo.txt file on /usr/blah, it works if you export MY_DIR=/usr/blah

[ZaurM]

Just to clarify @chadyred answer and @asarkar question about where this variable should be defined.

In my case it works if REAL_NAME_OF_VOLUME is defined in .env file.
It doesn't work if it resides in custom env_file