Compose: .env file not used when running docker-compose from sub directory

[alfredvaa]

Description of the issue

I have a docker-compose.yml in the root of my project. There is also a frontend.env and backend.env which is referenced by two different containers in the env_file parameter in docker-compose. Besides that, there is also a .env file in the root of the project to set some args inside the docker-compose

      args:
        - NODE_ENV=development
        - NPM_REGISTRY_TOKEN=$NPM_REGISTRY_TOKEN

The NPM_REGISTRY_TOKEN is defined in the .env file. When running docker-compose config from the root directory I can see that it sets NPM_REGISTRY_TOKEN and all variables for backend.env and frontend.env correctly. But when running docker-compose config from the subfolder /frontend I can only see the frontend.env and backend.env variables set correctly, but the NPM_REGISTRY_TOKEN is now an empty string.

I did not even knew that it is possible to run docker-compose from a sub directory but it appears so. My problem with all this is that I accidentally run it from the sub directory and since the backend.env and frontend.env variables was populated correctly I was confused about why my .env-file did not work anymore.

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018

Output of docker version

Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:26:49 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:32:21 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Steps to reproduce the issue

1. create a docker-compose.yml and an .env-file.
2. Use a variable from the .env-file in docker-compose.yml
3. cd into a sub directory and run docker-compose config

Observed result

.env is not used when running docker-compose from sub directory.

Expected result

I think that either none of .env, backend.env and frontend.env should work, or all of them should work.

Stacktrace / full error message

none

Additional information

[ndeloof]

This is how docker-compose is expected to behave, see https://docs.docker.com/compose/env-file/

Compose supports declaring default environment variables in an environment file named .env placed in the folder where the docker-compose command is executed

[hozmaster]

I think that the env_file-variable needs little bit more clarifications to documentation where this parameter effects to and when should to use it...

And same time .env file usage should be clarifications too ... Is there possibility to some other file than .env instead ?

[martens-web-netz-de]

Approved .env not being respected from sub directory even when using _--file_ and _--project-directory_:

Scenario:

$ cat .env 
TAG=v1.5

$ cat docker-compose.yml 
version: '3'
services:
  web:
    image: "webapp:${TAG}"

$ docker-compose config
services:
  web:
    image: webapp:v1.5
version: '3.0'

Test:

$ cd sub/

$ docker-compose --file ../docker-compose.yml --project-directory .. config
WARNING: The TAG variable is not set. Defaulting to a blank string.
services:
  web:
    image: 'webapp:'
version: '3.0'

[ndeloof]

This issue might have been solved by https://github.com/docker/compose/commit/b09d8802edb498a91a73bcd63603774c9b027b47

[glours]

@martens-web-netz-de we don't reproduce the issue with your last scenario on both MacOs & Linux with 1.24.1 & 1.25.0-rc2 version. It's behave as expected & the tag is replace by the value defined in the .env file

You can try to use the last RC release https://github.com/docker/compose/releases/tag/1.25.0-rc2
and use the new --env-file flag

$ docker-compose --file ../docker-compose.yml --env-file ../.env config

[martens-web-netz-de]

So sorry! ... my fault 🤦

$ docker-compose --version
docker-compose version 1.14.0, build c7bdf9e

Update to 1.25.0-rc2 fixed it 👍

$ docker-compose --file ../docker-compose.yml --project-directory .. config
services:
  web:
    image: webapp:v1.5
version: '3.0'

--project-directory .. still required though but will try --env-file 😉

[alfredvaa]

This is how docker-compose is expected to behave, see https://docs.docker.com/compose/env-file/

Compose supports declaring default environment variables in an environment file named .env placed in the folder where the docker-compose command is executed

I see! I already read that article but I can see now that I misread it a bit. I understood it as the .env should be placed where the docker-compose.yml was placed, but it is should be, as you say, placed in the directory where the command is executed.

It might be a matter of taste but in my opinion it would be better if the .env file had a relation to where the docker-compose.yml file is rather than from where I execute the script.

[ndeloof]

Unfortunately we hardly can't change behaviour for such a feature that has been introduced a long time we go and many people rely on. --enf-file introduces a very reasonable workaround so most use cases can be covered.

[tkhanh]

@martens-web-netz-de we don't reproduce the issue with your last scenario on both MacOs & Linux with 1.24.1 & 1.25.0-rc2 version. It's behave as expected & the tag is replace by the value defined in the .env file

You can try to use the last RC release https://github.com/docker/compose/releases/tag/1.25.0-rc2
and use the new --env-file flag

$ docker-compose --file ../docker-compose.yml --env-file ../.env config

@glours
I have updated to 1.25.0-rc2 but the issue "the variable is not set. Defaulting to a blank string." still remaining

docker-compose -f ./docker-compose.yml up --env-file ./.env --detach

I put .env file at the same folder of docker-compose.yml

[glours]

@tkhanh can you give more inputs, your docker-compose.yml & .env files (without sensible info) for example?

[tkhanh]

@glours it works for me now.
Previously, I called the command from another directory, it does not work. So I have to cd to the same folder of .env to make it work