Compose: .env file is not read

[creynders]

I'm trying to use ENV variables declared in a .env file in my docker-compose file (I.e. I don't want to use the .env file in a container, just in the build process).

I have the following:

# file: build/docker-compose.development.yml

...
foo:
    volumes:
        - "${PROJECT_SRC}/foo.txt:/app/foo.txt"

# file: .env
PROJECT_SRC=../src



md5-0228655d431f2f2e73b24ad488ae6256



$ docker-compose -f build/docker-compose.development.yml build



md5-92beec8441567666920004423e210998



WARNING: The PROJECT_SRC variable is not set. Defaulting to a blank string.

Am I doing something wrong? Or doesn't it work with the -f flag? Or?

[dalejung]

I ran into the same issue. The environment variables are set within the container itself and is accessible via the environment block. Maybe volumes are executed before env_file is run.

[dalejung]

Taking a look, it looks like the env_file is processed in a finalization step. The volumes has already been interpolated with the incomplete environment dict by this point.

[rsynnest]

@creynders The .env file must be placed in the directory where docker-compose is run from (here's the documentation). I've tested this with latest build, it still works. Your post is a bit confusing though because it looks like you are running a build command, which doesn't actually run the container, and hence doesn't use runtime variables like the one defined in your volume block. If you are trying to pass arguments to your docker build process to build an image, you should instead be using the args: key within the build: block in docker-compose.yml, and then you can work with those ARGs in your Dockerfile.

I've noticed there is a bit of confusion surrounding environment variables (I was confused myself for a while). The documentation can be a bit unclear with its language on these terms, especially on of the official examples which confusingly loads the .env file in the the env_file block. This is the best documentation page I've found on various environment variables: https://docs.docker.com/compose/environment-variables/.

Some blurbs to try to clear up any confusion:

env_file
This file (or files) contains a list of docker environment variables which will be passed to the container as if they were in the environment: block. These files are explicitly defined in the env_file: config block. Anything in these files is passed as if it were via docker run --env-file=FILE.

.env file (https://docs.docker.com/compose/env-file/)
This file must be placed in the same directory where docker-compose is run from, and is automatically loaded by docker-compose. It contains a list of environment variables which you would like to use as variable substitutions (aka string replacement) within your docker-compose.yml. These vars are not passed to the docker container. For example:

volumes:
    - "${PROJECT_SRC}/foo.txt:/app/foo.txt"

where your .env file contains PROJECT_SRC=my/source/directory.

[creynders]

@rsynnest thank you very much for this explanation! So when I run docker-compose build the .env file is NOT automatically loaded? But when I run docker-compose up it is?
I only need the environment values _inside_ my docker-compose file, both during building and upping, which, if I understand correctly, isn't possible using .env, right?

[rsynnest]

@creynders When you run a docker-compose build, compose only uses 3 values from the docker-compose.yml file: context, dockerfile, and args. These are defined in the build config block. The .env is still loaded, it is always automatically loaded, even when you run docker-compose build.

So, when you define your volume in your docker-compose.yml file, that volume is only created and mounted at runtime, not at build time. If you wish to create a volume at build time, you need to define a volume in your Dockerfile, with the one restriction being that you can't mount a host directory, you can only define a volume for the container (for example, Dockerfile: VOLUME /foo/bar is identical to the following runtime command: docker run -v /foo/bar).

Docker-compose is mainly a runtime tool. Image building is still almost entirely handled in the Dockerfile.

[creynders]

Then why does it throw the warning? I have an .env file with PROJECT_SRC=foo and whether I run docker-compose build or docker-compose up it will always warn me that PROJECT_SRC isn't set, even though it is set in the .env file. However if I run PROJECT_SRC=foo docker-compose up the warning isn't thrown. I don't know, probably I'm missing something here, but it doesn't seem to make sense to me. Or it's broken.

[rsynnest]

Hmmm strange. What version of compose are you running? The .env file was introduced in 1.7.0

[ulugbekov]

I have the latest build docker-compose version 1.11.2, build dfed245
It is not reading .env variables.

[rsynnest]

Ok, I just helped a friend with this same scenario. The issue could be syntax in your .env file.
~Make sure the .env file has readable permissions, then~ check that the .env file syntax matches the rules defined here: https://docs.docker.com/compose/env-file/
Each line must be formatted like KEY=value. In my friend's cases he was using spaces around the equals sign (KEY = value) which fails silently.

Note that you environment variables on the host supersede .env file variables, so if you really can't get the .env file to work and you need a quick fix, you could write a bash wrapper that sets environment vars on the host and runs docker-compose.

EDIT: It's possible you have your var set to a blank string PROJECT_SRC='' on the host? This would override the .env file.

[florianm]

@rsynnest thanks for your explanation, helped me to understand. My confusion was that I factored out variables after building the images, and docker silently re-used cached images without my new variables. I had to delete images and volumes before the re-build finally recognised my variables.

[giovanniKleinCampigoto]

make sure you didn't insert export before the env variables...

[ababushkin]

I'm having the same issue on my end.

Here's my structure:

├── docker
│   ├── .env
│   └── docker-compose.yml

When I run docker-compose up -d from the docker folder, none of the contents of my .env file are being registered.

My .env file is written in the following format:

AWS_ACCESS_KEY_ID=<secret value>
AWS_SECRET_ACCESS_KEY=<secret value>

However, when I change my docker-compose to contain:

services:
  my_special_service:
    image: something
    env_file:
      - .env

Then everything works as expected.

Looks like a genuine bug?

Running MacOS.

docker-compose version 1.14.0, build c7bdf9e

[rsynnest]

@ababushkin please read my comment above

To summarize, .env file variables are not passed to the docker container at runtime. The .env file is used for variable substitutions in config files only
The env_file block on the other hand passes env vars from a file to the container at runtime, serving the same purpose as the environment block or docker run -e.

This bug is still not reproducible and I think is being caused by confusion around the term "environment variables".
@creynders are you willing to close this issue or are you still experiencing the problem?

[shin-]

@rsynnest Thank you for taking the time to help people here.

I'm going to close this issue now as it is 1. not a bug and 2. a duplicate of #4189 (and several others)

[ababushkin]

@rsynnest thanks for the clarification - I must've skimmed over your previous comment but can see it now :)

[thebetterjort]

Can the .env file be named variables.env in the env_file block. I'm only able to build using .env filename.

[jibusayone]

@thebetterjort I managed to find a solution! I used the env_file setting itself, and defined the variables.env within the same line. And it worked like a charm!

Here is how my docker-compose.yml looks like:

services:
  web:
    env_file: variables.env

[KirillAmurskiy]

@creynders I had the same problem. The variable is in .env file, but on docker-compose run I had warning "Bla_bla_variable not found".
In my case problem was with...
FILE ENCODING
If .env file is in Unicode -> will be warning.
If .env file is in ANSI -> will be OK.

[realpablodias]

@KirillAmurskiy Great! I changed from UTF-8 to ISO8859-1 and worked. But you know what? I just changed back to UTF-8 and it is still working (is this real life?)

[hynese]

I'm also experiencing this problem on OSX 10.13.2

docker version

Client:
 Version:      17.09.1-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   19e2cf6
 Built:        Thu Dec  7 22:22:25 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.09.1-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   19e2cf6
 Built:        Thu Dec  7 22:28:28 2017
 OS/Arch:      linux/amd64
 Experimental: true

[agenerette]

"The .env file was introduced in 1.7.0"

This (from rsynnest's Feb 18, 2017 comment) is a very, very important thing to mention. I spent I don't even want to say how long, going round and round on the "variable is not set" issue, before I read that line, above, and check my running version of docker-compose: I was on 1.6.2.

[radovanobal]

I get the warnings that the variables from the .env file are not loaded when I run docker-compose in CMD on windows but I do not get them if I run the same command from powershell.

[floatingpurr]

@ababushkin please read my comment above

To summarize, .env file variables are not passed to the docker container at runtime. The .env file is used for variable substitutions in config files only
The env_file block on the other hand passes env vars from a file to the container at runtime, serving the same purpose as the environment block or docker run -e.

This bug is still not reproducible and I think is being caused by confusion around the term "environment variables".
@creynders are you willing to close this issue or are you still experiencing the problem?

@rsynnest, just a side question. Vars in env_file: are passed to containers. Are those vars used also for variable substitutions in the Compose?

[rsynnest]

@rsynnest, just a side question. Vars in env_file: are passed to containers. Are those vars used also for variable substitutions in the Compose?

No

[filipbekic01]

If you guys have trouble loading different .env than one in root docker composer directory, consuding using system link. For example sudo ln -s /path/to/env .. This might solve confusion.

[iquito]

For me the .env file was not loaded because I was using the --project-directory option: in that case docker-compose reads the .env file from the --project-directory directory, not from the directory the compose file is in, which was not obvious to me at first. I wanted to add this in case someone else had the same problem and stumbled across this issue.

[jeterpan]

I was wrong using require('dotenv'),config() in my main project .js file. The solution was using it in the begning of the config.js (a module where I load all the environment)

[ereztaiar]

I am using mac and phpstorm,
Using the terminal in phpstorm the .env file is not working with docker-compose command and not setting the variables on the docker-compose.yml
Using the iterm there is no problem and docker-compose runs fine with no errors

[thaJeztah]

@ereztaiar I would recommend opening a ticket in the JetBrains issue tracker; https://youtrack.jetbrains.com/issues/WI