Compose: Having multiple env files in monorepo

[zefexdeveloper]

I have setup a monorepo using lerna and yarn workspaces and I can easily config the env_file for each service I want, the thing is, for example:

version: "3.7"
services:
  postgres:
    image: postgres:11.1
    ports:
      - 5432:5432
    volumes:
      - pg-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=$DB_NAME
      - POSTGRES_USER=$DB_USER
      - POSTGRES_PASSWORD=$DB_PASSWORD

  server:
    build:
      context: .
      dockerfile: packages/server/Dockerfile.dev
    ports:
      - $GRAPHQL_PORT:$GRAPHQL_PORT
    depends_on:
      - postgres
    volumes:
      - ./packages/server:/usr/src/app/packages/server
    env_file: .env
    environment:
      - NODE_ENV=development
    command: yarn start

volumes:
  pg-data:

As you can see I have the variables for the database (DB_NAME) and also for the the server (GRAPHQL_PORT). I wanted to create a .env for server and one for the root with general variables for things like the database.

The problem:

docker-compose -f docker-compose.dev.yml up --build

It will read the .env in the root directory for filling the docker-compose.dev.yml and so I was wondering:

I can use -f to separate the docker compose files but if I create another .env inside the server directory it won't read the .env for that folder (with the server variables) and also the .env in the root for database variables.

It would be something like that:

├── packages
├── ├── server
├── ├── docker-compose.dev.yml
├── ├── .env
├── ├── .Dockerfile.dev
├── .env
├── docker-compose.dev.yml

and in order to run: docker-compose -f docker-compose.dev.yml -f ./packages/server/docker-compose.env.yml up --build

it works ONLY if the root .env has variables for both server and database, but if I put the server variables in the .env on the server folder the docker compose won't load that .env for the server and will throw me an error saying that the variable wans't set and is empty.

Is there a way to accomplish that or I really have to have one .env in the root for everything or I can separate my .env (how)?

[shin-]

Hi @zefexdeveloper

First off, please note that the .env file and the env_file property are two distinct concepts, though they often get conflated. You can only have one .env file at the root of your project, but the number of env_file is unlimited.

I would recommend splitting your environment files thus:

• One .env file at the root for Compose configuration values and variable substitution including things like your GRAPHQL_PORT value.
• Scoped environment files (postgres.env, server.env, etc.) for variables that need to be available in your container at runtime (that includes the POSTGRES_* values in your example).
• On a case by case basis, you may want to add the .env file to the env_file list for the rare service that needs to be aware of some of its values.

HTH - and in the future please use the forums for this type of question instead!

[zefexdeveloper]

@shin- Thank you for your answer.

One .env file at the root for Compose configuration values and variable substitution including things like your GRAPHQL_PORT value.
...
Scoped environment files (postgres.env, server.env, etc.) for variables that need to be available in your container at runtime (that includes the POSTGRES_* values in your example).

Docker will read for me the root .env file, I'm aware of that, and for that reason I have GRAPHQL_PORT and also variables for POSTGRES_*, the thing here is that, how am I able to create scoped environment files like postgres.env and server.env to pass as env_file in my docker_compose.yml if those variables are already inside the root .env file? I have two options here as far as I know, either use env_file: .env or write the scoped environment duplicating the variables (which I don't think is a good alternative).
That is why I asked, how can I create more than one .env so the Docker can read and replace the values in the docker-compose.yml and also I can pass those values as env_file for my containers at runtime?

I didn't knew there was a forum, next time I promise it will be there.

Thank you in advanced.

[shin-]

I may be misunderstanding the issue, but as indicated in the docs I linked to, you can just do the following:

env_file:
  - .env
  - server.env

Does that answer your question?

[zefexdeveloper]

@shin- GRAPHQL_PORT is a variable that should be on server.env because it's a server variable, but as docker-compose.yml needs the variable to replace the ports, it should be in the root .env, the question is: does Docker allows me to set more than one .env for docker-compose.yml to replace the variables in the file? If it doesn't, it means that I should put general variables in the root .env no matter the fact if it's for server or anything else and put others server variables in server.env, right?

[shin-]

Yes, GRAPHQL_PORT will always need to be in your root .env because it is needed to resolve your configuration.

[zefexdeveloper]

@shin- It's all clear now, thank you very much. Can you send me the forum link? I didn't knew there was one to be honest.

[shin-]

Glad I could help!
Sorry I didn't include the link to the forums earlier. You can find them here: https://forums.docker.com/