Compose: Global env variables

[analytically]

I'd be great to be able to declare 'global' env variables, that are set for all containers.

[d11wtq]

I think you can use YAML features for this. I haven't tested it myself and maybe Python's YAML parser doesn't support the syntax, but:

global: &global
  FOO: bar
  BOB: baz

web:
  environment:
    <<: *global
    EXAMPLE: more

The only downside here is that you'd need to explicitly fig up web, rather than just fig up, since fig will probably assume global is a service.

[bfirsh]

Maybe services starting with underscores could be ignored.

[d11wtq]

Hmm, I know it's purely aesthetic, but I think that could make the fig.yml look a bit ugly and wouldn't be immediately apparent to somebody reading the fig.yml for the first time. Maybe it would be best to introduce some sort of a reserved namespace key after all, so things of this nature could go under it. Say meta. That increases the complexity of implementing the feature though, as (IIRC) YAML alone won't help you out here.

[d11wtq]

I wonder if there's a way for fig to detect that a key is marked as an anchor and ignore it in that case (i.e. ignore global: &global, but not global:). Depends what the YAML parser allows you to know.

[ryanbrainard]

I have run into a similar issue where I have multiple entry points into the same image, so I end up duplicating a bunch of env vars and other metadata in my fig.yml. Instead of global properties, it would be nice to have some form of inheritance for the service definitions. Perhaps it could follow the Docker style and have a from attribute for the service definition. You could then create an "abstract service" with a base definition and then have the other services inherit from it. For example, something like this:

base:
  abstract: true
  build: .
  environment:
   FERNET_SECRET: 123449b81234
web:
  from: base
  command: web
  ports:
    - "5000"
  environment:
   PORT: "5000"
worker:
  from: base
  command: worker

[humphries40]

I think you can use YAML features for this. I haven't tested it myself and maybe Python's YAML parser doesn't support the syntax, but:

global: &global
FOO: bar
BOB: baz

web:
environment:
<<: *global
EXAMPLE: more

The only downside here is that you'd need to explicitly fig up web, rather than just fig up, since fig will probably assume global is a service.

So I was looking at this and looking at the service.py file. Couldn't you modify that file so that when it's looking for services if it finds one named say "global" or "globalvars" (or something) instead of doing the normal service stuff it would do something a little different? Say it would modify all existing services and all future services and add the values in Global to the environment values. Or you force Global to be declared at the beginning of fig.yml so that it can just add everything to the services as they are created. This would at least give you global static variables that wouldn't require having to repeat them everywhere.

[dnephin]

For #318 I need a section in the config that isn't a service definition (similar to this).

Maybe a single key like meta (or _meta) could be used at the top level, and everything else (global envs, includes, other things that might be added later), could be added under this single key. That way there are fewer "reserved words" in the config.

[d11wtq]

Yeah, in support of the idea of "reserving" a key for fig-specific things. meta is probably a good choice. global is a bit too specific to a particular problem domain.

[humphries40]

@ryanbrainard have you tried implementing anything like your suggestion?

[kanzure]

@d11wtq, yep that works. You can use yaml to share environment configuration between containers like so:

api:
    environment: &environment
        key: value
ui:
    environment: *environment
db:
    environment: *environment
logtaker:
    environment: *environment

This is presently working on my end.

[thomasf]

non container root level entries

For me somthing along the lines of abstract: true (as suggested earlier) or maybe autostart:false would be a good fit.
EDIT: A root level entry not specifying a build or image key could also be used for metadata only(?)

profiles

I think that this change also should take something like "profiles" in consideration even if that is to be implemented later. My reasoning here is that both features call for some kind of conditional selection in the yaml tree and might benefit from being discussed together before implementing one of them.

One example is that I might want to switch between mounting volumes on host or in containers.

Another example is switching between binding to a specific port on the host and not doing that, since you cannot do "a fig run ..." if that container is running and also is configured to bind a specific host port.
This example is also a use case for the autostart:false suggestion.

[shin-]

@dnephin @aanand bump - with the introduction of the new format, is this something we're still considering?

[dnephin]

Now that we support environment variable interpolation these variables can be provided outside of the compose file. There's still an open issue about providing defaults, or reading values from a file, which I think are more appropriate now, so we can close this I think.

env_file and extends can also be used to solve this problem.

[sw-carlin]

@dnephin Even with the interpolation we have to list all the environment variables that we want to pull in, for each service that needs them. That can lead to a lot of repetition in the docker-compose.yml.

extends was dropped in v3 #4315

[jpic]

@sw-carlin agreed

[mcobzarenco]

Is there a workaround for this or does one have to specify the variable individually for all containers?