Compose: Proposal: make project-name persistent.

[thaJeztah]

By default, Compose bases the project name on basename of the directory compose
commands are run from. The project name can be overridden either by
passing a -p / --project-name option for each command or setting the
COMPOSE_PROJECT_NAME environment variable.

Using the --project-name option for _each_ command is error-prone and for-
getting to pass the option when doing (for example) docker-compose up, will result in
_another_ instance of the project being created under a different name or even
containers of a different project being replaced.

Using the COMPOSE_PROJECT_NAME environment variable is not useful when dealing
with multiple projects and can cause similar problems.

Proposal: make project-name persistent

To solve these problems, compose will save the name of a project to a file in the
build-context when the project is first started / built.

If a project-file is found, compose automatically uses the project-name from that
file and throw an exception if the user is trying to override the project-name
using the --project-name option.

File location

To allow further expansion of options, compose creates a hidden .docker-compose directory
in the "root" directory of the build-context. Inside that directory, a
project-name file is created, containing just the name of the project;

tree -a
.
├── .docker-compose
│   └── project-name
└── docker-compose.yml

Request for comment

There are a couple of things left to be discussed;

• Should the configuration-file be created _automatically_ or should this
  be an explicit action by the user (e.g. docker-compose init --project-name=foobar)
• Should a command be added to _remove_ the configuration file(s)? (e.g.
  docker-compose destroy)
• Compose allows specifying an alternative compose-file (--file) should the
  project-name also be applied to those? Should each compose-file get its own
  configuration?

And, in a wider scope;

• Is the project-_name_ actually important? If not, compose could create a _random_
  project-name and store that inside the configuration. This would greatly reduce
  the risk of conflicting names with other projects running on the same server.

edit: renamed Fig to Compose

[bfirsh]

The project name isn't important. I like the idea of a random project name. Even better – could we generate a hash of the path to fig.yml and use that?

[dnephin]

I actually think the project name is pretty important. If you want to be able to do anything with the images in a CI pipeline, you need to know the project name to be able to retag them as something else.

Being able to override project name from an environment variables makes it easy to keep these unique, and predictable. I think no matter what happens, we need to support the override from environment variables.

I do like the idea of making the project name configurable from a file. I think a similar discussion happened in (#45). Having the project name in.fig/project-name seems like it will lead to a lot of really small files. I think it would be easier to just put it into the fig.yml itself (like it is suggested in #45, and one of the docker compose proposals).

What are the advantages of putting it in a separate directory and file?

[thaJeztah]

The reason for putting it in a separate file has a number of reasons, I'll try to explains my thinking behind this;

• To preserve the name that was used to _start_ the project (i.e., --project-name=foobar), which can be different from the automatic project-name.
• Having multiple instances or _versions_ of a project running on the same server, without them conflicting each other
• Or, in short, the .fig/project-name holds the name of _that instance_ of the project.

Perhaps the file should be called instance-name or similar.

I do like the idea of making the project name configurable from a file

Although _possible_ to do this with this proposal (by manually creating the project-name file), my primary use case was to have _fig_ create the file in order to save the name that was used.

will lead to a lot of really small files.

True, the alternative is to skip the .fig directory, however, if additional things are needed in future, extra file(s) can be added without cluttering the root of your project. Git does it similar and I think it's a clean approach.

I think it would be easier to just put it into the fig.yml

I think both options can be complementary; use the name from fig.yml as the default if the name is not overridden by a flag or env-variable. Put the name that is actually used to _start_ the project inside the .fig/project-name

Being able to override project name from an environment variables makes it easy to keep these unique, and predictable. I think no matter what happens, we need to support the override from environment variables.

Curious; how do you handle managing multiple projects? I.e. cd project-a && fig ps then cd project-b fig <something>?

Thanks for your feedback!

[dnephin]

Curious; how do you handle managing multiple projects?

I usually run fig from python-tox, which lets me set the environment, so I'm never really setting it in my shell. I also tend to use tmux, so I have a different shells for every project.

I think both options can be complementary

Cool, I agree.

I don't know if I'm sold on .fig/instance-name vs say .fig-instance.yml that could contain any instance specific overrides, but I think this is more of a minor point.

[thaJeztah]

Thinking of the project-name; If I understand your situation correctly, it's important to be able to have control over the _images_ being produced, e.g. myapp:build12345, or also names of the _containers_? Just to get a "feel" for your situation.

My thoughts behind "random" project-names was that, as long as Fig is able to locate the containers for a project, nice-looking names aren't important. If I, as a user, can access a services' container via its service-name (e.g. fig stop web), the name of the actual container doesn't matter.

Even have been thinking that fig could keep track of containers by ID in a local storage inside the .fig directory (which could be a performance gain on machines with a lot of running containers). But that's really something for a separate issue.

Any thoughts on "implicitly" creating the 'project-name' file or "explicitly" via fig init? Perhaps if I have some time during the coming holidays I can experiment a bit (never coded anything in Python, so it will be _really_ experimental :))

[dnephin]

it's important to be able to have control over the images being produced, e.g. myapp:build12345, or also names of the containers?

I guess image is the really important one, but being able to ensure that container names don't conflict is also really important on shared hosts. I see this proposal actually aims to deal with that issue as well.

But I do think that predictable names for containers are still important. I can think of a couple tasks where it's important for external systems (not just fig) to be able to identify a container by name:

• cleanup scripts to remove old containers, being able to identify what project created a container
• debugging/inspecting containers

Both of these are pretty easy right now because I already know what the container name will be. If I have to lookup a name it's somewhat more involved.

Even have been thinking that fig could keep track of containers by ID in a local storage inside the .fig directory

It is true this could be a performance gain, but I worry this is the wrong approach. Adding any persistent state to fig introduces the possibility of lots of bugs (that come along with state). I would much rather see this being handled by a labeling system in dockerd. If fig is able to just label containers and then query for all containers with that label , it keeps all the state in a single place (dockerd). I know there was talk of this at some point, I'm not sure if it's on the roadmap.

Any thoughts on "implicitly" creating the 'project-name' file or "explicitly" via fig init?

I guess implicitly would be more backwards compatible. I would like to avoid a fig init unless absolutely necessary. Instead of using the basedir, I could see it implicitly creating a project name of <basename>-<4 digit random id>

[frank-dspeed]

i can only say if fig starts generating random names now i will switch away from it because i run 1000+ containers on a single host and when i cant identify them by name any more its nothing for me.

[thaJeztah]

@frank-dspeed thanks for adding your use-case. If I understand correctly, you _start_ your containers with fig, but _"manage"_ them directly via Docker after that, making use of the naming convention that Fig uses?

Perhaps this is of interest to you as well: https://github.com/docker/docker/pull/9882 - having meta-data would offer more ways to identify containers, not only their name.

[frank-dspeed]

ah yes i did such proposals a lot for example simply adding new filds and that but i ended in that case with simply adding a ENV and then i have that as costum fild i can parse it :dart:

[funkyfuture]

I think managing the unambiguity of the project name shouldn't be fig's responsibility.

when i understood your proposal correctly, the generation of a random name would render the FIG_PROJECT_VARIABLE and the --project-name-option useless. which is handy for 'templating' services.

[thaJeztah]

I think managing the unambiguity of the project name shouldn't be fig's responsibility.

I'm not so sure; Fig "silently" overwriting another projects' containers because if happened to be in a directory with the same name is sometimes nasty.

when i understood your proposal correctly, the generation of a random name would render the FIG_PROJECT_VARIABLE and the --project-name-option useless. which is handy for 'templating' services.

Following the discussion above, I think something like this would work

• If --project-name is provided, use that (and write/update the .project-name? Not sure)
• No --project-name is provided, but FIG_PROJECT_NAME is, use FIG_PROJECT_NAME (and write/update to .project-name?)
• None of the above, but .project-name is set, use .project-name
• None of the above; create a _random_ name and write it to .project-name

[funkyfuture]

from someone's perspective who uses fig in scripts that take care of a name and pass it to fig, storing a name in the project (which is in my case rather a template) location makes no sense.

and still, i sensed it very intuitive that the directory name is used in the name of the resulting containers when i just ran fig up.
and let say, you look at docker ps -a, that won't be very helpful if it's filled up with random names.

would an option --random-name help in your situation?
beside that i think the ability to prepend some [a-zA-Z0-9_].* to reflect some namespacing would be useful for broader deployments.

[jderusse]

What's about using the fig.yml to store such informations ?

schema-version: 1.1
project_name: foo
containers:
  web:
    build: .
   command: python app.py
    links:
     - db
    ports:
     - "8000:8000"
  db:
    image: postgres

And to keep BC, in compose/project.py::from_config

@classmethod
    def from_config(cls, name, config, client):
        if 'schema-version' not in config:
            config = {
                'schema-version': '1.0',
                'containers': config
            }
        dicts = []
        for service_name, service in list(config.containers.items()):
            if not isinstance(service, dict):
                raise ConfigurationError('Service "%s" doesn\'t have any configuration options. All top level keys in your docker-compose.yml must map to a dictionary of configuration options.' % service_name)
            service['name'] = service_name
            dicts.append(service)
        return cls.from_dicts(name, dicts, client)

[mikehaertl]

@jderusse I'm very much for something like you suggested. Maybe you like to add your proposal to #45 as this is probably the better place to discuss it?

[michaeljs1990]

anyway to come to resolution on this ticket? It looks like the proposed #1233 gives you the best of both worlds by letting the user decide the intended action to take while not breaking any backwards compatibility.

[dnephin]

Originally I wanted the name in the docker-compose.yml, but after thinking about it more, I really like the idea of a separate file.

A separate file give you the option to keep it out of version control if you want (for cases where you want each user to be able to have a different project name).

With the new networking support there is feature request to be able to configure the network name as well (it defaults to the project name). So with a separate config file we could include things like default network name, default scale etc. All the meta-configuration that isn't part of the application definition, but is still relevant to running the composition.

[mikehaertl]

I agree to what dnephin said. How about a file .docker-compose? We could list default options for all command line arguments there.

[thaJeztah]

@dnephin sgtm

@mikehaertl I used a .docker-compose folder in my example, but a file could work as well, depending on how much information we want to store

[mikehaertl]

@thaJeztah Oh, right, sorry. I missed that.

[thaJeztah]

@mikehaertl just renamed them from .fig to .docker-compose ;-)

[mikehaertl]

Right :)

Actually I'd prefer a simple file. How about ini style? Global options on top, command specific options in a section:

project-name = blabla

[build]
no-cache

[dnephin]

Some ideas for semi-related information we may want to store either in the same file, or same directory:

• network name (users may want a network name different from the project name. We've already had one request for that.)
• client api version
• docker host
• default scale value

Now that we support multiple compose files, it might also be interesting to include the paths to the compose files to use as config.

[thaJeztah]

client api version

possibly docker_host ?

[dnephin]

That sounds like a good one as well (added)

[jdmarshall]

I just ran into this problem.

I'm at a point where the basic structure of our docker configuration has been settled, and now I'm replicating it to other services. Since all of the projects have the same structure, they all use the same subdirectory (and hence, default project name). Which means I can't safely spool up 2 services on the same host.

I may have to change the structure so the project directory name ends up being the default, but I'm not 100% positive what that does about copying files at image build time.

[schmunk42]

Same here, I solved a similar issue with Makefile targets. But still, when you want to execute custom docker-compose commands, you need to use -p customname.

[chris-martin]

Right now I'm trying to figure out how to prevent docker-compose setups in /home/alice/myproject and /home/bob/myproject from stepping on each other's containers. Is there a way to deriving container names from the full file path instead of just the last directory name?

[schmunk42]

@chris-martin This is just a workaround...

alias docker-compose="docker-compose -p ${PWD}"

[chris-martin]

docker-compose strips / out of project names, though?

Is it documented which characters are allowed in project names?

[jdmarshall]

I did end up changing my directory structure and putting docker-compose.yml in the project root, which is a decent enough workaround

It grants some protection, if you tend to put all your git repositories into the same directory (eg, $HOME, or $HOME/projects), but it gets troublesome if you use another structure (say, one directory per organization, as I often do), or if your machine uses boot2docker, which tends to leak information across users on the same box unless you are careful to make sure docker-machine gives you distinct VMs.

I think #2294 or just being more hygienic with docker-machine would also fix my problems.

[schmunk42]

I'd also recommend to put it in the project root and have a different folder name.

[josephearl]

In my case I'd like to set the initial project name because I have a shared machine on AWS and although I have the docker-compose.yml in the root a user may clone the repository into a directory with a different name. If that happens currently then they run into issues starting/stopping containers.

[schmunk42]

WIth the new v2 syntax this could be implemented like mentioned in https://github.com/docker/compose/issues/745#issuecomment-74028861

It's actually just a container_name_prefix, isn't it?

[cr7pt0gr4ph7]

@schmunk42

It's actually just a container_name_prefix, isn't it?

It's also used as volume_name_prefix, image_name_prefix and network_name_prefix, so I guess that project_name is a more generic term.

.docker-compose sounds familiar to docker-compose.override.yml in a lot of way. Plus, why introduce another configuration format (e.g. ini) when we already got YAML?
I'm therefore in favor of @jderusse's suggestion in #745 (comment) to add a project_name key to the docker-compose.yml, to enable the use case of @JosephEarl.

User-specific customization can already be implemented by using docker-compose.override.yml and excluding it via .gitignore.
If a third explicitly user-specific override file is desired, I'd suggest .docker-compose.local-override.yml.

Regarding the structure of the YAML, I'd suggest creating a new top-level key called project, which contains project_name, and might contain other variables in the future. This avoids polluting the top-level namespace. To illustrate:

project:
  project_name: "your-project"
  network_prefix: "abc"

Note that, instead of using network_prefix, it may be more understandable to directly customize the names of the networks themselves. As I understand it, there are two cases:

• The first case is "just give the networks a unique name, but I don't care which one exactly". For this case, project_name is enough.
• The other case are networks which are referred to by external systems. In this case, explicitly specifying the network names themselves is probably better anyway.

[josephearl]

The proposal suggested above sounds good to me, I agree we don't want to introduce any extra config files

[simonvanderveldt]

+1 on the concept of this issue
But also +1 on using a key in docker-compose.yml instead of adding another file to our repos.

[cr7pt0gr4ph7]

To sum up earlier comments: One proposal for the lookup sequence of the project name would be as follows; earlier items take precedence over later ones:

1. Use --project-name if specified
2. Use COMPOSE_PROJECT_NAME if specified.
3. Use project_name key from docker-compose.yml (or wherever that setting is stored).
4. Use the basename of the project directory

I'm unsure of the priority of 2 vs. 3:

• For consistency with --project-name, COMPOSE_PROJECT_NAME should definitely override project_name:.
• But: Making COMPOSE_PROJECT_NAME take precedence over project_name: makes it possible to accidentally use the wrong project name due to the environment variable being set; this can be hard to debug. Maybe a warning message can be issues in this case?

[jderusse]

what's about introdusing a new property container_name_pattern with the default value %(project_name)s_%(service_name)s_%(instance_number)s to keep BC
this parameter lets us free to change it to hardcodedproject_%(service_name)s_%(instance_number)s and definitively fix this issue

[atd]

I have just arrived this issue after looking for this functionality for 10 minutes.

The first thing I did was searching for the right key in the Compose File Reference document, so +1 for project_name key in docker-compose.yml

[dmitry]

:+1: for @cr7pt0gr4ph7 strategy

[zeroows]

+1 for @cr7pt0gr4ph7 strategy

[jdmarshall]

Compose has syntax for environment substitution. Rather than using an environment variable for project name, maybe let people do it themselves if they care?

On the other hand, docker machine uses the end to decide which machine a compose build will be built on, so maybe the project name and the target should work alike. Hmm, this is a hard one.

[dnephin]

https://github.com/docker/compose/issues/745#issuecomment-182296139 sounds right. The environment variable would take precedence over the value in the compose file, which acts as a default (assuming the project name belongs in the compose file).

Something to consider, what happens when you use multiple compose files that each define a project name?

I don't think it should be an error, that would force files to be either "primary" or "extras", which is not the case right now. Any file can be used in isolation or in combination with others.

Maybe a warning is reasonable, or maybe we just ignore it completely (since the value can be ignored by setting an option or environment variable as well).

[josephearl]

Something to consider, what happens when you use multiple compose files that each define a project name?

If you do something like docker-compose -f compose1.yml -f compose2.yml up and both files define the project name, the name used should be from compose2.yml.

[kajmagnus]

I think everyone is onto a solution I like already :- ) I.e. project_name in docker-compose.yaml. I just want to mention another use case for having the project name in the docker-compose.yaml file:

In HTTP 500 Internal Server Error pages in my project, I tell the developer how to fix or troubleshoot the problem, for example I tell him/her that s/he can zcat database-dump.tgz | docker exec -i projectname_db_1 psql if the server notices that no PostgreSQL database and user has been created — and then I want to choose projectname myself, otherwise the help messages cannot be copy-pasted into the console.

[josephearl]

Agree that there should be a warning when COMPOSE_PROJECT_NAME overrides the project name in the compose.yml - this behaviour makes sense based on the existing Docker commands, but is not particularly logical or obvious to a newcomer.

[omercnet]

+1 for @cr7pt0gr4ph7 strategy

[intellix]

Tried searching for something before opening an issue myself as I couldn't find anything.
+1 for project_name in docker.compose.yml file. With v2 I guess this makes more and more sense.

[simonvanderveldt]

Agree that there should be a warning when COMPOSE_PROJECT_NAME overrides the project name in the compose.yml - this behaviour makes sense based on the existing Docker commands, but is not particularly logical or obvious to a newcomer.

It's pretty normal behaviour for environment variables to override config settings and for command line arguments to override config settings and environment variables.
Do we really a warning for that? It's not like anyone would accidentally create an environment variable named COMPOSE_PROJECT_NAME.

Would it be OK to create a PR that just uses the project_name from docker-compose.yml to get this going? Or do we want the extra stuff like the using the project_name value from the last yml file referenced immediately?

[fesor]

How about this?

version: "2"
project:
    default_name: "app"

I will change my implementation (https://github.com/docker/compose/pull/3118) for this format. But I'm not sure is there any need of project section....

[timgriffiths]

+1 be keen to see this feature

[deizel]

@timgriffiths, looks like this is possible in the latest RC by adding an environment file to your project:

• https://docs.docker.com/compose/env-file/
• https://github.com/docker/compose/releases/tag/1.7.0-rc1

[timgriffiths]

@deizel So i looked at the environment file, but i still think there is a good use case for being able to define project name within the compose file.

In our situation is we have Prod, Staging, UAT, Dev compose files just so we can start up different version of our application stack, and sometimes we want to stand up a Staging and UAT environment on the same swarm, so i can use environment variables or what not but that relies on the user all remembering to set this, where as if it was in the compose file everything would be checked into the repo and we could keep everything inline and simple to explain across all the users.

[edevenport]

+1 We have a situation where our compose file has references to a preconfigured remote volumes. Compose automatically adds the project name to those volume names which then fail to mount due to mismatched names. Being able to explicitly set the project name in the compose yml file would make the naming convention much more obvious to others involved in the project rather than hiding environment variables in obscure external files that might vary across environments.

[fesor]

@edevenport in your case you could use external option of named volumes:

# docker-compose.prod.yml
volumes
  dbdata:
    external:
      name: my-project-db-data

[edevenport]

Thank you @fesor - I should have gone into more detail. I am actually mounting glusterfs volumes using a configuration similar to this:

    ...
    volumes:
      - media:/data/media:ro

volumes:
  media:
    driver: glusterfs

In this case, media becomes projectname_media on the host and fails to mount unless the glusterfs volume is created ahead of time with the project name. I had thought mounting the glusterfs volumes on the host manually and using external in docker-compose file, but I would rather not have to do that manually across all nodes ahead of time.

[fesor]

@edevenport I understand that. But you could just add this volume manually with docker volume create and use it:

    volumes:
      - media:/data/media:ro

volumes:
  media:
    external:
      name: my-glusterfs-media

So there is no need for changing project, and get rid of any prefixes.

@timgriffiths's case is something more complex, and it doesn't have any workarounds. I use simple wrapper around docker-compose just to not remembering project names.

[timgriffiths]

@fesor so we have worked around the problem by just sticking the compose files in descriptive folder names, this gets around this limitation. But would be very nice to see it added in later releases

[fesor]

@timgriffiths I made a PR (https://github.com/docker/compose/pull/3118) - maybe this would help. But it doesn't handle your case if you have multiple compose files.

[timgriffiths]

@fesor that PR would be perfect is it a possibility to get it merged?

[mkuzmin]

Now Compose provides Environment file, so the project name can be persisted there.

[fesor]

@mkuzmin it's sad that there is COMPOSE_FILE but there is no COMPOSE_OVERRIDE_FILE.

[schmunk42]

@fesor AFAIK you can specify multiple files

[fesor]

@schmunk42 I could specify project name either. But this doesn't solve problem. I want to have in deploy script something like this:

docker-compose up -d

instead of

docker-compose -f $COMPOSE_FILE -f $COMPOSE_OVERRIDE_FILE \
            up -d

COMPOSE_PROJECT_NAME in example above is specified by CI. And feature like .env file is cool but... useless in case of project name persistence.

I use .env to DRY env variables (and for docker-compose < 1.7 I have simple wrapper around), but it doesn't solve any other issues. @timgriffiths already pointed on case with deployment on swarm cluster.

[dnephin]

COMPOSE_FILE=one.yml:two.yml is how to set multiple files. So just include the override file in $COMPOSE_FILE

[fesor]

@dnephin missed that feature, cool.

Well... about swarm deployment, I already handling that via COMPOSE_PROJECT_NAME in my jenkin's job ENV variables so problems only with manual deployment. And my implementation doesn't allows default project name to be overridden so...

[mikehaertl]

This issue is almost 2 years old and still unresolved. I wonder what exactly is hindering the docker team to finally add a proper fix. Is it really so hard to add a simple new configuration directive for docker-compose.yml files?

While the workaround with .env is nice it's not alway usable (the .env file may already be used by your application). And it's also still a workaround.

And I've even seen other new issues which may be related to the project name creep into latest versions (#3966).

So what is the problem really?

[aanand]

Is it really so hard to add a simple new configuration directive for docker-compose.yml files?

No! The issue has never been difficulty of implementation.

While the workaround with .env is nice it's not alway usable (the .env file may already be used by your application). And it's also still a workaround.

It's not a workaround, it's a solution! The .env file contains environment variables that are to be read by Compose. If you have environment variables that are to be read by your application, put them in a different file (say app.env) and reference it using the env_file option.

So what is the problem really?

Adding the project name to a docker-compose.yml file makes it less portable, which is not something we want to encourage. Now that there's a way (with .env) to persistently specify the project name without sacrificing the portability of docker-compose.yml, the case for adding it in as a configuration option is weaker. That's the primary reason this issue hasn't moved.

[mikehaertl]

Adding the project name to a docker-compose.yml file makes it less portable,

Couldn't this be optional? Personally I never check in my docker-compose.yml but only supply a docker-compose-example.yml in my repo which I tweak locally. E.g. during development I may add some extra env vars or map additional volumes into my container. Why not also give us the option to specify the project name?

Following your argumentation, you should also move all the network options to .env as they usually are not portable at all and depend on the network setup of your host machine.

Put another way: What makes the project name so special? There already are many other options in the docker-compose.yml that are not really portable. Why not give the developer the option to choose whatever is best for his use case?

[axsuul]

I also think the option should be made available. As Docker use becomes more wide-spread, projects with only one developer using containerization within their workflow will become more and more common. In those use cases, the project name being constant is fine. A good example of another project allowing flexibility like this is vagrant with their Vagrantfile. They realize that there are certainly use cases for large teams but then also recognize the lone wolf developer scenarios.

I think implementing a project name option here is important for adoption. It could throw off solo developers that don't care about a "scalable project name configuration strategy".

[simonvanderveldt]

Adding the project name to a docker-compose.yml file makes it less portable, which is not something we want to encourage. Now that there's a way (with .env) to persistently specify the project name without sacrificing the portability of docker-compose.yml, the case for adding it in as a configuration option is weaker. That's the primary reason this issue hasn't moved.

How does adding the project name to the docker-compose.yml make it less portable?

IMHO it's actually the contrary, as shown by #3966 created by @mikehaertl. This issue clearly shows that not having the project name stored in docker-compose.yml makes things actually less portable (as in: having a bigger probability to clash with other compose projects started from the same machine) since Compose relies on a convention about the container folder name that many people, at least initially, won't know.

Adding adding an .env file not only adds an additional thing to learn for new people that want to adopt Compose but since I'd also have to add it to my repo next to docker-compose.yml to prevent these collisions I don't see how that would be any different portability wise than just defining the project name in docker-compose.yml.

[fiveanddone]

I'm also for adding the name into docker-compose.yml. The .env file is used in other frameworks I use and it's important that remains off the repo. However, for maintaining localized containers across my developers, the less they know about docker ( at first ) the better. I want a simple entry point that won't collide with other projects. I don't want to explain why things are the way they are. Just a simple

clone repo
docker-compose up

My workaround now is still to create the file that @schmunk42 proposed.

[joequery]

The container itself shouldn't care about the project name. Why use environment variables as a mechanism only intended for the host to care about?

[thasmo]

I'd love to see support for this in the docker-compose.yml file. If one needs to set it via environment variable, it can be defined within the .env file (if not set directly on the host) and used via variable substitution in the docker-compose.yml file.

[hsimilli]

If named containers are allowed why would named projects not be allowed?

The use case where a project name definition in the .yml file would be handy is the following:
We have the same docker-compose file for different webservers (different database credentials, different data-volumes, some small differences like logo, etc). There are 3 services per application running, and they are configured in a compose file. When multiple servers are run next to each other, it would be nice to see which service belongs to which project.

Variable substitution in the container name comes in handy, with the $COMPOSE_PROJECT_NAME as prefix. Well I understand the .env-file solution, but this makes it not more "developer-friendly" in a (CI) deployment environment.

[RobIsHere]

Having one docker-compose kill containers of a totally different docker-compose because there was no -p or environment variable set and you created the docker-compose in a "docker" subfolder is a critical bug in my opinion.

There is simply too much room for error in serious deployments. Just forget to specify the project name, whether in -p, .env or elsewhere: you go offline for some time until you notice what happened. Even worse: the colliding service goes offline, not the one you are working on. You will have a happy day :(

[mikehaertl]

@dnephin What exactly is holding up the docker team to finally implement this? How many more complaining users are required? This is such a trivial fix that it's hard to believe, that still nothing has happened.

[schmunk42]

Maybe I am the only one here who has some concerns with this, but anyway.
In our setup, we modify only .env and app.env files for switching environments, but this relies on having multiple files and merging yml files.

If this gets implemented, please think about BC.

Eg. if a project name is defined in docker-compose.yml and .env the latter should take precedence. Otherwise people who rely on managing this via .env, get simliar problems with overlapping stacks, like with the current workflow and relying on folder names.

[mikehaertl]

@schmunk42 We're not saying, that the .env feature should be removed. If you prefer to have your project name in .env then simply don't configure it in docker-compose.yml.

But others prefer to have it in their docker-compose.yml as the discussion here clearly shows. They should be able to do so, if they want. It's an optional feature. What takes precedence if both are configured is a matter of defining a simple convention.

[davidjeddy]

Srsly, please add named projects. Use case:
Project one:
docker-compose up --build --remove-orphans

• nginx
• static content
  Project two:
  docker-compose up --build --remove-orphans
• nginx
• static content

When project two starts it kills project one's containers with a exit 137 (code 128 + level 9).

Now I have to run docker system prune and rebuild networks every day to keep myself from having dozens of dead containers.

[dnephin]

export COMPOSE_PROJECT_NAME=somethingnew

[mikehaertl]

Can someone from the core team maybe finally explain, what is holding up this feature? It's so frustrating, how things that are so easy to fix are blocked for no good reason.

The only argument so far was to keep the docker-compose.yml portable. This makes no sense, because

• project name in the composer file does not automatically make it not portable anymore
• not everyone even has this requirement
  
  
  
  • we already have other options that can limit portability in the same way: networks, port, ...
  
  

It all very much depends on the specific use case. But just because some don't want this option shouldn't mean that everyone has to follow their opinion!

[fesor]

@mikehaertl You already have persistent project names. Just put .env file and set COMPOSE_PROJECT_NAME variable there. I don't see any benefit in project name in compose file anymore.

[thasmo]

@fesor I generally dont' have .env files under version-control because they contain machine/environment-specific settings - that's also why I'd like to see project names being configruable in the docker-compose.yml file.

[nhooey]

@fesor: I think @thasmo is correct on this. The project name should have the option of being specified in the compose.yml file because if it's relevant to all developers using the project, it should be in version control.

[mikehaertl]

@fesor I'm sorry, but I don't think that personal preference is a real argument. The real question is not, why we don't want to use .env or environment variable. It's the other way round: Why was it not put in the config file where it belongs?

Almost all options that you can pass to docker on the command line can be specified in that file. Only the project name is for some mysterious reasons excluded. Is this a form of apartheid among arguments? :P I claim equal rights for all of them! Let the developer decide and don't put artificial constraints on them.

And again: No one is taking away the old option from you - we only finally want this extra option. That's only fair.

[nhooey]

@dnephin: It seems as though you and everyone else is happy with the solution stated in cr7pt0gr4ph7's refinement of this issue.

Would you be willing to accept a pull request that implements it, or do you intend to have one of the maintainers write it?

@cr7pt0gr4ph7

[davidjeddy]

@dnephin @fesor having the project name in the *compose.yml facilitates a unified configuration schema

[dnephin]

It's not clear to me why the project name matters. No part of the config file should rely on any specific project name.

The only important quality of the project name is that it doesn't conflict with the name of any other project, which is actually an argument against putting it in the docker-compose.yml file. A developer on a project will often use a different directory name for each project, so the default is often correct for most use cases.

When the default isn't correct there is a way for the developer to override it (-p, or COMPOSE_PROJECT_NAME), which can either be aliases or put into the .env file.

I'm not against a project file that would define the project name, I'd just like to understand why this feature is so critical for some. If it's because the Compose file requires a specific project name, I see that as a separate problem that should be addressed differently,

[RobIsHere]

I wanted to have 4 compose files in 1 directory. The directory name is wrong by default here. The only option I have is .env that only works for 1 compose file in 1 directory.

_Why I rule out COMPOSE_PROJECT_NAME and -p:_
The COMPOSE_PROJECT_NAME and the -p are in my opinion not save in production as you have to remember to set them. Or create a startup script and remember to not use docker compose up directly. When Person A relies on this mechanism and Person B doesn't know it, that is a certain failure.
(I already detailed it above)

[fiveanddone]

@dnephin For me and my team, application logic is usually stored in an htdocs folder in our project folder. This is allows us to keep other related documents/resources together in one directory. As a dev ops working with remote developers. It's easy for me to have them adopt Docker if I don't have to assume anything about their folder structure. I've stated this in the past, .env is not an option for me because it's typically not part of the repo we all share.

[mikehaertl]

It's not clear to me why the project name matters. No part of the config file should rely on any specific project name.

It happened more than once to me, that I did a 'docker-compose down' on a host - and had a different container go down than the one I wanted! Just because I didn't remember, that there was also configuration in a .env file.

The only important quality of the project name is that it doesn't conflict with the name of any other project, which is actually an argument against putting it in the docker-compose.yml file. A developer on a project will often use a different directory name for each project, so the default is often correct for most use cases.

Maybe that's true for you - it is not for me and for many others. My app structure is like myapp/app/docker-compose.yml. So all my apps share the directory name app. And I have more than 1 container on a host.

When the default isn't correct there is a way for the developer to override it (-p, or COMPOSE_PROJECT_NAME), which can either be aliases or put into the .env file.

Seriously, I start to feel like I'm in a Kafka novel here. Isn't it obvious, that you can put basically all the command line options for docker into a docker-compose.yml - except for this one big exception?

Instead of asking us over and over and ignoring our answers, why can't someone finally justify the resistance. And no: " ... but I don't need it" is not a valid argument!

[kachkaev]

@dnephin I have multiple projects in their own git repos and in order to keep the root project directories clean, I put all docker-related stuff to a subfolder called docker (build context just becomes .., all works beautifully).

To run multiple projects on a single server, I currently have to create docker/.env.dist in each repo and cp docker/.env.dist docker/.env after git clone. Otherwise, project name is just docker, which I obviously don't want. In a few cases .env contains passwords etc, so copying .env.dist is required anyway, but in most of the cases .env exists simply because there is no way to define COMPOSE_PROJECT_NAME in docker-compose.yml.

I understand that there may be glitches in how the containers are started and stopped if the same COMPOSE_PROJECT_NAME is used twice inside docker-compose.yml, but that's already happening if I forget to copy .env.dist or when I accidentally assign the same name in two different .env files on the same server. For me it would be ideal if I could define default_compose_project_name right in docker-compose.yml and then override the value in .env if, let's say I want to run a staging copy of some project. This would be 100% BC, but more convenient.

[schmunk42]

I've stated this in the past, .env is not an option for me because it's typically not part of the repo we all share.

Maybe, that's the root-cause of it all, I said before that docker-compose "hijacked" this file and we were forced to rename our stuff to app.env.

Maybe docker-compose.env would have been the right choice.

FWIW: We're only changing .env files in production, the yml files are merged with a docker-compose.override.yml if needed.

A workaround is also to define your yml files in a way, that they do not start without a .env file, eg. using a variable image: $IMAGE.

[RobIsHere]

This is a workaround for my 4 docker- compose files in 1 directory case above @schmunk42 ? Really?

[schmunk42]

@RobIsHere But you need to use -f anyway, right?

[nhooey]

In reply to @dnephin in this comment:

It's not clear to me why the project name matters.

It matters because it affects the name of the Docker containers that docker-compose manages. If you forget to set the project name, docker-compose ps won't find the containers that you've created before with docker-compose --project-name <project_name> up -d <container_name>.

Also, when you run the global command docker ps, the project name will be included in the list of containers that are running, so you know where they came from. For example, if you are testing several code projects at once that all use MySQL, and they each have a mysql container, then docker ps will show an ambiguous list of containers. So, the project name is important in the context of many Docker projects running on the same machine.

No part of the config file should rely on any specific project name.

Having to set the project name in a different place than _every other variable associated with the Docker Compose project_ doesn't make sense.

The only important quality of the project name is that it doesn't conflict with the name of any other project, which is actually an argument against putting it in the docker-compose.yml file.

For reasons stated in my first paragraph, this isn't the _"only important quality of the project name"_. It's for ease-of-use and understanding.

A developer on a project will often use a different directory name for each project, so the default is often correct for most use cases.

In the non-default case where a developer puts _all_ related Docker files in the docker directory, which is a perfectly reasonable way to organize a Docker project, the magic that sets the project name will create conflicting project names. So the same conflicts that you mention occur in this particular case. You don't need to protect the developer against project-name collisions when they are explicitly setting the project name anyway.

When the default isn't correct there is a way for the developer to override it (-p, or COMPOSE_PROJECT_NAME), which can either be aliases or put into the .env file.

Setting environment variables is an extra level of complexity that isn't necessary. When you're sharing a project between developers with version control, it's makes sense to use only files. Having to include command-line params with each invocation is tedious and error-prone. And the .env file can't be shared in version control because it's supposed to contain _user-specific_ variables, not _project-specific_ ones. Therefore, the current ways to set a project name are insufficient.

I'm not against a project file that would define the project name, I'd just like to understand why this feature is so critical for some. If it's because the Compose file requires a specific project name, I see that as a separate problem that should be addressed differently,

All variables that affect the functioning of docker-compose should go in the same place, the docker-compose.yml file. The current ways of setting a project name introduce unnecessary complexity.

Just about everyone subscribed to this issue agrees with these points.

Adding the ability to set the project name in docker-config.yml won't even conflict with any current functionality. There isn't a reason not to implement it.

[fiveanddone]

@schmunk42 Last time I checked, docker didn't allow us to pick the name of our .env file. Did I miss something ( it's hard to keep up with the changes ). Changing it to something like docker-compse.env would fix it for my use case. The problem I have is that I'm using two very popular technologies that both require .env file. PHP Laravel framework does not track that file in the repository and in production environments it sometimes doesn't exist.

I found this to be a real stubbling block when first adopting Docker because like @RobIsHere illustrates. All of my project directories are in themyapp/htdocs/docker-compose.yml format. In early days adopting Docker I accidentally deleted other containers that I didn't intend to. It would have ben better if docker named it's project randomly rather then using the folder name in my case.

I have several other remote developers that are starting to adopt Docker. As a team, it's always easier for me to instruct these developers to always/only docker-compose up to get these applications running. The less these early adopters know about Docker the better. Once they see the power behind it, they'll pick up on their own.

[dnephin]

So to summarize, the use cases seem to be when the default isn't appropriate. There are already ways to override the default, but those could be error prone and not obvious to early adopters.

Cases where the default isn't appropriate are:

• multiple compose files in the same directory (already requires you to specify -f to run them)
• using the same directory name for all projects (I guess this also requires you to specify -f, possible workaround is to use a unique directory name).

[fiveanddone]

@dnephin I would add, one less possible stumbling block for early adopters of Docker.

[nhooey]

@dnephin You also had these concerns in this comment:

Originally I wanted the name in the docker-compose.yml, but after thinking about it more, I really like the idea of a separate file. A separate file give you the option to keep it out of version control if you want (for cases where you want each user to be able to have a different project name).

Setting the project-name resolution order of precedence like this will solve that problem (bigger numbers override smaller ones):

1. docker-compose.yml
2. The COMPOSE_PROJECT_NAME environment variable
3. The --project-name command-line option

[schmunk42]

Last time I checked, docker didn't allow us to pick the name of our .env file.

@fiveanddone AFAIK you can not do that, but it would also just move the problem, since you'd then have to know which ENV-file you'd need for the project.

In our case with phd5, we moved the (app)-environment to src/ and .env is solely the "control-environment-file", I tried to explain that here.
I don't like this and wouldn't have changed it on my own. I know that this might not be possible for a lot of projects.
But having a .env from your application on the same level as your docker-compose.yml becomes super-annoying, because a) you get variables in your "control-environment" which do not belong there and b) you have to forward these variables to your container, if you need them there and c) you can not override/change them anymore in your app.env file.

@dnephin What's about introducing docker-compose.env as the preferred env file and use .env as a fallback. You did the same with fig.yml once.

[dnephin]

I don't think that fixes the problem for everyone. The naming of the .env file feels like a different issue to me.

I think the crux of the problem is this:

The original design of docker-compose up was to be a single (short) command you could run to get services running. This is what developers expect from it. However that behaviour can't be achieved for some users (in these cases).

There are plenty of ways to work around the problem, but none that seem to work for everyone.

I think the order of precedence (from highest to lowest) needs to be something like this:

1. --project-name (always overrides)
2. COMPOSE_PROJECT_NAME environment variable
3. A default defined by the user, separate from any versioned file
4. A default defined by the project, that can be checked in
5. The directory name (the default when nothing is specified)

(3) could be achieved with a .docker/project-name file (as proposed in the OP)
(4) could be achieved with a field in the docker-compose.yml file

It's unfortunate that there needs to be so many different ways to set the project name.

[simonvanderveldt]

@dnephin regarding 3, letting a user set a default, that's what env vars are for, no need to create a file

[kachkaev]

Hmm... And what if we generalise the problem a bit and simply introduce default_environment section in docker-compose.yml? This way we will be able to set COMPOSE_PROJECT_NAME without introducing a special field in the yaml, but will be also able to define other default variables, which are likely to be present elsewhere throughout the file. This will reduce the number of cases when .env.dist needs to be copied to .env and the cost of forgetting to do so will be less.

Usage example:

# ~/projects/sketches/sketch-42/docker/docker-compose.yml
version: "2"
default_environment:
  - SUBDOMAIN=sketch-42
  - ROOT_HOST=example.com
  - COMPOSE_PROJECT_NAME=sketch-42
services:
  web:
    build:
      context: ../
      dockerfile: docker/Dockerfile
    environment:
      - VIRTUAL_HOST=${SUBDOMAIN}.${ROOT_HOST},www.${SUBDOMAIN}.${ROOT_HOST}
      - VIRTUAL_PORT=80
      - VIRTUAL_NETWORK=proxy
      - LETSENCRYPT_HOST=${SUBDOMAIN}.${ROOT_HOST},www.${SUBDOMAIN}.${ROOT_HOST}
      - LETSENCRYPT_EMAIL=admin@${ROOT_HOST}
    restart: always
    networks:
      - proxy

networks:
  proxy:
    external:
      name: proxy

This yaml is very similar to what I have very often - it describes a vis sketch, which always has a service called web, but can also be backed up by a mini API server and maybe a container with a database. It is assumed that https://github.com/jwilder/nginx-proxy sits at the front of everything and listens to real ports 80 and 443.

At the moment, no matter if I'm on a local machine or on a remote server, I need to remember to maintain .env and also to make sure that _all_ environment variables are set there. If, let's say, I have forgotten SUBDOMAIN, the container starts, but the web server remains broken.

With default_environment section at my disposal, I could have all my production variables in place in one file and I would not have to copy .env.dist when on the server. On the local machine, in the above example I'd just put a single variable into .env, which would be ROOT_HOST=example.com.dev (or maybe I could even export ROOT_HOST=example.com.dev in bash profile?)

To summarise, default_environment section in docker-compose.yml can not only solve the problem we are currently discussing, but can also enable a few extra nice tricks, while being 100% BC and easy to understand!

WDYT?

[gergelypolonkai]

That sounds nice, it may come in handy for me in some scenarios.

However, be aware that in light of the above (heated) discussion, your
solution roughly translates to this: “Instead of adding a new file to the
yml, how about we add a new section?” :)

On Tue, Feb 28, 2017, 00:02 Alexander Kachkaev [e-mail]
wrote:

Hmm... And what if we generalise the problem a bit and simply introduce
default_env section in docker-compose.yml. This way we define
COMPOSE_PROJECT_NAME without introducing a special field in the yaml, but
will be also able to define other default variables, which are likely to be
present elsewhere throughout the file. This will reduce the number of case
when .env.dist needs to be copied to .env and the cost of forgetting to
do so will be less.

Usage example:

~/projects/sketches/sketch-42/docker/docker-compose.yml

version: "2"
default_env:
SUBDOMAIN=sketch-42
ROOT_HOST=example.com
COMPOSE_PROJECT_NAME=sketch-42
services:
web:
build:
context: ../
dockerfile: docker/Dockerfile
environment:
- VIRTUAL_HOST=${SUBDOMAIN}.${ROOT_HOST},www.${SUBDOMAIN}.${ROOT_HOST}
- VIRTUAL_PORT=80
- VIRTUAL_NETWORK=proxy
- LETSENCRYPT_HOST=${SUBDOMAIN}.${ROOT_HOST},www.${SUBDOMAIN}.${ROOT_HOST}
- LETSENCRYPT_EMAIL=admin@${ROOT_HOST}
restart: always
networks:
- proxy

networks:
proxy:
external:
name: proxy

This yaml is very similar to what I have very often - it describes a vis
sketch, which always has a service called web, but can also be backed up
by a mini API server and maybe a container with a database. It is assumed
that https://github.com/jwilder/nginx-proxy sits at the front and listens
to real ports 80 and 443.

At the moment, no matter if I'm on a local machine or on a remote server,
I need to remember to maintain .env and also to make sure that all
environment variables are set there. If, let's say, I have forgotten
SUBDOMAIN, the container starts, but the web server remains broken.

With default_env section at my disposal, I could have all my production
variables in place and I would not have to copy .env.dist on the server.
On the local machine I'd just put one variable into .env, which would be
ROOT_HOST=example.com.dev (or maybe I could even export
ROOT_HOST=example.com.dev in bash profile?)

To summarise, default_env section in docker-compose.yml can not only
solve the problem we discuss now, but can also enable a few more nice usage
scenarios!

WDYT?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/745#issuecomment-282885661, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAQJJZumr10j3i17gPxrSyA-n8CwvsXTks5rg1X_gaJpZM4DLBNs
.

[oliviermadre]

Having the "project_name" inside the docker-compose.yml would solve an issue with our stack.
We have a monolith repo with a standardized tree handling multiple projects, which looks like more or less like this :

projectA/infrastructure/docker-compose.yml
projectB/infrastructure/docker-compose.yml
...

It's obviously more complex than that in the real world, so we can't just move the docker-compose.yml file to the "projectA/projectB" folder.

We have a CLI at the root that ask which projects to boot, and since the project name depends directly on the parent folder of the docker_compose.yml, we're facing conflicts.

As we don't write ourself the docker-compose command (wrapped inside our CLI script), we can't just add the "-p" arg to the command manually.
A solution for us would be to always generate a project_name inside the CLI script so we always have a "-p" when calling docker-compose, based on the absolute_path, but it sounds weird.

Some guy here said "it works most of the time 'cuz on most projects, the docker-compose.yml is located inside the root folder of the project, and projects have different name, avoiding conflicts", I just can't agree on this. It's not always inside the root folder.

Why is this possible to define the "container_name" in a docker-compose.yml file (disabling scaling) for a service, but can't define a "project_name" for the docker-compose.yml file itself, at the same level as "version" / "services" / "networks" / "volumes" ?

[shaunwarman]

This seems necessary with the use of scale so that container names are obvious. (e.g. <project_name>_<container_name>_1).

This will help when using the container name for dns.

[jbogatay]

Came here hoping this was done. Left disappointed. 3 years+

[nhooey]

Yeah it's one of the most stupid usability problems of Docker and it goes unfixed.

We spent so much time arguing with the developers about it, and even though there was consensus, there was no admission that it's a bad design.

[jbogatay]

Its frustrating because there is obvious user support for this, but since it does not fit with the developers own use-cases --we are left rehashing this year after year.

[src386]

What's the point of docker-compose if we need to add options using cli ?
I had some containers overwritten thanks to the naming scheme.

[jonaskello]

Today I was setting up yet another docker-compose project with yet another .env file to give the project a persistent name. So I thought I check into this issue since it been some years since I checked last. But nothing new here as far as I can see? I guess I'll have to keep using the .env workaround. To me, naming a project in the yaml file seems like the most basic thing you could do, but I guess there is something about this I simply don't understand since it has not been solved yet.

[DavidAntliff]

Having to specify the project name externally (by command line or variable) but being unable to set it in the docker-compose.yml file - the file that specifies absolutely everything else - is just mindless. There are plenty of cases where a process might need to address a specific stack of services based on an arbitrary docker-compose.yml file regardless of parent directory name, shell environment or presence of a specific line in a specific .env file. I don't want my CI jobs to have to continually "figure out" what the project name should be - it should be clearly stated in the docker-compose.yml so my jobs can extract it and use it.

This seems to happen time and time again with this project. Docker is a wonderful technology but sometimes I wonder if the developers actually have any real-world experience, or are they all just 18 year old hackers who think they know best? People want to use this stuff for real work, not spend their time constantly looking for workarounds.

[mikehaertl]

My guess is that the team is meanwhile ignoring this issue. Trying to convince docker developers can be a real pain to put it mildly.

Clear arguments are countered with some weird logic that doesn't make much sense. The opinion of real world users that all come here for the same reason is ignored. It's a shame.

[schmunk42]

I don't want my CI jobs to have to continually "figure out" what the project name should be - it should be clearly stated in the docker-compose.yml so my jobs can extract it and use it.

Do you have the issue that your stack is named like "build1234" and/or how would like to you use the name of the stack in another place, like docker exec build1234_myservice script.sh?

---

Might not fit any use-case, but for having it said ...

In our setups we only change .env files when setting up a (clone) project. If we need changes or a dynamic configuration, we re-use variables from .env in docker-compose.yml. An advantage is that docker-compose warns you (or fails) if a variable is missing.

I don't want to tell you you have to change your workflows, because I was also annoyed by this issue.
It's basically something like, we treat changes to docker-compose.yml more like changes to source-code, while .env is solely configuration. But I also agree that there are several options in docker-compose.yml, like network which are highly project depended. Coming back to the above; I'd define a .env variable for the network. 🤐

[DavidAntliff]

I don't want to tell you you have to change your workflows, because I was also annoyed by this issue.

I thought about my own situation some more. I don't actually have a problem with externally setting the project name, and in fact I do this with the -p <project_name> flag, which makes it possible to isolate unique instances of the stack in CI and run multiple stacks concurrently if necessary. Only that CI chain needs to know that project name so it can be auto-generated or random (but known), no problem. I also don't have a problem with using an environment variable to override the default name, however I do find the .env trick non-obvious.

However there are other situations where it's useful to have the _default name_ of the project explicitly defined somewhere. So why does it take the parent directory name as the source of this name? That, to me, is the weak point here. It could be taken from anywhere - the choice of a parent directory just seems arbitrary and without much reason (e.g. in almost all my projects, the docker-compose.yml file sits in a subdirectory called docker). So if it's going to be arbitrary, store it somewhere that's more explicit, like the yml file, rather than creating external side-effects by influencing parent directory names which in many cases should be completely independent of the content of the directory (based on years of experience working with reusable resources).

[schmunk42]

It could be taken from anywhere - the choice of a parent directory just seems arbitrary and without much reason (e.g. in almost all my projects, the docker-compose.yml file sits in a subdirectory called docker).

If you need something else than a random ID, then I see no real alternative to the parent directory. At least you've the chance of knowing where your stack "came from".
But I also agree with you, since in our projects we've test-stacks in tests folders which - without an .env file - would also collide badly with each other. That's why I proposed to prefer a file called docker-compose.env which would be much clearer IMHO.

[matsaman]

Would you guys either just close this bug admitting you don't care, or implement the incredibly simple thing the vast majority of us want? It's 2.86 years later. Get off the pot. Own your decisions or fix your mistakes. However you want to look at it. Do more than nothing.

[benjaminwood]

@matsaman and all of the people giving his comment the thumbs-up:

Would you guys either just close this bug admitting you don't care, or implement the incredibly simple thing the vast majority of us want? It's 2.86 years later. Get off the pot. Own your decisions or fix your mistakes. However you want to look at it. Do more than nothing.

Who are "you guys"? In open source, the community owns the decisions and fixes the mistakes of others. Docker is open source. Consider submitting a pull request instead of whining.

I too am sad that this has not been implemented yet. But, in open source we either contribute, or wait patiently.

[colinmollenhour]

CLI arg:

$ cd foo
$ docker-compose up
$ docker-compose -p bar up
... some time later wanting to take down bar forgetting '-p'
$ docker-compose down
Stopping foo_nginx_1 ... done
Stopping foo_mysql_1 ... done
Removing foo_nginx_1 ... done
Removing foo_mysql_1 ... done
$ FU@$_!@*#%$(!_*@
-bash: FU@!@*#%$: command not found

Fail.

Env file:

$ cd foo
$ source foo.env
$ docker-compose up
$ source bar.env
$ docker-compose up
... some time later wanting to take down foo forgetting to `source .foo.env`
$ docker-compose down
Stopping bar_nginx_1 ... done
Stopping bar_mysql_1 ... done
Removing bar_nginx_1 ... done
Removing bar_mysql_1 ... done
$ FU@$_!@*#%$(!_*@
-bash: FU@!@*#%$: command not found

Fail.

Proposed solutions with project name in yml file:

$ cd foo
$ docker-compose -f foo.yml up
$ docker-compose -f bar.yml up
... some time later
$ docker-compose down
ERROR:
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml

Yay!

[schmunk42]

$ COMPOSE_PROJECT_NAME=foo docker-compose up -d
$ COMPOSE_PROJECT_NAME=bar docker-compose up -d
... some time later
$ docker-compose down -v
Removing network project_default
WARNING: Network project_default not found.

o/

[mikehaertl]

Who are "you guys"?

@benjaminwood It's the maintainers of this project. Your argument would make sense if we were talking about a complex change and no one would find the time to do it.

This is quite obvious not the case here: The implementation would probably be very easy for those familiar with the codebase and much harder for someone from outside. So it's not that no one has time to do it but rather that the maintainers don't want to do it. And this is what the supporters here find extremely annyoing, given the long time that this issue is open now and the mass of :+1: here.

The workarounds posted again and again here are all well known but not applicable to everyone. And I still couldn't find an answer why of all things almost only the project name was left out in the features of docker-compose.yml. If someone thinks, it doesn't belong there then just don't use it! But don't obstrain your opinion on others if the demand for it is so overwhelmingly obvious.

[schmunk42]

Let's view this from other point.

If I am in a folder with a docker-compose.yml file, how do I find out which Project Name my stack has?
That's a fairly basic question, but there's no easy answer to it.

Currently it's (sorted by priority):

• the value of -p
• the value of COMPOSE_PROJECT_NAME from your environment
• the value of COMPOSE_PROJECT_NAME from your .env file
• the current directory name

To be the devil's advocate, why adding a 5th one to this already confusing setup?

Despite the decision of that, there should be a "dry-runnable" info about the current name. Neither docker-compose config has that, nor docker-compose ps if the stack is not running.
Maybe docker-compose create is the best currently available option, but far from perfect.

There should be at least something like docker-compose config --project-name. Currently I need to know the above locations and check them in the correct order.

[mikehaertl]

If I am in a folder with a docker-compose.yml file, how do I find out which Project Name my stack has?

Yet again: If you don't want to use it don't use it! If you're happy with your solution - good for you!! Nothing will change for you, so you'll not be confused at all!

But can't you accept that other's have different requirements? All we want is that this logically missing piece is finally added.

To be the devil's advocate, why adding a 5th one to this already confusing setup?

The only reason why it may be confusing is because the project name was left out of docker-compose.yml in the first place. Otherwhise everything would be in that file now. We can define a list of precedence - not a problem.

Currently I need to know the above locations and check them in the correct order.

Why don't you know the docker name of your project and why is that even relevant to you? And why don't you use the same mechanism in all your projects if it confuses you so much?

[schmunk42]

Yet again: If you don't want to use it don't use it! If you're happy with your solution - good for you!! Nothing will change for you, so you'll not be confused at all!

My question was not about using a new feature, but the lack of an existing one. The project name is the one and only relevant information which docker-compose uses to start and stop a certain set of containers.
People in this thread complain about killing the wrong containers, because the project name is not part of the stack.

But can't you accept that other's have different requirements? All we want is that this logically missing piece is finally added.

If you add this people will complain about killing the wrong containers, because it is part of the stack, since it was using the directory name beforehand.

The only reason why it may be confusing is because the project name was left out of docker-compose.yml in the first place. Otherwhise everything would be in that file now. We can define a list of precedence - not a problem.

Actually you can not add this in a BC-way, since it would have to have a lower precedence as the directory name, which would make it useless. That's why I was asking "how to get the project-name".

Why don't you know the docker name of your project and why is that even relevant to you?

Because I usually use the directory name, but sometimes a custom value in .env - it's relevant because the project name defines the containers on which actions are run against.

cd /some/path/test
docker-compose up -d
cd /a-completely-different-path
docker-compose -p test down -v --remove-orphans

This will kill your first stack, would be the same with project name in the yml file.

And why don't you use the same mechanism in all your projects if it confuses you so much?

I do, I am just using the default docker-compose features; most of the time the directory name is fine, but in case I need to rename that, but want to continue working with the same set of containers I add a .env file.

[mikehaertl]

If you add this people will complain about killing the wrong containers, because it is part of the stack, since it was using the directory name beforehand.

That doesn't make sense. If someone doesn't add the project name to docker-compose.yml nothing will change. And if they add it then it's because they want it there! No one will add it unless they need it.

And even if they do, everything is fine. I don't get which problem you try to construct here. You're overcomplicating things.

If you mix all the options where a project name could be configured, then I'd say this is your problem. You shouldn't do that at all.

[YoJeff]

3 of the 4 options in the priority tree are worthless once you have multiple named compose files in a directory. Don't tell me to make subdirectories, because I still share an .env file across them.

[schmunk42]

Don't tell me to make subdirectories, because I still share an .env file across them.

Do you share an application specific .env file used in env_file?
Or the one docker-compose uses?
Or do you mix them?

That doesn't make sense. If someone doesn't add the project name to docker-compose.yml nothing will change.

That's not the problem, but when it would be added the behavior drastically changes.

When we copy a stack to a new directory for spawning a temporary instance (ie. for testing an upgrade), we don't have to touch docker-compose.yml at all. But if this is part of the stack we'd need to decide whether we override it with .env or whether we change it in docker-compose.yml.

Yeah, it might be that it should not have been added in the first place, but having another option here also does not make things easier and more concise.

[YoJeff]

I have a common single .env file that 4 named compose files (in the same directory) use. No environment variables. No shell scripts. I use just compose.

I would prefer this

docker-compose -f service1.yml up -d

Instead it's more like this

docker-compose -f service1.yml up -d
# F#&$, forgot the -p flag.   Curse the compose devs for 3 years of c#*$blocking
docker-compose -f service1.yml down
docker-compose -f service1.yml -p service1 up -d

[colinmollenhour]

@schmunk42 you seriously believe that is a solution to type 'COMPOSE_PROJECT_NAME=...' each time for every docker-compose command, possibly in addition to the compose filename? That also requires you to know and remember the project name. Being able to specify the project name in an environment variable has it's uses for sure but in some cases you just want to have a directory with a couple yml files in it and be able to easily manage projects in a fool-proof way without having to remember the project names you chose months ago and without having to remember to export environment variables.

[schmunk42]

Can't you use one folder per service and still have a .env file in the top folder?

docker-compose -f service1/docker-compose.yml up -d
docker-compose -f service2/docker-compose.yml up -d

Just write the project-name into the file(-path) 😉

[YoJeff]

I give up. It's obvious the customer-base for this has changed.

[schmunk42]

Let me know what's wrong with the proposal I made to you.

I had the same concerns in the beginning, but we've adjusted our workflows to the features available.

I can't remember accidentally killing a stack in the past 2 years, by no-one of our team. We're running >200 stacks with around 1.000 containers and 1.000s of automatic or manual redeployments this way.

[nixpulvis]

I don't really want to get involved here... but I can't resist.

There is a very important difference between a "solution" given the way things are now, and the "right" solution given the possibilities that exist. What this issue should be focused on is finding the "right" solution, which as far as I'm concerned is pretty obvious, though some might still disagree.

[benjaminwood]

@schmunk42 & @mikehaertl

I can't remember accidentally killing a stack in the past 2 years, by no-one of our team.

I have to ask. I see that you're both part of the codemix org. Are you coworkers? Beginning to wonder if behind the scenes you two are rotfl.

[joshuajabbour]

Currently it's (sorted by priority):

1. the value of -p
2. the value of COMPOSE_PROJECT_NAME from your environment
3. the value of COMPOSE_PROJECT_NAME from your .env file
4. the current directory name

To be the devil's advocate, why adding a 5th one to this already confusing setup?

The problem as I see it is _every single value_ that is currently available for use is _environment-specific_. There is no way to provide a project-specific option.

1+2. Must be provided manually by the user invoking the command.

1. _Could_ be shared, but in practice .env should be ignored and not shared for good reason. (I work around this by sharing .env.example, but this still requires an extra manual step.)
2. Is effectively random, and is user/environment-specific. (I also view this as a fallback, because compose _needs_ a project name. It's not even a very good fallback, because there could be projects in different directories named the same that would conflict.)

In my opinion, and many of those in this thread, there should be an option that lives between 3 and 4, where a default project name _can_ be provided in whatever docker-compose file is being used for this project and is used as the first fallback. This value would then be shared by anyone deploying this project. Options 1-3 would override that value, and option 4 would only be used if it wasn't provided.

[schmunk42]

@benjaminwood I can assure you that no-one is laughing about this issue. Yes, we know each other for quite some time and it's not the first time we fundamentally disagree. I might sound annoying to several users here, but it's not my intention. In fact I'd like to share my experiences and propose solutions. I believe that I have a vast experience about this topic. If the feature would get introduced as proposed it would introduce a huge source of possible errors in our setup.

[schmunk42]

@joshuajabbour

1. Could be shared, but in practice .env should be ignored and not shared for good reason.

That depends on your setup, we have this ignored in all of our project repositories.
But we do not ignore it in our staging and production stack-only repositories. If you would share the name committed in docker-compose.yml you can also commit .env (this file is only for the docker-compose command - nothing else)

1. Is effectively random, and is user/environment-specific.

It's not more or less random than a value in docker-compose.yml, you need to view the stack configuration as a folder, not a single file. If you do that, you can either have the name in .env or a subfolder name.

---

I have one last alternative for you all:

docker stack deploy -c docker-compose.yml the-project-name

(*) Only available in swarm mode

I can't see how a project-name could be placed into the yml file in the future, it would totally contradict the idea of a stack definition.

[benjaminwood]

Thanks @schmunk42. I believe you and I think you've handled things quite well amongst all the strong opinions displayed in this thread.

[YoJeff]

it would introduce a huge source of possible errors in our setup.

There's the rub. Close the issue.

[joshuajabbour]

That depends on your setup, we have this ignored in all of our project repositories. But we do not ignore it in our staging and production stack-only repositories.

So this feature everyone else wants should not be introduced because it'd break your unusual repository architecture?

If you would share the name committed in docker-compose.yml you can also commit .env (this file is only for the docker-compose command - nothing else).

Well, if the only thing in my .env file was COMPOSE_PROJECT_NAME, then yes. But it's populated with many other _secure_ variables that I don't want committed. And I import them into docker-compose using the environment property. This is what .env files are generally for...

I still really do not understand how this would break your setup, as in my proposal it only overrides the default project name taken from the directory. And if you are depending on that (because it is a directory committed _into_ your repo as opposed to the directory your repo is cloned into), then how would it ever get overridden by the docker-compose.yml setting (because that is also committed to your repo; you control both things in your scenario)?

[mikehaertl]

@schmunk42 What you say is "I don't want new features, because otherwhise I don't understand my project setup anymore".

Seriously? This is your argument why you want everyone else who needs this to do without?

Just for the record: I'm out here, too. Feel free to close the issue. The discussion here has become pointless.

[schmunk42]

So this feature everyone else wants should not be introduced because it'd break your unusual repository architecture?

I am voicing my concerns that it might introduce additional sources of errors.

Well, if the only thing in my .env file was COMPOSE_PROJECT_NAME, then yes. But it's populated with many other secure variables that I don't want committed. And I import them into docker-compose using the environment property.

Use a secrets.env for those and import it via env_file.
How would that affect your workflow?

This is what .env files are generally for...

True ... but I sill think the problem is that docker-compose hijacks the .env file.

How would it be, if you can set these variables plus ones used in the top levels of docker-compose.yml, like IMAGE_VERSION, in a file called docker-compose.env?

I never dared to propose COMPOSE_COMMAND_ENV_FILENAME=.env - until now :)

I still really do not understand how this would break your setup,...

It true that it does not break immediately, but it's merely about my first point - and introducing even more options.

Think about using multiple files -f docker-compose.A.yml -f docker-compose.B.yml, let's say A is your project and relies on project-name by directory (we do that, since we control it!), while B is a set of extra services with project_name: extra, accidentally introduced while testing in a folder which had a .env file which overwrote the project-name with COMPOSE_PROJECT_NAME=testing.

But now any project, which is started without an .env file, would be named extra. 💥

We're merging files on over several levels, including multiple *.env files, it's a very valid use-case.

[schmunk42]

Seriously? This is your argument why you want everyone else who needs this to do without?

Warning: Slightly Off-Topic, but still docker related...

• "I don't think we really need it"
• "has more layers than neccessary. more complexity"

@mikehaertl I really wonder that you see it this way ;)

[shin-]

Hey folks,

We've picked up on the passionate discussion in this thread (thank you to those of you who kept it civil and cordial!) and while we still fundamentally believe that project name does not belong inside a Compose file, we did come up with what we believe to be a reasonable middle-ground with the following PR: #5378 . That said, we'd appreciate your feedback on this to make sure it answers your needs.

Here's the gist:

• You may add a x-project-name entry inside your Compose file. This key is ignored by default.
• When COMPOSE_X_PROJECT_NAME is set in your environment (or .env file), Compose will attempt to retrieve the project name from the x-project-name entry inside your Compose file.
• This option has a a lower priority than COMPOSE_PROJECT_NAME and --project-name

Happy to address any questions or concerns regarding this.

[schmunk42]

@shin- So COMPOSE_X_PROJECT_NAME would have to be set to 1 in order to activate the feature?

If so, COMPOSE_ENABLE_X_PROJECT_NAME might also be a naming to think about, but these are just my 2 cents - I won't use it anyway 😄

[shin-]

@schmunk42 Any "truth-y" value will work, but yes, that's the idea.

[shin-]

@matsaman Cool, thanks for the useful, actionable feedback.

[matsaman]

Instead of you telling us that we'll have to tell end users to set a variable or call a param, you're telling us that we'll have to tell end users to set a variable or call a param.

Honestly, & truly not for nothing, I can't imagine how you have considered this a solution in any way.

[nhooey]

@shin- I'll explain how this change doesn't make any useful difference:

Having to set an environment variable to enable looking for the project name in the compose file is roughly the equivalent {amount of work, risk of forgetting, and convenience}, as setting an environment variable for the project name itself.

The whole point about setting the project name in the compose file is to avoid having to use environment variables. You can use precedence to the project name environment variable to give users the option to override the project name in the compose file.

[schmunk42]

I think it is a very good solution, since it won't break BC and it will avoid introducing errors during merging like I described above.

[schmunk42]

Having to set an environment variable to enable looking for the project name in the compose file is roughly the equivalent {amount of work, risk of forgetting, and convenience}, as setting an environment variable for the project name itself.

It's not, you can do this setting once, globally in your environment - you do not need to do this in every project.

[nhooey]

It's not, you can do this setting once, globally in your environment - you do not need to do this in every project.

I'm assuming you mean globally in all of your shells that you spawn on your computer. If you do that, then you could get unwanted behaviour in completely unrelated projects. It will also cause confusion when a developer pulls a project from version control and forgets to set the environment variable.

One of the most important goals of having the project name in the compose file is to ensure every variable that has to do with the compose project gets stored in version control.

[shin-]

@nhooey @matsaman
It's specifically intended to help people who have multiple compose files in the same directory, for whom the .env file solution is not an option. If this needs to always be on for your projects, it's easy to set it in your .env file, your .profile, or anywhere else that will ensure x-project-name will always be taken into account.

Instead of you telling us that we'll have to tell end users to set a variable or call a param, you're telling us that we'll have to tell end users to set a variable or call a param.

Our main concern with this has always been that the "end-user" has projects of their own and want to avoid name conflicts at all costs. In that regard, they should be in control of the project name, not the distributor. If you're shipping "turnkey" projects to customers, it's similarly trivial to set COMPOSE_X_PROJECT_NAME in the associated .env file - so I'm confused what part of this is a concern to be honest.

[schmunk42]

If you do that, then you could get unwanted behaviour in completely unrelated projects.

Could you give an example? I can't think of any unwanted behavior just by setting COMPOSE_X_PROJECT_NAME in my shell.

One of the most important goals of having the project name in the compose file is to ensure every variable that has to do with the compose project gets stored in version control.

Why can't you use .env for that? Nobody forbids storing .env in version control (yes, it is not common).

[matsaman]

We had .env at the outset. You have clearly missed the entire conversation.

[shin-]

@matsaman You seriously need to tone down your attitude. You've been nasty since you started commenting in this thread and it's done nothing but make the conversation worse as a result. If you can't be civil when sharing your thoughts, we can do without your opinion.

[dnephin]

To clarify some of the confusion

You could do that already

Previously you could set a project name using an environment variable. The new environment variable is not a name, it's a toggle which enabled a feature which lets you set a project name in the compose file.

I'm assuming you mean globally in all of your shells that you spawn on your computer. If you do that, then you could get unwanted behaviour in completely unrelated projects.

If you mean "other projects" as-in, "something that is not compose could read this and break" I think that is not realistic. There are plenty of environment variables set in every shell. The environment variable is properly namespaced as "COMPOSE_X_".

If you mean "other compose projects" then I think you are actually making an argument for why this variable is necessary. If the feature were enabled by default then anyone relying on the current default behaviour would break.

If it's neither of those, please do clarify.

It will also cause confusion when a developer pulls a project from version control and forgets to set the environment variable.

If a project only works with a specific project name then I think there are other problems. There should never be a case where a project only works with a single name.

I see this as another argument for not putting the project name in the Compose file. If the project name is in the compose file then any two project could use the same name, causing a conflict, which would break both projects. The name should really be set by the developer who is aware of what other project names are already in use.

[fiveanddone]

Unfortunately this won't help for me either for reasons I've already mentioned over a year ago.

A solution that involves environmental variables would require users that need to know how to make them in the first place. I know that it may seem hard to believe, but not everyone works inside of the terminal. Again, putting a .env in the repository is not an option for me.

The beauty of docker-compose is up. Not instructions on how to setup that call.

[shin-]

@fiveanddone

Again, putting a .env in the repository is not an option for me.

Can you expand on that? I totally hear you on users with lower technical knowledge, but if they're not meant to interact with the code or environment, why is it an issue to include a .env file with the project?

[fiveanddone]

@shin- Yeah, no problem.

I have in some cases included that file in the repository but it doesn't work all the time. My use cases involve mostly web development where a developers technical knowledge can vary vastly.

The .env houses parameters that are used in other frameworks. Some of the variables inside my .env are credentials for outside services like SMTP. Most of the developers on my team point those services to their own endpoints using their own .env file. If the .env isn't there, I have defaults.

For less technical developers that maybe just want to change some CSS, the application works fine without the .env file. That is assuming they don't name every project folder the same. :)

It might not seem like much but when you're working with multiple developers and designers that span the country -- the simpler the better.

Docker has made my life so much easier and I'm super grateful for this project, but I can relate to the frustrations around this thread.

[shin-]

@fiveanddone Thank you for the insight! So if the .env file was called docker-compose.env (or similar) instead, would that alleviate your concerns?

[fiveanddone]

@shin-

Yes, it would for my use cases.

[nhooey]

Is the env file loaded automatically by Docker Compose?

If not, could docker-compose.env be loaded automatically?

[AyushyaChitransh]

@nhooey The env file is loaded automatically if it is named .env. And you can also specify env file for each service individually. See docs.

[schmunk42]

The env file is loaded automatically if it is named .env. And you can also specify env file for each service individually. See docs.

Not to be the nitpicker here, but that is not really correct. And IMHO the biggest source of misunderstanding of the whole topic.

None of the variables of .env is passed through to a container/service, if you do not explicitly forward it, by using either

env_file: 
  - .env

or

environment:
  - VAR_FROM_DOTENV
  - FOO=${ANOTHER_VAR_FROM_DOTENV}

The .env file is loaded automatically, yes, but without further configuration it only applies to these Compose CLI environment variables.

I'd strongly support renaming that to docker-compose.env by default and even better - a variable for that, so this could be used in a BC-way with just setting one ENV-var.

[mikehaertl]

(thank you to those of you who kept it civil and cordial!)

I hear you and I'm surely guilty of not always retaining my composure. Sorry for that. Some comments sometimes really get me started ... will work on that.

If the project name is in the compose file then any two project could use the same name, causing a conflict, which would break both projects.

Our main concern with this has always been that the "end-user" has projects of their own and want to avoid name conflicts at all costs. In that regard, they should be in control of the project name, not the distributor.

@shin- This sounds as it's commonly agreed, that docker-compose.yml is always part of a project and lives in the project repository. I'd say that's not neccessarily true. Some of us do not share docker-compose.yml but want to keep their local version with local customizations (e.g. development settings, tryouts, etc.).

And regarding "avoid name conflicts": Instead of a name conflict with the project name we now have a (IMO) way more critical name conflict on the filesystem level: .env is a common name that projects use to define their app specific settings. It should not be polluted with docker settings unless a developer really wants to do that. But this decision should be left to the developer.

Here's a suggestion: How about adding two other options for the project name and deprecating use of .env in V4 of the docker-compose.yml? One is project_name in docker-compose.yml, another is a file .dockerproject which only contains the name and never should be submitted to a repository. It should be a local file.

Here's my suggestion for precedence (highest first):

• -p CLI argument
• .dockerproject file
• project_name in docker-compose.yml (should be avoided, if docker-compose.yml is distributed with the project)
• COMPOSE_PROJECT_NAME from environment
• COMPOSE_PROJECT_NAME from .env (deprecated)
• directory name

This way end users could still always override a project name in .dockerproject even if someone hardcoded it in a distributed docker-compose.yml.

UPDATE: I could also live with using .dockerenv or docker.env instead of .dockerproject. But it should have higher precedence to solve the problem, that some here need to override hardcoded project names in docker-compose.yml.

[estarter]

Hi all,

First of all, thanks @dnephin and @shin- for looking at this case. The change @shin- has submitted looks like a great step towards resolving this issue.

I've validated the PR 5378 on the test project for the following use-case: A default project name defined by the project, that can be checked in (https://github.com/docker/compose/issues/745#issuecomment-282858900). For that I made a git repo illustrating different possibilities: https://github.com/estarter/compose_745

I found PR #5378 covering the use-case partially. The problem is that the solution requires COMPOSE_X_PROJECT_NAME env variable. Possible solutions:

1. persist the env variable in the project's folder. I tried .env file but it doesn't work when docker-compose is called from another location (see commands example , compose files)
2. define COMPOSE_X_PROJECT_NAME globally (.profile or equivalent). That would work but it can't be specified in the project, i.e. it's not a solution for our use-case defined by the project, that can be checked in
3. is there another option?

Hopefully I managed to illustrate the use case that is addressed in here.

Dear maintainers, please consider how the use-case is solved by PR #5369 which introduce optional project_name property in docker-compose file. This property takes low priority but does not depend on the environment or working directory.

[schmunk42]

1. is there another option?

There is this option docker-compose --project-directory <PATH> (Specify an alternate working directory). Should also work with a .env file.

[benjaminbradley]

To add another nuanced use case to the mix:
In my use case, I have two repositories, one for the main project (used to build/deploy) and one for devops (containing the Dockerfile's, docker-compose.yml, etc.)
In our current setup, the main project repo is at the root ./ and the devops repo is checked out to a subdirectory ./devops/

In this setup, using a .env file does not work because it must be :

placed in the folder where the docker-compose command is executed (current working directory).
(doc ref)

Which doesn't work, since it is invoked as: docker-compose -f devops/docker-compose.yml
Of course, we can add the -p parameter here, which is what we're using currently, but it's prone to human error as has been referenced above.

If the .env (or proposed docker-compose.env) file could be read from the same directory where the docker-compose.yml lives, then defining the project name there would work for me.

Using the env_file directive inside the docker-compose.yml doesn't work, because those env vars are only applied inside the container, not to the creation of the containers.

Ultimately, my goal is to keep my devops-related files self-contained in a way that doesn't pollute my actual project code base, but can still be accessed/run in parallel with that repo.

[estarter]

There is this option docker-compose --project-directory (Specify an alternate working directory). Should also work with a .env file.

@schmunk42 could be a good shot, but --project-directory doesn't work for .env file. I tried as following, .env file has been ignored (project files):

docker-compose -f PR_5378/docker-compose.yml -f PR_5378/docker-compose2.yml --project-directory PR_5378 down

[shin-]

Going forward, I'll be deleting comments that do not contribute to the conversation in a constructive way. I have no interest in interacting with children.

[shin-]

On having a settable .env name
The issue with having an environment variable designating which .env file to use is that it's essentially a circular dependency. We'd have to introduce an exception in the way environment variables are interpreted which, while sensible, is inherently counter-intuitive.

On the .env name itself
It's obvious by now that the .env name is used by a multitude of software and frameworks, and isn't usually meant to be committed in version control. We're very sensitive to making breaking changes, but we could introduce an alternative name (e.g. docker-compose.env) with a fallback to the .env file if the former doesn't exist, with the intent that it may be shared with other users.

On adding project_name to the Compose file
We've reiterated our stance on this several times already - see Daniel's latest comment for some of the reasons we're opposed to making that change. It's not that it's difficult. It's not that we didn't think about it. We spent a lot of time evaluating it, and we feel the quality of the project would suffer too much as a result to justify adding it (in this form). Realistically speaking, #5378 is probably the best concession we can make for this to work in a way that's opt-in and backward-compatible (EDIT: Of course, I'm open to suggestions that improve that proposal within these parameters)

On --project-directory
A bit of a tangent, but I'm aware this option is a bit broken right now. In retrospect, I would have preferred not adding it at all, because it creates more problems than it solves. I'll try and spend some time to see if it's fixable, but it has a lot of weird edge cases that make it really unreliable.

---

With all that said, is there anyone for whom docker-compose.env + #5378 would not be a satisfactory solution? I understand it might not be your favored solution, but I'm asking if it's making reasonable concessions you can cope with.

[nixpulvis]

All joking aside...

I still don't understand how Daniel's latest comment relates to "quality of the project". Further it's a general rule of thumb that adding a key to a dictionary should never cause backward compatibility breakage. There are always cases where programmers have implemented things in a way that this is not the case, but I digress.

I think the camp that wants the project-name (or whatever) field is justified, since the "name" of the project is fundamental to many aspects of the tooling. People in this camp may want to simply write this name down somewhere in stone to avoid issues with more temporary solutions like environment variables or command like flags.

I hope this has been constructive.

[schmunk42]

@estarter I'd consider this a bug, maybe it's worth to open separate issue for it

Actually I'd expect this to work with the .env file in the directory PR_5378:

docker-compose --project-directory PR_5378 -f docker-compose.yml -f docker-compose2.yml down

[mikehaertl]

On adding project_name to the Compose file

@shin- It seems, the decision is already made. But I still try to understand the background. I've asked this before and never got an answer (which is one reason why the discussion felt so annoying to me).

Could you please help me understand, why we have container_name and even network in docker-compose.yml? The first could be source for potential conflicts, too. And the latter is highly host specific. Following your logic those two (and others) shouldn't be there either. I fail to see the difference to project_name.

Please, don't ignore this question again.

[AyushyaChitransh]

@michael-k Are you saying project_name is same as container_name? If you are saying so, then I would like to inform you that one project may consist of multiple containers. Or have I misunderstood you?

While for network, it is not necessarily host-specific. There are many cases when a project requires a central isolated network in which one container interacts with other network. And its configurations are stored in the docker-compose.yml.

[mikehaertl]

@AyushyaChitransh You misunderstood. The question is: Why is container_name, network and others in docker-compose.yml while project_name should not be allowed to be there? They all share the same potential for conflict or host specific configuration that should not be shared.

[schmunk42]

@aanand I know about this opportunity. Also, the same could be told about container names, but we do have an opportunity to set container name in compose file, right?

Yes, but I don't think adding that was a good idea.

From related discussion Specify network name in docker-compose file.

[estarter]

@estarter I'd consider this a bug, maybe it's worth to open separate issue for it

@schmunk42 i see that maintainers are aware of this - see On --project-directory in Joffrey's comment plus #4709 , #4933 and #4841

Actually I'd expect this to work with the .env file in the directory PR_5378:
docker-compose --project-directory PR_5378 -f docker-compose.yml -f docker-compose2.yml down

You assumed that --project-directory affects -f parameter.
In fact it's not the case, this command gives IOError: [Errno 2] No such file or directory: u'./docker-compose.yml'

It's very good that --project-directory is not affecting -f - otherwise imagine what a nightmare would be to use --project-directory and multiple docker-compose files all in different directories.

[thedeeno]

I'm very surprised to see so much deliberation over something I view as a solved problem.

This is the order of precedence I'm used to seeing:

1. hardcoded
2. cli option
3. environment
4. project.rc
5. sane default

For project_name we currently do not have either option #1 or option #4. This is the fundamental problem people are facing and it seems easy to resolve in a backward compatible way.

We seem to be having philosophical arguments about purity which might not be helpful. Using a very common OSS cascading configuration scheme on the other hand, is very helpful.

[shin-]

We seem to be having philosophical arguments about purity which might not be helpful.

Having this argument is really not the aim though. From the outset, we have been very clear about where we stand regarding defining the project name in the Compose file.

But let me be transparent on this. This will never ever happen in docker-compose. If that is the only resolution path you're willing to accept for this issue, then you'll have to go with the very OSS process of forking the project and doing it yourself. The code is available and shared with a very permissive license.

Could you please help me understand, why we have container_name and even network in docker-compose.yml? The first could be source for potential conflicts, too. And the latter is highly host specific. Following your logic those two (and others) shouldn't be there either. I fail to see the difference to project_name.

For container_name, that option was introduced very early in the lifetime of the project. As @schmunk42 pointed out, if we were to do it all over again, this would definitely not be in the spec. It's constantly misused and has been the source of countless issues for users over the years.
As for network, I'm not entirely sure how it compares in your mind?

[thedeeno]

What about option #4 I mentioned? I saw some talk of introducing a .docker-compose for local directory specific settings. This would be far preferable (imo) to introducing the _x_ environment flags (which still require environment management).

It'd be like this:

# <my-project-dir>/.docker-compose
project_name: foobar

The above is distinct from the environment variable option and lower in the precedence list.

This solves my use case nicely and ticks another standard config path off the list.

[shin-]

@thedeeno In our mind, COMPOSE_PROJECT_NAME inside the .env file already accomplishes that (with the same priority order as it would have in your outline). However, we have heard you on .env being a poor choice of name, which is why we're considering docker-compose.env as an alternative. Would that work for you?

[thedeeno]

In our mind, COMPOSE_PROJECT_NAME inside the .env file already accomplishes that

@shin- this is where we disagree. I don't think it does.

I personally love that we source the .env file. This is a very conventional place to put local secrets and other local hardware specific configuration.

This works great for assisting with number 3 above, environment based configuration.

It does not give us an escape hatch (number 4) for when we don't want the environment to own the project name.

In my case our developers will each have their own .env file (not VC). It contains secrets for injection into docker-compose.yaml. Love this.

We do not want the developers to have control over the project name, however, since it is strongly coupled to some tooling. Since .env is not in version control, we have to have our developers manually set the COMPOSE_PROJECT_NAME with this approach; and it's the _one thing_ not like the others, because it's not a secret.

So sadly no, simply changing the default source path of the .env wouldn't help us here. We actually love that .env is sourced automatically. We just don't want to specify project name that way.

[glensc]

i'll show one usecase from my practice:

i'm using .env file to specify image version, and to avoid weird sed/awk/perl/whatever parsing, i just overwrite the value in CI:

    echo APP_VERSION=$CI_COMMIT_REF_NAME > .env
    docker-compose pull
    docker-compose up -d

later in docker-compose.yml, image is specified as:

    image: my.registry.example.net/app:${APP_VERSION}

now if i need more variables of that kind, i need to do some parsing, unless multiple .env files are supported.

so, perhaps also add "directory" support, like: docker-compose.env.d/* or .docker-compose.env.d/* or docker-compose.d/*.

but as such glob can instantly create problems of matching editor backups (*~, *.bak) it's better to use some extension instead: docker-compose.env.d/*.sh

... but then again, maybe just make it configurable in docker-compose.yml what root-level .env files are loaded, or that would introduce chicken-egg problem? i'm not aware how the config parser works :)

[glensc]

a bit offtopic, but COMPOSE_PROJECT_NAME env does not allow fully control prefix, it would be still stripped to match A-Za-z0-9_... i wanted to use - in my prefix. (i can't find where i reported this right now, but afaik it's not addressed)

it would be logical to allow same characters that docker itself limits to.

[thedeeno]

@glensc That's a lot of infrastructure to manage the environment. I personally don't think that's the responsibility of docker-compose.

Only thing I've seen other project's do is automatically source .env

[shin-]

@glensc @thedeeno So ideally, we'd have two *.env files, one meant to be shared with other users, and one kept private, with the latter overriding the former when both define the same var?

I addressed configurable .env names in my comment last week.

[thedeeno]

Having a public docker-compose.env (while still auto sourcing private .env) would work great for us!

We'd treat it like an rc file and commit it source control.

[glensc]

i think having public/private env files, likely solves most of the use-cases noted here. if - would be allowed in project name as well, that would be great.

[schmunk42]

@thedeeno @glensc Could you please give a use-case where you need a private ENV file overriding the values in the public one?

Secrets should not be defined in docker-compose.env, since they won't be auto-forwarded to containers. You could still use a .env file then and do whatever want with it. But it looks confusing to me to mix settings for docker-compose and containers in the stack.

[thedeeno]

@schmunk42 I don't have that use-case so I won't be much help there. I just need to commit the project_name to version control. docker-compose.env solves that.

Not sure I follow your second paragraph. We use .env for secrets and manually inject them in docker-compose.yaml.

[glensc]

@schmunk42 https://github.com/docker/compose/issues/745#issuecomment-346491062

it's just because ONLY place to define ${variables} for image value seems to be .env file.

[schmunk42]

@glensc Agreed!

But instead, I would propose docker-compose.override.env to align it with the current override feature for yml files.

docker-compose.env
docker-compose.override.env
docker-compose.override.yml
docker-compose.yml

Not sure I follow your second paragraph. We use .env for secrets and manually inject them in docker-compose.yaml.

@thedeeno I assume you're doing this:

environment:
  - ${SECRET_VAR_FROM_DOTENV}

which should also be possible with docker-compose.override.env. Or you could do:

env_file:
  - .env

[thedeeno]

Are you suggesting dropping .env in favor of a more docker-compose specific
path?

If we skip the automatic sourcing of .env it will negatively impact my
team's workflow. We rely on this file for other tooling. We'd end up
putting our secrets in two (.env and this other file you're suggesting)
places if docker-compose stops looking at .env.

On Thu, Nov 23, 2017 at 1:15 AM Tobias Munk [e-mail]
wrote:

@glensc https://github.com/glensc Agreed!

But instead, I would propose docker-compose.override.env to align it with
the current override feature for yml files.

docker-compose.env
docker-compose.override.env
docker-compose.override.yml
docker-compose.yml

Not sure I follow your second paragraph. We use .env for secrets and
manually inject them in docker-compose.yaml.

@thedeeno https://github.com/thedeeno I assume you're doing this:

environment:

• ${SECRET_VAR_FROM_DOTENV}

which should also be possible with docker-compose.override.env. Or you
could do:

env_file:

• .env

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/745#issuecomment-346538315, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFIdxtbI7y3wW2TFa401Go6Msj0gC74ks5s5Q1vgaJpZM4DLBNs
.

[schmunk42]

Are you suggesting dropping .env in favor of a more docker-compose specific
path?

Not necessarily, for BC docker-compose could still look at .env if the others are not found, same as with fig.yml once. Although it might be tricky to decide the following cases:

.env
docker-compose.env

.env
docker-compose.override.env

.env
docker-compose.env
docker-compose.override.env

.env would not be used in the last case, but I am undecided how to handle to first two, since some users would use .env as an override, others as a fallback.

[thedeeno]

To me .env should be used in all cases. It is a file with special meaning
in the wider ecosystem.

In your third cases it's just the file with the lowest precedence.

All I need is a way to commit my project name to source control. I don't
particularly care about more sophisticated cascading environment
management. I'd actually prefer this project name strategy had nothing to
do with environment variables.

If a configuration file is off the table and there's a way to commit my
project_name with this env cascade you're taking about I'll happily use it
though!

On Thu, Nov 23, 2017 at 1:31 AM Tobias Munk [e-mail]
wrote:

Are you suggesting dropping .env in favor of a more docker-compose specific
path?

Not necessarily, for BC docker-compose could still look at .env if the
others are not found, same as with fig.yml once. Although it might be
tricky to decide the following cases:

.env
docker-compose.env

.env
docker-compose.override.env

.env
docker-compose.env
docker-compose.override.env

.env would not be used in the last case, but I am undecided how to handle
to first two, since some users would use .env as an override, others as a
fallback.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/745#issuecomment-346539891, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFId95kZQIsiWp488JyPQOtGJju0OPbks5s5RFbgaJpZM4DLBNs
.

[mikehaertl]

As for network, I'm not entirely sure how it compares in your mind?

@shin-

services:
    web:
        build: ./
        networks:
            - my_project_network_on_my_localhost
networks:
    my_project_network_on_my_localhost:
        external: true

Guess I'm doing it wrong, because I never commit my docker-compose.yml to my repo. It always contains highly host specific settings. It can have quite different content in development an production.

[dnephin]

I am a fan of "4 project.rc" although I would probably use something like docker-project.yaml instead of a different format. This is exactly what the original issue describes. As far as I'm concerned any discussion about putting the project name in the compose file should be in a separate issue.

All I need is a way to commit my project name to source control.

This is still a point I don't understand. There should be no reason to require a single hard coded project name. If there is external tooling that expects a consistent name, why is that tooling not setting a project name before it calls docker-compose ?

my_project_network_on_my_localhost seems wrong to me. Why do you need the project name in an external network? It doesn't need to match the compose project name.

[mikehaertl]

my_project_network_on_my_localhost seems wrong to me.

This was only an example. I have a different network setup on my local development machine than on a production server. Many other settings in my docker-compose.yml also regularly differ from production, e.g. I have utility containers defined there to build project specific stuff or maintain database or whatever.

As I understood it, initially the main idea behind docker-compose (or better fig) was, to nicely organize all those lengthy docker command line options for a complex container setup in a simple yaml file. That's why it feels so weird, that there's one single option left out. But it seems I somehow missed that this is no longer the main idea behind it.

[schmunk42]

Many other settings in my docker-compose.yml also regularly differ from production, e.g. I have utility containers defined there to build project specific stuff or maintain database or whatever.

We have the same workflow here. In our case the "common" docker-compose definition boils down almost nothing, but these are the only things which are really shared among dev, testing, staging and production.
The main parts of the development settings are defined in a separate file, automatic merging is done via a .env file. It's the same for the testing setup.

Because it is in a folder in our case, we can do something like

(cd tests && docker-compose up -d)
(cd tests && docker-compose ps)
(cd tests && docker-compose run php codecept run)

for testing ... or this

(cd production && docker-compose logs -f --tail 100)

for production (Bonus: You could define DOCKER_HOST in production to point to another machine, but only with docker-compose).

But yeah, it's a folder, not a file - and requires cp .env-dist .env in two places to work, otherwise it fails with an invalid compose file. Actually, I just wanted to share this IMHO neat workflow.

[thedeeno]

This is still a point I don't understand. There should be no reason to require a single hard coded project name. If there is external tooling that expects a consistent name, why is that tooling not setting a project name before it calls docker-compose ?

@dnephin There are reasons.

Yes there are work arounds. The ask is to make it more convenient and predictable.

Here's one example where a hard-coded project name helps: traefik's docker configuration defaults to creating host rules for containers in the following format: <service>.<project>.<domain>. This formatting is not easy to change. It's very beneficial for our developers to use the same fqdns for our docker-compose services. It'd be great if we could use the zero-configuration setup, as it stands the only way to guarantee consistency is to either: a) force them to use a specific directory name for their clone b) manually specify the rules for traefik. Both suck.

Here's another: referencing images built with docker-compose. If we cannot guarantee the project name we cannot guarantee these image names and can't reference them with confidence. Sure, we can hardcode the image_name for each definition but it frankly feels redundant because we _want to use the convention_ but are just concerned that developers choosing to use different filenames will have very surprising failures in our tooling.

[Hronom]

Guys, so many discussions... what the status of this, can I set default project name in my .docker-compose file?
Or still need more discussions in this thread?

[askoretskiy]

Storing project name in docker-compose.yml file makes much sense when used with other tools, e.g. PyCharm.

It would be great to have store it in the same YAML file but possibly override if needed.

[flaviovs]

I think that the core of this issue come from the fact that the default project name is simply (should I say _naively_?) taken as the base name of the current directory. This is effectively compressing the file system name space (hierarchical) to a flat one, which is problematic because of name conflicts. For example, this causes issues for people running services in ~/fooproject/master and ~/barproject/master.

I do think that project names are unimportant. So perhaps a way to make docker-compose robust is to generate project names bases on the full path (i.e. home_flaviovs_fooproject_master)?

I like the persistent idea, however. To be able to commit the project name to Git is such a common use case. Perhaps reading a .env.default file before .env is a simple solution for this (which BTW would take care of the long standing #210 as well)?

[helmus]

I solved this by wrapping docker-compose binary with my own function so I can call the function from anywhere. This loads the .env file and uses the right docker-compose file anywhere.

Obviously you could extend this to have a function per project or something. I think there might be drawbacks to it but I haven't found them out.

DEFAULT_DOCKER_COMPOSE=~/my-project

function dcompose() {
    pushd $DEFAULT_DOCKER_COMPOSE &> /dev/null;
    /usr/local/bin/docker-compose $@;
    popd &> /dev/null;
}
## maintain auto complete
function _dc {
    pushd $DEFAULT_DOCKER_COMPOSE &> /dev/null;
    _docker_compose;
    popd &> /dev/null;
}

complete -F _dc dcompose

[vedmant]

I have a problem with images collision, and it looks really weird that I can not set project_name in the docker-compose.yml as it looks very logical and strait forward solution. Actually taking name from the project folder in the first place was a bad idea. In many cases projects folders can be like /home/project1/repository, /home/project2/repository and so on, in this case docker generated the same images names like repository_app.

[schmunk42]

@vedmant You can set the image name in docker-compose.yml, just use image and build.

[bscheshirwork]

So... How about separated by projects on same-named folder now?

docker-compose -f /foo/bar up -d
docker-compose -f /foo/foo/foo/bar down

[glensc]

Perhaps this was already proposed. but why not make a breaking change in version: 4 and add some new keyword to allow setting project name from docker-compose.yml?

[MyJoiT]

my .docker-compose/project-name did not work ...
--project-name worked

[dazinator]

I would like to set project name in the docker-compose.yml also.

My use case is, I have a docker-compose.yml file which has a web app and a postgress database service.
I also want seperate docker-compose.yml files for adhoc tasks - basically describing adhoc tasks that I'd like to orchestarte with docker-compose. These adhoc tasks need to leverage the same same services / networks / volumes as used in the main project.

An example of an adhoc task that I'd like to orchestrate with docker-compose might be. migrating data into the postgress database. Its useful to orchestrate this with docker-compose because the task might involve spinning up additional services and volumes, networks, etc, alongside the current ones, in order to fulfill that task.

So in order to achieve this, I can create a seperate docker-compose.adhoctask.yml file to represent the task, so that I can run that task on demand only when required. Because it's in the same directory as the original docker-compose file, it gets the same project name by default and so it has access to the same networks, volumes etc? and stuff works.

However the problem comes when I don't want this adhoc task yml file to live in the same top level directory (or even the same repo) as the main docker-compose.yml. For example, perhaps I want to put the adhoc task docker-compose.adhoc.yml in a subfolder, with the DOCKERFILE and assets that the task needs nicely isolated / grouped together as it's own concern. As soon as I move it to a subfolder, it's project name no longer matches. This means it no longer has access to the same volumes / networks etc defined in the main project.

If I could specify the projectname in the docker-compose.yml and docker-compose.adhoc.yml - then I could structure them however I want (even put them in seperate repo's) and still execute them without having to constantly specify additional command line arguments or switch environment variables around.

[fprochazka]

This is a real problem for me, because it causes naming conflicts

I have several projects in the structure

/company
    /ab   # project prefix
        /backend   # actual project
        /webapp
    /cd
        /backend
        /webapp

now when I start a docker-compose in one project, it creates network backend_default and prefixes containers that are not explicitly named with backend_ .. now If I want to start the other project, I have to go back to the right folder and first run docker-compose down to prevent conflicts - otherwise it would start new containers in the same network, and then when turning them off, it would complain about orhans, when there are none.

[simplesmiler]

Also an issue for me. I have multiple projects, and use docker-compose for dynamic demo stands on a single machine, a stand per branch.

/project-1
    /dev # docker-compose.yml inside
    /master # docker-compose.yml inside
    /feature
        /new-feature # docker-compose.yml inside
/project-2
    /dev # docker-compose.yml inside
    /master # docker-compose.yml inside

So "masters" and "devs" are conflicting between between projects. A .docker-compose file could solve that.

[shin-]

@simplesmiler At the risk of repeating myself, a .env solves that already.

[thedeeno]

@shin- .env is often used for secrets not in version control.

What @simplesmiler wants is a way to commit the project name to version control _while keeping .env outside version control_.

As far as I can tell there is still no way to do this easily.

[shin-]

@thedeeno that's not what their post say. Are you working together? If not, we shouldn't presume what someone else's use case is beyond what's explicitly written.

[thedeeno]

We're not working together. That's my interpretation of their problem. Totally fair point, I might be wrong and I might be presuming.

I wish WE worked together though so we could grab some coffee. I hope you don't think I'm being a jerk here.

Maybe I'm just remembering this thread incorrectly: Is there a way to commit the project name to version control while keeping .env ignored?

[shin-]

Is there a way to commit the project name to version control while keeping .env ignored? Or is this still a missing capability?

Not currently. As I mentioned a few months ago^1, the plan is to have docker-compose.env as a secondary env file that can be included in version control. It has slipped down the list of priorities somewhat but I hope to get to it soon.

1/ https://github.com/docker/compose/issues/745#issuecomment-345054893

[thedeeno]

That's great news! Accepting PR's?

[mikehaertl]

I gave up hope long ago but let's give it another try:

I think many here still fail to understand why we are forced to maintain a second file for this setting.

As I understand it, for some few here it breaks their project setup. But for the majority in this thread it would not be a problem. So how can it be that those few impose this restriction to all the others who are missing this feature? Why can't we have several options for how to configure the name and then leave the decision to us?

It's not even logically making any sense: The lack of this setting in docker-compose.yml is so obvious that I couldn't even believe that this option is missing when I looked for it in the manual. Almost each and every other aspect of your setup can be configured there. To me that's such a clear break in the configuration logic.

[schmunk42]

I think that one could say, (almost) all settings in docker-compose.yml are "scoped" by the project name.

[roipoussiere]

As suggested in #4841, an option to specify which .env file to use (ie --env-file) should be very useful.

[schmunk42]

@roipoussiere see https://github.com/docker/compose/issues/4841#issuecomment-414543951

[roipoussiere]

Thanks @schmunk42.
But that means I must create a hierarchy like:

.
├── project_a
│   ├── .env
├── project_b
│   ├── .env

instead of something easier like:

.
├── a.env
├── b.env

[schmunk42]

Yes, see also https://github.com/docker/compose/issues/745#issuecomment-345054893 - you might need to open that in a new window, since it's sometimes hidden here :nerd_face:

[nickredmark]

I second the --env-file proposal. Made a ticket just for that. Go and like it if you want this option.

[arvera]

So I only read a few comments on this bug/feature, and the fact that discussion continues and that it is open brings me to the conclusion that it has been 4 years since this was originally opened and there is still no solution? Can we make a partial decision and then move forward..?

[nickjj]

I'll admit, I haven't read this entire issue and all of the related issues but there are interesting problems around this topic that makes this more complicated than it seems at first glance. For example, as we all know .env is commonly used for secret values, so you typically ignore that from version control.

But let's say you're working with Stripe, which happens to allow you to have test keys and live keys, AKA. 2 sets of keys to work with. In development it would be very common for you to use the test keys, and in production you would end up using the live keys.

That automatically means you can't use .env by itself to deal with secret values because you'll have different keys in each environment, and you don't want to have to switch the values in your file between running your app in dev or prod (that would be crazy and super error prone).

What you likely end up doing is creating both a .env and .env.prod file and ignore both from version control. This way you can have your test / dev keys in development and then reference that with env_file in your development compose file.

But then in production you reference both .env and .env.prod in your production compose file.

But do you see where this is going? Right now Docker Compose doesn't let you have optional env_file files. If the file doesn't exist, then Docker Compose is going to blow up as soon as you try to run Docker Compose.

So you automatically need a .env file to exist in the end if you plan to use it with env_file and this isn't limited to Stripe either. A lot of web frameworks have certain settings that are environment specific but would change in both development and production (ie. SERVER_NAME with Flask).

So that means we're limited to having something like a .env_example file that you commit to version control and it's expected the end user renames that file to .env, and then inside of this file would contain our old friend COMPOSE_PROJECT_NAME.

Which now brings up the question. Do we actually need an alternative way of setting the project name? I'm not convinced that we do, but I am convinced setting a custom project name is a good idea in any project, or at least something other than the folder name since you can easily run into name conflicts.

[landsman]

Is this issue on some road map please?

[cdobbyn]

The use case for this would be to modify the COMPOSE_PROJECT_NAME in the case where your project has multiple docker-compose files (e.g. one for dev, test, maybe a base, maybe a test local, etc.) You may have a need to spin up 2 very different environments that are separate from one another.

In our case, we can kind of get around this docker-compose limitation by creating the COMPOSE_PROJECT_NAME in our makefile.

I would like to see this configurable from within the compose yaml itself.

[brunobertechini]

Is this still open? Oh my god...

[lopezator]

Subscribed. I also want the be able to set the project name on the compose yaml file.

This is taking forever 🤦‍♂

[F-Yousri]

would be very helpful to have this feature

[HairyMike]

this feature would be very welcome in the project I'm working on

[terjebraten-certua]

Ok, so from this long thread we can conclude the project name will not be in the docker-compose.yml file, becuase the main developers are against this and will not merge any such pull request.

So, can we please get back to the original proposal then at the top of this issue: to make project-name persistent. Using either a file .docker-compose or a directory .docker-compose sounds fine to me. But it would be nice if we finally could get some progress on implementing that.

[amancevice]

I'd add an ENV variable to that list of alternatives.

[terjebraten-certua]

A substantial root of all the disagreement is this thread/issue is that for many people it does not work if it is in an ENV variable. They want to be able to commit it to a repo (optional of course, the standard recommandation can be to not commit it.)

[timanovsky]

Having it ENV rather than content of yaml contradicts to philosophy of sharing yaml files between different projects. In fact it is officially documented: https://docs.docker.com/compose/extends/ (See "Example usecases" for example). If you use single base yaml file, and then override files for dev, test, and prod, and god knows what else, it makes no sense to run it under same project name or set project name by ENV/ cli. It will just create N*N combinations while on N are valid.

[artnaseef]

I have traversed several of these discussions and see a TON of comments from people wanting project name to be configurable in the YML file. But not 1 single comment explaining why that should not be done. These discussions date back a long way.

As someone looking for an easy-to-use solution and finding that there is no easy way to set the project name that does not require careful user attention to the command-line executed (which competes with the core idea of a tool to which you pass up and magically start a full cluster of servers) - I am shocked that this is an open question. If there were some clear explanation of why it is problematic or concerning to implement, I would be interested.

[jderusse]

an elegant solution could be to use placeholder in the container_name property (and other dynamic naming like volume, network, ...)

services:
  my_service:
    container_name: {project_name}_{service_name}_{instance_number} # default value

with such configuration, one could use

services:
  my_service:
    container_name: fixedprojectname_{service_name}_{instance_number}

edit: this is not a perfect solution, as it'll conflict for people who use 2 distinguished project within the same folder name

[Ladeiras]

@jderusse

an elegant solution could be to use placeholder in the container_name property (and other dynamic naming like volume, network, ...)

services:
  my_service:
    container_name: {project_name}_{service_name}_{instance_number} # default value

with such configuration, one could use

services:
  my_service:
    container_name: fixedprojectname_{service_name}_{instance_number}

edit: this is not a perfect solution, as it'll conflict for people who use 2 distinguished project within the same folder name

I'm also doing this and thought this would fix the conflict problem, although we still need to set the COMPOSE_PROJECT_NAME variable to some unique value on the .env file.

[artnaseef]

At this point, I really just want to understand the reason the maintainers are rejecting solutions. There is already a pull request for a solution.

Once we know what is wrong, then we can work on fixing it. At this point, I see no feedback other than "rejected".

[terjebraten-certua]

We have these 2 comments:

https://github.com/docker/compose/issues/745#issuecomment-345054893
https://github.com/docker/compose/issues/745#issuecomment-346487757

But I must admit even reading those comments, I do not fully understand why not just add the project_name to the yaml file. For those that think it makes their docker-compose.yaml files less general, they should just not put the name in the yaml file then, but please let the rest of us have an option to do so.

[artnaseef]

OK, I'm doing my best to read through references to other comments; would really appreciate at least a short blurb that references content in the reference, in addition to the reference to help make sure I'm reading the right thing and getting the right content.

Let me attempt to summarize what I'm seeing are the concerns. Any help to clarify this down would be appreciated.

(1) Adding project name to the YML file may make the project less reusable since it would prevent, or at least complicate, running the one YML file with more than one project name.
(2) "the quality of the project would suffer too much as a result to justify adding it (in this form)"
(3) "If the project name is in the compose file then any two project could use the same name, causing a conflict, which would break both projects."

I have thoughts on these, but I will hold off with comments until we can agree on the concerns that need to be addressed. Is that reasonable?

[schmunk42]

(4) Implementation in a backward compatible way (precedence/order of values)

[timanovsky]

(1) Adding project name to the YML file may make the project less reusable since it would prevent, or at least complicate, running the one YML file with more than one project name.

Yet again, there is official way to split config into base and file and inherited - https://docs.docker.com/compose/extends/. One could not set up project name in the base config.

For me the primary use case I'm looking for is ability to run multiple instances of the project locally (dev, test environments etc).

[rijnhard]

Maybe the solution is to allow you to either set the project name explicitly or a path traversal.

If I set the project name in the yaml file explicitly like my-project then can simplify things for a single instance of a project or where I do something like add all of my docker files in a docker folder across multiple projects.

.
├── project_a
│   ├── docker
│   │   ├── docker-compose.yml (project_name: my-project)
├── project_b
│   ├── docker
│   │   ├── docker-compose.yml (project_name: some-other-project)

But this doesn't help with multiple instances of the same project, so how about doing something with the path traversal

project_name: ../(*) so with the same example

.
├── project_a
│   ├── docker
│   │   ├── docker-compose.yml (project_name=project_a)
├── project_b
│   ├── docker
│   │   ├── docker-compose.yml (project_name=project_b)
├── project_b_copy
│   ├── docker
│   │   ├── docker-compose.yml (project_name=project_b_copy)

similarly project_name: ../(../*)

.
├── dev
│   ├── project_a
│   │   ├── docker
│   │   │   ├── docker-compose.yml (project_name=dev_project_a)
├── test
│   ├── project_a
│   │   ├── docker
│   │   │   ├── docker-compose.yml (project_name=test_project_a)

how to specify the traversal should probably be thought of a bit more in-depth than my thumbsuck, but at least by doing this I think we have flexibility to cover all use cases.

[ankon]

Only half-jokingly: That means to configure a fixed name in my docker-compose.yml I would create a file with the name I want, and then set project_name: name-of-the-file-named-like-the-name-i-want?

[flatline-studios]

I currently have a load of docker-compose files in the same directory. Like dc-work.yml, dc-server.yml, etc., that all load different, unrelated services.

When I run docker-compose up dc-A.yml, and then run docker-compose up dc-B.yml, Docker nags me that there are orphan services running (of course there aren't, it's because of the damned project name).

Is there really no way to solve this? Moving each dc-A|B|C|etc.yml file in to its own directory, and then traversing into each to load up all my services seems like a real waste of time, or am I missing something?

[CharlieReitzel]

Why hasn't this issue been addressed? The clear and straight-forward solution was presented by @cr7pt0gr4ph7 here: https://github.com/docker/compose/issues/745#issuecomment-182296139. It received a lot of up votes. Every single concern has been addressed in the discussion. And, most importantly, it solves a lot of issues for a lot of people. What gives?

[ThaDaVos]

Also running into this issue because we're using a fixed project directory structure with a .docker folder which contains all docker-related stuff, so I'm ending up with lots of .docker project names - I'm working on a microservices architecture so it's normal to have multiple up... but this small little thing, keeps getting in the way -> also, PHPStorm etc doesn't have the option to pass the -p/--project-dir argument and I can't really use a .env for this because, believe me, setting the right value will be forgotten...

[AyushyaChitransh]

Developers, please let us know how do you want it to be resolved.

[ThaDaVos]

Maybe add a property to the docker-compose file to set the project-name?

[CharlieReitzel]

@AyushyaChitransh

Developers, please let us know how do you want it to be resolved.

This is the best summary:
https://github.com/docker/compose/issues/745#issuecomment-182296139

There has been plenty of discussion. There is no need for any further discussion. The project maintainers just need to read the threads (plural) and do the necessary.

[schmunk42]

For me it is still this: https://github.com/docker/compose/issues/745#issuecomment-248644429

[CharlieReitzel]

For me it is still this: #745 (comment)

Yes, there is an existing, but poor solution. Those comments have been down voted and the problems have been clearly stated by many. The linked comments simply do not address most of the legitimate concerns raised in these threads.

Tbh, it's kind of a moot point. Many of us have moved to Kubernetes for docker-compose type functionality. It's a little verbose. But it's working and is well supported by the cloud vendors.

[dazinator]

@AyushyaChitransh
We want to use mechanism to set project name as described here: https://github.com/docker/compose/issues/745#issuecomment-182296139

1. Command line arg.
2. Environment variable (.env file)
3. In Compose file directly
4. Folder path

Don't think there is any objection this.

[schmunk42]

There is, it is about portability, as mentioned above, somehow similar when using container_name, which should also not be part of the docker-compose.yml. You can not reuse the same config with existing settings.

If your workflow is to rely on the folder-path and somehow project-name get injected into your config, eg. through merging, multiple compose files, templates, ... - you'll end up with the same problem - overwriting existing stacks.

Sorry to say this, but this should be closed as wontfix .. (edit) or release a new MAJOR(!) version

[dazinator]

@schmunk42

If your workflow is to rely on the folder-path and somehow project-name get injected into your config, eg. through merging, multiple compose files, templates, ... - you'll end up with the same problem - overwriting existing stacks.

Firstly, project name would only be set in compose files by those that require it and it is precisely because they want the compose files to work with existing stacks. You could add your warning to the docs around this setting.
Secondly for those that rely on a specific project name, consider this:

1. What if the compose file is downloaded from a git repo, then in the next commit the repo owner moves it to a different sub folder? Or renamed the folder it is in? The impact on workflow is already present as files can be moved and that impacts project name.
2. If you rely on a specific project name we have proposed the mechanism to ensure it, by either using environment variable (.env file) or using the command line arg when executing Docker compose - both of which would override any value that happened to be in the compose file (if one was added for some reason).

Closing this as wontfix is not very helpful without explaining why the above isnt good enough.. or offering a path to a solution.

[dc-pm]

I don't want my workflow (or rather the workflow of my entire team) to be dependent on what they name the folder they check code out into.

My main problem with the .env file solution is this file must be in the working dir of the command run - whereas docker-compose resolves the compose file to a parent directory. It, therefore, becomes essential that all developers either (A) check out code into the same-named folder (assuming a docker-compose.yml file in the root of a code repo) or (B) only run docker-compose commands in the root folder of the repo otherwise create conflicting stacks or (C) they have different stack names and all documentation and scripts needs to say replace stack name here.

If (C) is what docker suggests as the recommended 'portable' solution then docker-compose CLI needs to be feature-complete with docker CLI commands and, even then, I still see a use case for this feature.

If your workflow is to rely on the folder-path and somehow project-name get injected into your config, eg. through merging, multiple compose files, templates, ... - you'll end up with the same problem - overwriting existing stacks.

If a project-name gets added to the docker-compose file it certainly there for a reason (like every other line in a docker-compose file). Sadly right now even just renaming my checked out folder (not making any tracked changes in git) I also get a broken system.

I can't believe it's 2020 and we still haven't fixed this huge use case. This isn't a breaking change, is a totally optional feature and make perfect sense. Given Kubernetes is now the defacto deployment tool for docker I would imagine docker-compose to mostly see use in development / local deployments and having a docker-compose file in the root of a repository to run the code in the repo locally is a very popular use case.

[CharlieReitzel]

@dc-pm wrote: "Given Kubernetes is now the de facto deployment tool for docker"

I now no longer write docker-compose files. There's not enough benefit for something that will not see production. Kubernetes has it's quirks, too. But it is widely supported by cloud vendors.

Truth be told, even dealing with containers in my development environment is too much hassle day-to-day. I just run the programs ...

[DavidAntliff]

@dc-pm wrote:

I can't believe it's 2020 and we still haven't fixed this huge use case.

It's not just that it hasn't been fixed, but that the developers actively refuse to fix it.

The problem, I suspect, is that the teenagers who develop this software do not have the depth of experience to understand the implications of this use case, or listen to the vast number of their more experienced users requesting it be addressed. At this point I doubt they have worked in an environment with more than one stack, or perhaps even in teams. Otherwise, frankly, _this would be a perfectly obvious enhancement!_

[agravelot]

Some friendly reminders...
Open-source developers are not your "employees/colleges", in most cases they are working mostly for free.
If it's clearly an issue for you, take some time to investigate solutions (and push a PR).

Blaming is not the solution.

https://github.com/docker/code-of-conduct/blob/master/code-of-conduct-EN.md

[rijnhard]

It's pretty difficult to push a change when the developers are against it...

On Sat, 14 Mar 2020 at 03:19, Antoine Gravelot [e-mail]
wrote:

Some friendly reminders...
Open-source developers are not your "employees/colleges", in most cases
they are working mostly for free.
If it's clearly an issue for you, take some time to investigate solutions
and push a PR.

Blaming is not the solution.

https://github.com/docker/code-of-conduct/blob/master/code-of-conduct-EN.md

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/745#issuecomment-598991880, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABAXP2DRD7H4YI7KQ7B2URLRHLLQ5ANCNFSM4AZMCNWA
.

[timanovsky]

@agravelot I think there have been some PRs, not ideal may be. But through all the discussion owners have very clearly set expectation they don't want this feature. It is not a matter of coding.

[dc-pm]

@agravelot sorry for the delay. And I truly meant no disrespect to the many awesome and talented people that contribute here. But blaming a lack of effort is not helpful either.

I simply highlight the support of an idea that has not been listened to on countless occasions despite clear support over a very long timeframe. The entire point of this (and OSS generally) as defined in our code of conduct is to allow people to contribute code and be respectful of all ideas.

There have already been multiple pull requests authored but if it would help the situation I'm very happy to author another.

What would be a more constructive way forward?

[meden]

There have already been multiple pull requests authored but if it would help the situation I'm very happy to author another.

Yes, devs should go on with reviewing them, or close this as "Won't fix" (possibly argumenting)... Keeping this open for years, with no official feedback, it's just wasting the time of contributors (in the broadest sense). This too is disrespect, IMO.

[isaac-weisberg]

Hey, guys, as a user I wanted to just share my use case. So I have 2 docker-compose.yml files, one for development environment, one for production. They are located in the same folder. They both have a service with same name - called "database". I would like to deploy both of these services at the same time - just because I can, it's containerization we're talking about, I can scale as much as I want. They have different "container_name" values though.

So I run:

macbook@fuck-fuck autocat-server % docker-compose -f ./compose-dev.yml up -d database
Creating network "autocat-server_autocat-dev" with the default driver
Creating database-dev ... done
macbook@fuck-fuck autocat-server % docker-compose -f ./compose-prod.yml up -d database
Creating network "autocat-server_autocat-prod" with the default driver
Recreating database-dev ... done

Note how it says "Recreating database-dev" while I am clearly referring to the service located in the "prod" project. Actually overwrites the existing container. The database gets terminated and the container gets replaced with the latter.

I am doing this on an assumption that separate "docker-compose.yml" files mean separate projects, but apparently this is not a thing in real life. The project to which the service gets deployed is determined by the location of the "docker-compose.yml". To make them be part of separate projects, I need to separately specify the project name in an environment variable. It finally works with this:

macbook@fuck-fuck autocat-server % COMPOSE_PROJECT_NAME=autocat-prod docker-compose -f ./compose-prod.yml up -d database
Creating network "autocat-prod_autocat-prod" with the default driver
Creating database-prod ... done
macbook@fuck-fuck autocat-server % COMPOSE_PROJECT_NAME=autocat-dev docker-compose -f ./compose-dev.yml up -d database
Creating network "autocat-dev_autocat-dev" with the default driver
Creating database-dev ... done

So much for single-command deployment, what an absolute joke. In my opinion, .yml is great place to specify the project name in this context.

Alternatively, I was able to place the services into a single project - by giving them different names: "database-dev" and "database-prod". The only issue with this approach is that there is a warning about the formerly instantiated service becoming an orphan because it's not mentioned in the other docker-compose.yml.

macbook@fuck-fuck autocat-server % docker-compose -f ./compose-prod.yml up -d database-prod                
Creating network "autocat-server_autocat-prod" with the default driver
Creating database-prod ... done
macbook@fuck-fuck autocat-server % docker-compose -f ./compose-dev.yml up -d database    
WARNING: Found orphan containers (database-prod) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating database-dev ... done

This makes no sense tbh, why does it have to be like this. Why DOES it in fact have to be that way?

EDIT1: @dc-pm @CharlieReitzel whatchu think? can you give any comment on my use case, just in case if it's invalid :)
EDIT2: I've put both compose files into separate directories and voila

[MaxmaxmaximusGitHub]

sooo?

[lon9man]

maintainers, anybody here?
developers wait during 6 years!

[aiordache]

Hi! I created an issue in the compose-spec to have the project-name added to the compose schema as that governs what we can add to the compose file.
Once the project-name property is added to the compose spec, we can start the implementation in docker-compose.