Compose: The default naming scheme for containers created by Compose

[j4r3kb]

The default naming scheme for containers created by Compose in this version
has changed from <project>_<service>_<index> to
<project>_<service>_<index>_<slug>

Is there any way to disable this behavior apart from using container_name in yaml file?
We have many scripts that rely on container names and are not using swarm, just a single container stack. This change is very inconvenient for us.

[shin-]

Unfortunately no, sorry.

[ramonmoraes8080]

Same thing here. This slug feature should be optional. How hard it's to pass a flag on the up or build command to turn it off?

[theoturner]

I agree - this should be optional. Several programs making use of Docker Compose now only run if you downgrade to 1.22.

[crossdot]

I agree, this should be optional. We don't need this slug feature at all and a lot of bash-scripts are using old naming format.

[milan-janecek]

Same for us, we must stay on 1.22 as we use "volumes from" feature and rely on old naming scheme. Please, make it optional feature in 1.24.

[guglielmo]

This feature makes it difficult to refer to the containers after they've been upped.

[alex-lion]

I don't get the point of this change. Currently neither docker nor compose offer any service discovery. So, if I want to get a hostname from inside the container, what am I supposed to do? Roll my own service discovery?

BTW, what are benefits of this change? Is it really worth breaking backward compatibility?

[ryanwoodsmall]

The default _slug name change is the correct approach, but not having a way to disable it via an environment variable or command line option breaks a lot of existing workflows. Please consider this a feature request to add a flag of some sort to bring back <= 1.22.0 docker-compose naming behavior.

[jamct]

Sorry, but "external_links" are unusable now. I have to know the name of a container to link it.
+1 for making it optional

[lig]

@shin- this new behavior completely ruins a lot. Before that random slug it was possible to address a container launched from one docker-compose file inside another file like

services:
  app:
    image: app
    external_links:
      - "other-project_other-app_1:other-app"
networks:
  default:
    external:
      name: other-project_default

And that's it. Now, this won't work at all.

How this thing is not optional at least?

[lig]

Another step to the docker-compose death, I believe.

[lig]

So, there is a workaround. container_name parameter is used as is and slug is not used on it. At least, that helps.

@shin- please, don't consider this a bug report about slug is not being used with container_name. Leave it as it is. I'm literally bagging here.

[lig]

Oh boy! https://github.com/ansible/ansible/issues/47978

[achekulaev]

While I understand the incentive behind the change, it is highly disrupting, especially since there is no grace period to at least give developers time to adjust their script. Lots of scripts were written with this naming convention in mind which existed literally forever. Basically this change makes docker-compose 1.23 non-backwards compatible with any other docker-compose.

[lig]

@shin- You personally and all the docker-compose and docker team should learn already what backward incompatible change is and how to bring it to a widely adopted project the entire industry relies on.

First, backward incompatible change is not about you breaking what you've guaranteed earlier to not to. Nobody cares if this thing is used by nobody.

Backward incompatible change is when you break something your user base actually rely on. And it doesn't matter what guarantees were because it's Apache 2.0 after all and the entire project provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND. Your users decide what they rely on. And you (the entire team) should start to learn how to be aware who uses your project, why, and how.

Second, you should learn how to introduce such a change to your user base. This "random suffix thing" definitely breaks the way your users use external_links and volumes_from. So, the deprecation warning in case there is no explicit container_name set or when it looks like so would be highly appreciated. Please, consider these documents to be familiar with "what others do":

• https://www.python.org/dev/peps/pep-0004/
• http://api.mongodb.com/python/current/compatibility-policy.html
• https://docs.openstack.org/python-novaclient/pike/reference/deprecation-policy.html
• https://docs.djangoproject.com/en/dev/internals/release-process/#deprecation-policy

Third, @shin- , you personally should learn how to talk to people who are not direct contributors to your project and who are just your users but at the same time they are very busy people who are loyal to your project enough to invest their precious time filing requests and participating in discussions here with the only hope to bring some sense to the project future development and to try to avoid investing more resources in the migration away to another solution.

Fourth, Docker team is trying to make money out of the Docker but it is not about Docker itself. The only way Docker can be a good thing to pay for is if the entire Docker infrastructure proofs it's worth it to pay. I don't see how it is a product oriented to user success if the development team turns its back to the community most of the time.

Fifth, once again, we all here are your colleges and friends rather than enemies and we are really trying to help you with all this. And because we are your friends the exodus from the docker stack will be painful but in the way, it goes now, that farewell doesn't look unrealistic at all.

[ShotSkydiver]

This is a terrible implementation of a major version-breaking change, it completely breaks the ability to deploy our framework and I cannot even imagine how many others will experience the same exact issue, with absolutely zero advance warning; unless someone manually goes looking for the latest changelogs for docker, they'd have no idea about this, and I imagine most people won't do that

[slaughter550]

@shin- This change is highly disruptive to existing workflows and there should be a flag provided to not generate slugs. Thousands of hours of development time has gone into writing scripts and automating services which rely on docker-compose's predictability. Without proper service discovery, what is the team's plan for allowing automated targeting of individual containers?

[mauricioabreu]

I tried to use docker-compose container_name directive but it did not take effect on GitLab.
But it looks that on my latest version of docker-compose on Mac Os Mojave it is working. Not sure what is happening :)

My .gitlabci.yml file is downloading the last image from tmaier (it should be 1.23.1).

[shin-]

Hey all, a few general comments that I hope address some of the concern expressed here:

• I definitely understand that this breaks some scripts. Our release notes since 1.23.0-rc1 say so at the very top. This is very much a "ripping the bandaid off" type of situation where the momentary pain helps us moving forward with a healthier base.
• The benefits of the current solution are numerous and expressed in part in this long-standing issue: #1516 - and the issue with docker-compose run especially has been reported several times here: #4688 #4549 #5240
• Service discovery shouldn't be an issue as it should be handled using service names (rather than container names) and network aliases, which are static on their associated network. You may read the networking docs for more details on that specifically
• I reckon that this makes volumes_from and external_links across Compose projects more difficult to use. Please consider that even before this change, there was no guarantee that Compose would respect the "expected" format for a given container name (see e.g. #6155 or the prefixing that is mentioned in #3415), making it a flawed solution prone to run into obscure issues in the long run.
  
  
  
  • The recommended way to allow containers from different projects to communicate is for them to share an external network and use the other service's alias(es). Similarly, volumes_from across projects should instead leverage external named volumes.
  
  
• I'm interested in suggestions on how this change could have been communicated better. For reference, the change has been called out in our release notes and available for testing since 1.23.0-rc1, which was released September 26. 1.23.0 GA was released more than a month later, and knowing the change would be controversial, we put it as a big warning at the top of the changelog. If there's anything more I could have done to make this change visible, I'm happy to listen and do these things the next time we consider a risky change like this one.
  
  
  
  • On the flipside, if the general takeaway here is "don't make any breaking change ever, or let me opt out of all of them indefinitely", I'm sure most of you realize that it's not a healthy, reasonable way to go about maintaining a software project the size of Compose, which already jumps through a lot of hoops to maintain backward-compatibility with a wide array of Docker Engine versions, Compose file formats, and deprecated idioms.
  
  

Let me know if there's anything that isn't addressed here. I understand that this is a major speed bump for a lot of you and tempers may flare when we're dealing with unforeseen circumstances. Take the time you need to go through the upgrade and pin your Compose version for the time being. Happy to help with questions like "How do I do XYZ without predictable names?" until then, in this thread or on Slack. And please be considerate in the way you interact with others on these forums, double-check that the information you're sharing is correct (I've already seen a couple of threads getting mentioned or redirected there that had nothing to do with this change, which creates an unnecessary sense of alarm and doesn't help the people with the problem who are getting misdirected here), and don't derail the conversation. Thank you for your time and patience.

[achekulaev]

On the communications part, I would expect a change like this to be communicated and explained through some communications channel beforehand, basically in the way you just did it here in this thread, explaining why, and what to do for now, but doing it before all the aftermath.

[lig]

@shin- You haven't read my last comment and links I've provided, have you?

The last one from you looks kind but feels the same.

I'm interested in suggestions on how this change could have been communicated better.

TL;DR

• Introduce the new version of the docker-compose.yml
• Add DeprecationWarning for external_links with a link to a document which describes the update and provides a sample solution for those willing to migrate.
• Support the old behavior for all compose file format versions prior the new one.

For reference, the change has been called out in our release notes

The changelog is a thing for contributors. Users don't read it. A user installs docker-compose and prays it still works after yesterday.

On the flipside, if the general takeaway here is "don't make any breaking change ever, or let me opt out of all of them indefinitely", I'm sure most of you realize that it's not a healthy, reasonable way to go about maintaining a software project the size of Compose, which already jumps through a lot of hoops to maintain backward-compatibility with a wide array of Docker Engine versions, Compose file formats, and deprecated idioms.

I have a good news for you. You already have guaranteed "don't make any breaking change ever, or let me opt out of all of them indefinitely". And more, you have a feature for this. It's the version in docker-compose.yml file. So, consider reverting this change ASAP and introducing it for the next version number.

Happy to help with questions like "How do I do XYZ without predictable names?" until then, in this thread or on Slack.

Please, provide the solution for external_links and external volumes without container_name defined (as we see it doesn't always work) in another docker-compose.yml in this thread.

This is a popular way your users use your project. Most of your users use docker-compose for the local development. Often one has several interconnected projects in development. In such a case it is a common practice to direct several docker-compose files to the same network and to define external_links for services from different projects to be able to reach each other on the developer's machine.

[lig]

don't derail the conversation

@shin- All the conversations I've read so far involving you become derailed because of you doing nothing to solve problems people have.

Seriously, with such an attitude constantly denying users' requests and forcing your community struggle when you can avoid that you better pass this project to somebody else.

P.S.: Sorry for another "offtopic" comment. Feel free to lock this issue as you are used too.

[shin-]

Please, provide the solution for external_links and external volumes without container_name defined (as we see it doesn't always work) in another docker-compose.yml in this thread.

Already did, please refer to my earlier comment:

The recommended way to allow containers from different projects to communicate is for them to share an external network and use the other service's alias(es). Similarly, volumes_from across projects should instead leverage external named volumes.

The rest of your post is needlessly adversarial and I won't respond to it. I'm here and willing to have a discussion, but I don't have to take your bullying.

[er1z]

Honestly, @shin-

I'm interested in suggestions on how this change could have been communicated better

I don't see you are „interested”. Closing the ticket about pulling image before building compose file and marking it as „spam” (lol) tell me that you are definitely not interested what the community suggest. I really don't know what do you want to achieve but I feel that maintaining a relationship with the community should be held by someone else.

I'm here and willing to have a discussion

Even if we provide pure meritorical arguments, you just ignore them. So why bother?

PS. Mark this as offtopic, go ahead, show us how do you respect our opinion. Seeing so many upvotes on different opinions hurts?

[lig]

Already did, please refer to my earlier comment:

I don't see a working sample solution there, sorry.

The rest of your post is needlessly adversarial and I won't respond to it. I'm here and willing to have a discussion, but I don't have to take your bullying.

Sorry, if you feel like that. Please, consider supporting the old behavior for current docker compose file versions and introduce the new format version which will behave in the new way. This is the only option you have to provide a good solution to your userbase.

Please. consider reading these two my comments again with an idea that I'm willing to help the project in your mind:

• https://github.com/docker/compose/issues/6316#issuecomment-438564025
• https://github.com/docker/compose/issues/6316#issuecomment-438934935

[jordan-thoms]

Re the communications: We got the update through Docker for Mac, and it didn't seem to show any sort of changelog pointing out this change, so it took a little while to figure out why our environment variables had changed.

Once I was aware of this though I took it as an opportunity to upgrade our config from v1 to v3 and link using service names instead of environment variables, and it's actually made things a lot cleaner 👍

[joebowbeer]

FWIW, a backward-compatible hack to exec in a container created by compose:

docker container exec -it $(docker container ls -f name=expected -q) cmd

[lig]

Once I was aware of this though I took it as an opportunity to upgrade our config from v1 to v3 and link using service names instead of environment variables, and it's actually made things a lot cleaner

Could you please provide a short example here for the service name linking across different docker-compose files approach?

[lig]

docker container exec -it $(docker container ls -f name=expected -q) cmd

This has nothing to do with docker-compose though.

[joebowbeer]

@lig This works around the change to the expected name created by docker-compose. This workaround wouldn't exist if it were not for the change in the naming scheme.

[lig]

@joebowbeer the main problem after this change is about linking across docker-compose files. I see, there is a way to find a container launched by docker-compose via docker cmd. But this a little help on the subject:(

[ramonmoraes8080]

@shin- I think we are all still waiting the explanation of why we need docker composer to generate swarm line container names.

For me it's pretty clear nobody in their sane state would need to use composer for production purpose. Compose is a powerful to ramp up project to be tested locally. I don't see the appeal of getting a docker swarm perk/functionality to a tool that's massively used for local only tests. If we needed swarm-like name for containers we would use swarm. Right?

Second big question here is: Why this was not included as an option feature through an argument? I don't see on of my forums and communities people appealing to have this behavior as a default one. So we should probably introduced it as a new arg: docker-compose up --slug. Simple, elegant and would not break anyones scripts.

[nkedel]

  * On the flipside, if the general takeaway here is "don't make any breaking change ever, or let me opt out of all of them indefinitely", I'm sure most of you realize that it's not a healthy, reasonable way to go about maintaining a software project the size of Compose, which already jumps through a lot of hoops to maintain backward-compatibility with a wide array of Docker Engine versions, Compose file formats, and deprecated idioms.

Working in enterprise, in the past, we tended to have a "two major versions" rule on breaking changes -- deprecate in one, remove in the next. I'm not really sure how this could have been made opt-in, but a _temporary_ opt-out would have been really appreciated.

Docker-compose doesn't seem to have a concept of major versions, but I don't see how a temporary "disable this" option (whether from the command line or in the compose file) limited to one or two releases (1.23 and possibly 1.24) would constitute "indefinitely."

Given that this seems to be tied to automatic updates for the non-Linux users, a lot of people were taken by surprise, and instead of having the remainder of the quarter to of "run with this command line flag while we update our scripting" we've got a whole bunch of developers who have to either manually downgrade and/or drop what they're doing to update scripts.

[NicolasDorier]

This breaking change is a disaster. Broke lot's of automatic deployment suddenly. Please don't do that again.

[rockstardev]

* Introduce the new `version` of the `docker-compose.yml` 
* Add `DeprecationWarning` for `external_links` with a link to a document which describes the update and provides a sample solution for those willing to migrate.
* Support the old behavior for all compose file format versions prior the new one.

Makes too much sense... no idea why it's not done this way.

I'm here and willing to have a discussion, but I don't have to take your bullying.

@shin- it's quite obvious there is zero discussion here as you are not considering rational suggestions presented. Plus if anyone is being bully - you are - with this breaking change you are effectively bullying numerous devs around the world.

[jonnrb]

A lot of people (me and some other person probably) don’t use compose for swarm mode. I specify a lower version for my compose files because of issues I’ve had in the past where the assumption is a compose file is used to define services for swarms. I figured that the version I specified was tied to the results it produced on my docker node and by locking the version to something low, I’d be able to get consistent results and still take updates to compose to stay compatible with the version of docker I’m running. Apparently I can’t expect proper versioning here.

[NicolasDorier]

Can we dockerize the docker-compose, to make sure they don't suddenly break our stuff?

[level420]

I think the best solution would be to introduce a new configuration option in the docker-compose.yml file where adding the slug part to the container names could be disabled.

[jonnrb]

s/disabled/enabled/. At least on existing compose file versions.

This behavior needs to be tied to the compose file’s version number. Regardless of whether this behavior was “specified”, it’s pretty widely used, and therefore is a backwards-incompatible change. Why version the spec at that point anyway if what a compose file does isn’t guaranteed to work the same way when I upgrade my binary? Deprecating a compose file spec (with plenty of warning when docker-compose is run of course) and then removing it, it would definitely be my fault all of my stuff is no longer working, but expecting me to read through your release notes for a program that has the privilege of printing me warning messages at my face?

Making changes like this seems like pretty good motivation for versioning the spec’s behavior in addition to its format: you can deprecate others’ code’s assumptions with warning. I think the amount of external issues name-dropping this issue speaks to how surprising this was to a lot of people. It’s real great to see legacy behavior being smashed so that code doesn’t get complicated, but changes like that cannot be applied all at once. Can I expect other assumptions about compose container names to break? Will the service name always be in the container name or can that be removed like the predictable names?

This really reminds me of when Consistent Network Device Naming received widespread adoption in Linux (but this is the opposite thing wrt naming... idk). A lot of code was hardcoded for ethN (and still is in some surprising places) and that hasn’t really changed. There is however a way to get back the old behavior on systemd systems (and on other systems, but it can vary) by adding net.ifnames=0 to the kernel args.

(Sorry for the second-pass rant, but I don’t think my first one was very constructive.)

[skors]

It seems this change makes the --scale option useless. because the random postfix must be known to reach scaled instances of a service.

For example, multiple service instances as upstream servers for Nginx.

Edit: see #6365 for further informations

[dawnerd]

Wow, what an awful change. 👎

How are we supposed to get the randomly generated string for use in scripts?

[saramburke]

I ran into this issue as well and would like to express my recommendation for future backwards incompatible changes. We too have scripts relying on the project_service_index format, with many people using those scripts on Mac. In an ideal world, we would be able to control the version of Docker for Mac that people use, but for now people upgrade when the auto-upgrade dialog appears.

My issues and recommendations are:

1) The upgrade dialog had no indication of this significant change, so we had no obvious heads up on this change. In an ideal world we would check all the release notes for each upgrade, but we don’t at the moment. So, it would be greatly appreciated if something significant like this was called on in the update dialog explicitly.

2) Due to 1), there was no obvious warning. It would be great if a change like this was called out in the previous upgrade. For example, the upgrade before the latest would say something like “container naming scheme is changing in the next version, please upgrade your scripts”

3) I understand that, after reading this thread, the naming scheme we are using isn’t guaranteed, and I realize there are better ways we can be referring to containers. However, we all have busy work lives and need to plan out our tasks, so as the maintainer of our scripts, I’m not able to convert our use of container names to something better right away. I am happy to use the recommended best practice, but need time to migrate our code. Therefore, it would be great to have a deprecation strategy for a change like this.

The key takeaway here is most of the world assumes a container naming scheme that is fundamental to their use of docker compose, and changing the default behavior without an obvious deprecation strategy can be detrimental to these workflows.

People don’t always use software exactly the way it was intended, and the ownership is on those people to fix things when their assumptions fail. However, a few simple communications can help us prepare for future changes and would help motivate us to move to the latest best practices.

[jtreminio]

If you previously depended on docker container exec -it project_php_1 bash, you can no longer use this.

You have to use docker ps to find the service ID.

However, you can't use docker ps -q --filter=name=project_php because that'll match on services named project_php and project_php_xdebug or anything else that left-matches project_php.

The previously-readable docker container exec -it project_php_1 bash now have to become this:

docker container exec -it \
    $(docker ps -q \
        --filter label=com.docker.compose.project=project \
        --filter label=com.docker.compose.service=php) \
    bash

This was a poorly thought-out change.

[jonnrb]

@jtreminio FWIW, from the project you can still do docker-compose exec php

[shin-]

Thanks to everyone who took time to share their thoughts on the change. We agree this was poorly handled and a bit overzealous on our part, and it has been partially reverted in Compose 1.23.2 (we're keeping random suffixes for containers created by docker-compose run, which lets us handle these more gracefully: #4688 #4549 #5240, as was initially intended).

Let me know if there are any outstanding issues that need to be addressed.

[rwlogel]

Is there any plan to add either a command line option to turn this off --no-slug or more preferably --slug so that the default is to work like it used to.

[NicolasDorier]

1.23.2 revert to how it was before.

[rwlogel]

It was only reverted for docker-compose up but not for docker-compose run

[saramburke]

Thank you for the fast resolve of this!!

On Nov 28, 2018, at 8:09 PM, Joffrey F [e-mail] wrote:

Thanks to everyone who took time to share their thoughts on the change. We agree this was poorly handled and a bit overzealous on our part, and it has been partially reverted in Compose 1.23.2 (we're keeping random suffixes for containers created by docker-compose run, which lets us handle these more gracefully: #4688 #4549 #5240, as was initially intended).

Let me know if there are any outstanding issues that need to be addressed.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[jcrben]

@shin- I presume that this will be returned at some point - possibly in 1.24 (since it is such a breaking change?). If so, could you work with the community to understand and recommend alternative lightweight mechanisms for people who were using these without slugs? Not sure about the best place to have that conversation.

[shin-]

https://github.com/docker/compose/issues/6316#issuecomment-442666873

[abdennour]

changed from <project>_<service>_<index> to <project>_<service>_<index>_<slug>

This is a bug not a feature.
Thanks!
Think about ansible playbooks that are running with ansible_connection=docker... 😔!!
should we give explicit container_name ? or shall we keep updating our inventory with the random <slug> !!.
Really Very bad!