Compose: Docker Compose can't scale containers with hard-coded name

[pcgeek86]

I'm not entirely sure if this should be considered a bug or a design enhancement, but here's the scenario:

1. You create a docker-compose.yml file in an otherwise empty directory
2. The contents of the file are below
3. You run docker-compose up -d to launch in the background
4. You run docker-compose scale redis=10
5. You receive an error message about the container name being already taken

version: '2'

services:
  redis:
    image: redis:latest
    container_name: redis

Now granted, I understand that a hard-coded container name was specified in the docker-compose.yml file, but shouldn't it still append a unique container number to the end of the container_name property? I guess I'm thinking of container_name being more like container_base_name rather than a singleton value.

Any thoughts?

Cheers,
Trevor Sullivan
Microsoft MVP: Cloud & Data Center Management
https://trevorsullivan.net
https://twitter.com/pcgeek86

[johnharris85]

This is by design so not a bug IMO. $ docker service scale (in 1.12) actually does work like this, and appends .1 etc... to the service name for scaled instances. I don't see any specific reason compose _couldn't_ behave like this though.

In a scenario with used-defined networks, both the service name and the container_name are resolvable via DNS. However when scaling a regular (no container_name) service just appends an incrementing index and DNS doesn't RR them, so just appending an incrementing index to the container name could work in the same way. In that scenario, container_name would exhibit the same behaviour as the service name would if a container_name wasn't set?

Is there some technical reason I'm missing here @dnephin / @aanand ?

[aanand]

Adding container_name was probably a mistake - it's antithetical to scaling.

I can't imagine a good reason to want to customise the name prefix of a specific service, while the names of its multiple containers remain out of your control. What does it achieve?

[pcgeek86]

I tend to agree -- it is antithetical to scaling, hence why I called it out. :smiley: Thanks for chiming in @aanand

[villlem]

Its actualy mentioned in doc https://github.com/docker/compose/blob/master/docs/compose-file.md#container_name

If I remember correctly reason for container_name was some users demand. And their use case does not involve scaling.

One of issues https://github.com/docker/compose/issues/1518
and relevat PR https://github.com/docker/compose/pull/1711

[dnephin]

Yes, I think that's it exactly. Compose tries to satisfy a few different use cases. Some options are essential for one use case, but don't make sense in other use cases.

Specifying a container_name is important for anyone who is trying to use the Compose file as a way to store runtime container config (instead of using a bash script), but it doesn't work if you're trying to run a service that you need to scale.

I think this issue can be closed as by design, since it's already documented.

[pcgeek86]

Closing as "by design." Avoid the use of container_name if interested in scaling out Docker Compose services.

Thanks @dnephin @aanand @villlem!! :whale:

[jdsundberg]

I am new to docker-compose -- so I could be on the wrong path ... but I was searching for a solution - and I found this issue/3722.

I am running a process (long-running) for many customers ... it is hard to distinguish which docker is for which customer.

I see the naming used as a helper.
Example: (assuming my docker-compose.yml has the below in it)
container_name:"${NAME}"

docker-compose run -e NAME=3M rails

Which would give 3M_1 as a container name

if I do
docker-compose -e NAME=3M scale=2

I would get:

3M_1
3M_2

This would make it easier to identify which containers are for which customers (as an example).

Without something like this... you would just see:

rails_1
rails_2
rails_3
etc...

Which -- requires some other strategy to find what container is for what company.

BTW -- open to other suggestions.

Thx,
-John

[villlem]

I would suggest to use project name for this purpose.

Dne st 27. 7. 2016 19:31 uživatel John Sundberg [email protected]
napsal:

I am new to docker-compose -- so I could be on the wrong path ... but I
was searching for a solution - and I found this issue/3722.

I am running a process (long-running) for many customers ... it is hard to
distinguish which docker is for which customer.

I see the naming used as a helper.
Example: (assuming my docker-compose.yml has the below in it)
container_name:"${NAME}"

docker-compose run -e NAME=3M rails

Which would give 3M_1 as a container name

if I do
docker-compose -e NAME=3M scale=2

I would get:

3M_1
3M_2

This would make it easier to identify which containers are for which
customers (as an example).

Without something like this... you would just see:

rails_1
rails_2
rails_3
etc...

Which -- requires some other strategy to find what container is for what
company.

BTW -- open to other suggestions.

Thx,
-John

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/3722#issuecomment-235659541,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFABkVotG6oHUqkEAJkyeTIePjCFYdU2ks5qZ5XggaJpZM4JJ8x_
.

[jdsundberg]

A quick read — this sounds like it work — thx.

-John

On Wed, Jul 27, 2016 at 12:33 PM, Vojta Orgoň [email protected]
wrote:

I would suggest to use project name for this purpose.

Dne st 27. 7. 2016 19:31 uživatel John Sundberg [email protected]
napsal:

I am new to docker-compose -- so I could be on the wrong path ... but I
was searching for a solution - and I found this issue/3722.

I am running a process (long-running) for many customers ... it is hard
to
distinguish which docker is for which customer.

I see the naming used as a helper.
Example: (assuming my docker-compose.yml has the below in it)
container_name:"${NAME}"

docker-compose run -e NAME=3M rails

Which would give 3M_1 as a container name

if I do
docker-compose -e NAME=3M scale=2

I would get:

3M_1
3M_2

This would make it easier to identify which containers are for which
customers (as an example).

Without something like this... you would just see:

rails_1
rails_2
rails_3
etc...

Which -- requires some other strategy to find what container is for what
company.

BTW -- open to other suggestions.

Thx,
-John

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/3722#issuecomment-235659541,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AFABkVotG6oHUqkEAJkyeTIePjCFYdU2ks5qZ5XggaJpZM4JJ8x_

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/3722#issuecomment-235660224,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAC3_R2dvTCJ5rja2QwzsFqp32Qa-z8Gks5qZ5ZsgaJpZM4JJ8x_
.

_John Sundberg_
Kinetic Data, Inc.
"Your business. Your process."

651-556-0930 I john.[email protected]
www.kineticdata.com I community.kineticdata.com

[typeundefined]

Sorry guys, I can't agree that this feature is not needed.

There is another issue connected with the container_name.
I'm using docker to deploy Spark Standalone on my dev machine and it seems like by some reason Spark doesn't allow underscores in the executor names. The underscore appears due to the docker naming strategy (so it names the nodes as spark_slave_1 even if the service name was "spark-slave").
Using container_name would be a good workaround but it breaks the ability to scale the service.

So what I would really need for this very case is to control somehow the way how Docker names the containers (i.e. whether the underscores are used, where to put the indices etc).

[villlem]

From what I understand spark use hostname to create executor name. But host name is set to container id and this does not have dashes. Is there minimal example that I can try without to need to build spark?

[typeundefined]

I'm not sure that it uses hostnames.
Without "container_name" set, my container name is reporting_spark_slave_1 while its hostname is spark-slave.

But the executor process gets named with something like executor_reporting_spark_slave_etc

Is the container name exposed somehow into the running container?

[villlem]

Container names are only visible for docker itself. But if you know how spark create its names we'd know more where problem is. But some universal control over name creation is unlikely.

[typeundefined]

It seems like I wasn't correct in my previous post.
Spark does use the hostname and it also uses the network name.

So my issue appears if the network name contains underscores (e.g. "reporting_default") and/or the hostname contains underscore. As far as I understand, the hostname will get the underscore if I scale the container and this is quite uncomfortable (though it is not critical, of course)

[villlem]

No but some of network names are container names. So you should just use
hostname only

Dne st 10. 8. 2016 13:05 uživatel Konstantin Nekrasov <
[email protected]> napsal:

It seems like I wasn't correct in my previous post.
Spark does use the hostname and it also uses the network name.

So my issue appears if the network name contains underscores (e.g.
"reporting_default") and/or the hostname contains underscore. As far as I
understand, the hostname will get the underscore if I scale the container
and this is quite uncomfortable (though it is not critical, of course)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/3722#issuecomment-238835250,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFABkQ46e9sOR8TskiKcbQFXa4-QIJ3yks5qebCAgaJpZM4JJ8x_
.

[alwaysastudent]

Can we be able to capture the dynamically composed container_name in other environment variables ? We have a requirement to pass in the container_name into our container as an -e argument on startup.

[jtoberling]

I faced this problem also - finally I come to this solution - maybe it's useful for somebody else too:
(Needs ifconfig & getent in the container.)

```#!/bin/bash
NETWORKNAME=mynetworkname
SERVICENAME=myservice
MAXENTRIES=10

OWNIP=ifconfig eth0 | grep inet | head -n1 | awk '{print $2}'
echo $NETWORKNAME_$SERVICENAME "members:"
i="1" ; while [ $i -le $MAXENTRIES ] ; do
QUERYIP=getent hosts $NETWORKNAME\_$SERVICENAME\_$i | cut -f 1 -d \
declare "IP[$i]=$QUERYIP"
if [ "$OWNIP" = "${IP[$i]}" ] ; then
export AGENT_ID=$i
fi
echo " " $NETWORKNAME_$SERVICENAME_$i "MemberIp:" $QUERYIP
i=$[$i+1]
done
echo "Self: AgentID: $AGENT_ID IP: $OWNIP"

```

[titovanton]

Why don't we have patterns as a values for container_name and hostname properties? My purpose is to see shorter name in docker-compose ps and to see docker@human_readable:/...., when I get in to that with exec.
For example, I found, that we can run like docker run -h foo.fee.bar. Then inside, when I call hostname I see foo. With hostname -d -> foo.fee, with -f flag -> foo.fee.bar.
There are options how to let users use names they like. It is not determined by scaling impossibility, but naming inflexibility instead!