Compose: Service can't be reached by defined hostname

[brunoborges]

Given the following docker-compose.yml file:

version: '2'

networks: 
  mynet:
    driver: bridge

services:

  master:
    image: busybox
    command: top
    hostname: amasterservice
    networks: 
      - mynet

  slave: 
    image: busybox
    command: top
    networks:
      - mynet

The following ping fails:

$ docker exec -ti test_slave_1 ping amasterservice
ping: bad address 'amasterservice'

[brunoborges]

Adding depends_on (in slave to master) does not fix the problem.

[jake-low]

Looking at docker inspect test_master_1, I see two aliases:

...
"Aliases": [
    "master",
    "268ef7e1e5"
],
...

The first comes from the service name, and the second is the container's short ID. I think that by default Compose doesn't create aliases based on hostnames.

May I ask what you're trying to accomplish? Why can't the hostname also be "master", and why does the slave need to ping the master by hostname rather than by service name?

[brunoborges]

Hi @jake-low, I think question is why isn't hostname defined under aliases. What is the reasoning behind this decision?

[jake-low]

My guess would be because hostname isn't guaranteed to be unique, and aliases are required to be. How would Compose handle this situation?

version: '2'
services:
  foo:
    image: busybox
    command: top
    hostname: notunique
  bar: 
    image: busybox
    command: top
    hostname: notunique
  test:
    image: busybox
    command: ping notunique

Service name and container ID are both guaranteed to be unique -- service because YAML doesn't allow multiple keys with the same name, and container ID because Docker uses a unique hash automatically.

[dnephin]

There was some discussion early on about using hostname instead of container_name as the default name used by docker, but that didn't happen (I'm not sure why).

We add aliases for service name and short id to be backwards compatible and handle the common use cases. We're also adding support for your own net-aliases in #2829

As far as I know, it's not all that uncommon for the internal hostname to be unresolvable from the outside.

We could probably make it another alias, but I'm not sure how we'd deal with conflicts with the service names.

[jake-low]

This may be a decision better left to the Engine team, as there's currently an incongruity there that needs to be solved.

When you run a container with docker run, no alias gets created for it. However, if you give it a --name, both its hostname (a random SHA) and its container name are resolvable.

It's strange that, even though Docker gives every container a unique ID and nickname, neither is by default DNS-resolvable (based on my tests; someone feel free to correct me).

[brunoborges]

As a Docker Composer user, I expect that whenever I set a hostname to a service, that hostname should resolve to _any_ instance of that service. Same should apply for the service name.

If these things should not work the same way, and a user should only rely on service name, then hostname definition should not be allowed in the compose file. It just creates confusion.

[jake-low]

That's a tall order. DNS load balancing is crude and has many unusual failure cases due to DNS's hierarchical organization and client caching behaviour [1]. This is why application-level (so-called Layer 7) load balancing schemes have become popular [2].

Tutum provides DNS-level load balancing (mapped to services, not hostnames), as seen here (last paragraph), but as the rest of the article discusses, this feature is recommended for use as a second-level load balancer. Primary load balancing duties have been assigned to a purpose-built container.

[brunoborges]

If goal of Docker Compose is to make it easier for users to deploy a composition of services using Docker images and containers, then the file definition should be crystal clear.

When a user defines a hostname to a service, it will expect that hostname to be resolvable from other services, specially when these services participate in the same network (as per my example).

Since a service can have multiple instances, it is just logical that when I define a hostname to a service, that hostname resolves to multiple instances (thus using the new DNS feature in latest version).

If nothing of these can be implemented today, or should not be implemented at all, then it is a matter of clarifying the documentation to state either:

1. hostname should have restriction like container_name where one service instance only will exist.
2. hostname should not be allowed in the compose file because there is no value added to that rather creating confusion in users' mind.

[dnephin]

We should probably add a warning to the docs about using hostname. I think it is rarely useful.

[brunoborges]

We should probably add a warning to the docs about using hostname. I think it is rarely useful.

I think it should not be allowed at all.

[dnephin]

That is not backwards compatible, and while it is "rarely" useful, there are still cases where it is useful in its current form, so removing it completely is not really an option.

[brunoborges]

That is not backwards compatible, and while it is "rarely" useful, there are still cases where it is useful in its current form, so removing it completely is not really an option.

Then the feature should work as expected by users using Compose for the first time, by adding that information to the list of alises.

Could you please be more specific where, although rare, the current form is useful?

[jake-low]

That is not backwards compatible, and while it is "rarely" useful, there are still cases where it is useful in its current form, so removing it completely is not really an option.

Agreed; I use it in a couple of cases where legacy applications expect to be able to resolve themselves by their own hostname (in these cases matching hostname to the service name in the composition).

I've also used it in Kerberos environments, when the hostname on a host needs to be set to an FQDN (not necessarily unique) so that services running there can use their service principals to authenticate themselves.

IMHO this feature does work "as expected" because it works the same way it does in Docker engine. Running a container with the --hostname argument does one thing: sets the hostname. Aliases are defined by the user (with the caveat I mentioned earlier).

[brunoborges]

IMHO this feature does work "as expected" because it works the same way it does in Docker engine. Running a container with the --hostname argument does one thing: sets the hostname. Aliases are defined by the user (with the caveat I mentioned earlier).

Not really. With overlay network, the hostname set in a container is reachable by another container. So I disagree that it works as "expected". In fact, what users expect (and think about new users), is that when a hostname is set, specially in the case of Docker Compose, that hostname can be used by other containers.

[jake-low]

With overlay network, the hostname set in a container is reachable by another container.

That's not true.

$ docker run -d --hostname testing --net my_overlay_network alpine sleep 300
12e9d172d1964c2ad843f3bb4b61556eb75c0680ec95d8559fe77e617cf1371b
$ docker run --rm --net my_overlay_network alpine ping testing
ping: bad address 'testing'

What you're saying is true if you set the --name of the container, not the --hostname.

$ docker run -d --name testing --net my_overlay_network alpine sleep 300
ac8ec87ec8c2893588a8d271401deb07ca934d3d5590e6726ea17cea5166876e
$ docker run --rm --net my_overlay_network alpine ping testing
PING testing (10.0.9.3): 56 data bytes
64 bytes from 10.0.9.3: seq=0 ttl=64 time=0.525 ms
64 bytes from 10.0.9.3: seq=1 ttl=64 time=0.495 ms
64 bytes from 10.0.9.3: seq=2 ttl=64 time=0.590 ms
^C
--- testing ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.495/0.536/0.590 ms

Proof that this is a swarm, and an overlay network:

$ docker network inspect my_overlay_network
[
    {
        "Name": "my_overlay_network",
        "Id": "102259bd950d46e3d3b1694f09708dc3833d4cda80a525cde1fbd081ce5bbabd",
        "Scope": "global",
        "Driver": "overlay",
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.9.0/24"
                }
            ]
        },
        "Containers": {
            "1c1609f004bff373d2c26dd269d37411214bfb05c73607620bab286621585d27": {
                "Name": "testing",
                "EndpointID": "c2c5b645e36b52952f7744d02030b3e06e1637a781be55a93ab2eebb63849429",
                "MacAddress": "02:42:0a:00:09:02",
                "IPv4Address": "10.0.9.2/24",
                "IPv6Address": ""
            }
        },
        "Options": {}
    }
]
$ docker info
Containers: 5
 Running: 4
 Paused: 0
 Stopped: 1
Images: 4
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 2
 mhs-demo0: 192.168.99.102:2376
  └ Status: Healthy
  └ Containers: 3
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.17-boot2docker, operatingsystem=Boot2Docker 1.10.1 (TCL 6.4.1); master : b03e158 - Thu Feb 11 22:34:01 UTC 2016, provider=virtualbox, storagedriver=aufs
  └ Error: (none)
  └ UpdatedAt: 2016-02-17T01:32:30Z
 mhs-demo1: 192.168.99.103:2376
  └ Status: Healthy
  └ Containers: 2
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.17-boot2docker, operatingsystem=Boot2Docker 1.10.1 (TCL 6.4.1); master : b03e158 - Thu Feb 11 22:34:01 UTC 2016, provider=virtualbox, storagedriver=aufs
  └ Error: (none)
  └ UpdatedAt: 2016-02-17T01:32:04Z
Plugins: 
 Volume: 
 Network: 
Kernel Version: 4.1.17-boot2docker
Operating System: linux
Architecture: amd64
CPUs: 2
Total Memory: 2.043 GiB
Name: mhs-demo0

[brunoborges]

@jake-low indeed, thanks for checking. I missed that I was setting --name too on my shell scripts.

[hacknaked]

Any update on this issue?

[mafrosis]

It's long been an annoyance for me that compose only creates the DNS container name aliases when using docker-compose up, and not when using docker-compose run.

The --name trick solves it, but it feels like this shouldn't be necessary.

[Ocramius]

compose only creates the DNS container name aliases when using docker-compose up, and not when using docker-compose run.

This burnt almost 2 days of development on my end (with very cryptic failures).

Do you by chance know if docker-compose exec is also affected?

[slothkong]

May I ask what is the best practice to resolve this hostname resolving issue in 2018? Should hostname be specified in the compose file?

[barrer]

@slothkong

I solved this problem. When using driver: bridge, I use the --add-host hostname:172.x.x.x parameter in the docker run command.

The docker run needs to specify the IP (--ip) and network name (--net) for the container.

[strobox]

@slothkong
Also, do not mistake to refer service name not image name (as I did :-( ):

    mongodb:
        image: mongo

Mistake: MONGODB_URL=mongodb://mongo:27017
Correct: MONGODB_URL=mongodb://mongodb:27017

[pauldraper]

After using docker for years, this docker-compose bug is a surprise.

docker run --hostname myhostname --name myname --network=mybridgenetwork --rm myimage

and as expected the container is resolvable on somebridgenetwork by myhostname but not myname on the network

Surprisingly, in docker-compose, the container would be resolvable by myname but not myhostname on the network.

---

Side note: These are two different concepts: container names are meant for container management and scoped to a Docker instance, hostnames are meant for network management and scoped to a network.

[yuriy1999]

@slothkong
Also, do not mistake to refer service name not image name (as I did :-( ):

    mongodb:
        image: mongo

Mistake: _MONGODB_URL=mongodb://mongo:27017_
Correct: _MONGODB_URL=mongodb://mongodb:27017_

thanks bro))

[wahello]

What is the last solution to dns resolve for services with hostname or domainname in the same compose file ????

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[brunoborges]

Still doesn't work...

[stale[bot]]

This issue has been automatically marked as not stale anymore due to the recent activity.

[Bessonov]

activity occurs

[iraklisg]

This unexpected behavior still exists. The best solution I have found is this SO answer

[bayeslearner]

It's year 2020 and the issue seems to persist still. I need hostname with domainname to work. This is NOT a feature but a bug!!!
Docker-compose version 3.4

Containers are spun up by docker-compose up.

ambari-server:
    image: hdp/ambari-server
    networks:
     - dev
    hostname: ambari
    domainname: dev
...   

works:

• can be pinged by service name from other containers. i.e, ambar-server
• can ping itself by service name, i.e. ambari-server
• hostname command returns the set hostname. i.e. ambari
• can ping itself by <hostname>.<domainname> , i.e. ambari.dev

doesn't work:

• can not be reached from other container by <hostname>.<domainname>, i.e. ambari.dev
• can not be reached from ohter container by <hostname>, i.e. ambari

A work around is to use <hostname>.<domainname> as the service name but this brings out a whole bunch of other issues, e.g. this is not allowed in swarm-kit (https://github.com/docker/swarmkit/issues/2437)

[bayeslearner]

I think the following summarizes the design philosophy and the solution to this problem:

1> the hostname, domainname tag is for modifying the hosts file. It doesn't help with dns resolution except to the container itself.
2> using network aliases are the recommended solutions. You have to add it to every container but that seems to be most straightforward way to solve the problem.
3> container name as a matter fact can be thought of just another network alias.

It's year 2020 and the issue seems to persist still. I need hostname with domainname to work. This is NOT a feature but a bug!!!
Docker-compose version 3.4

Containers are spun up by docker-compose up.

ambari-server:
    image: hdp/ambari-server
    networks:
     - dev
    hostname: ambari
    domainname: dev
...   

works:

* can be pinged by service name from other containers. i.e, ambar-server

* can ping itself by service name, i.e.  ambari-server

* hostname command returns the set hostname.  i.e. ambari

* can ping itself by `<hostname>.<domainname>` , i.e. ambari.dev

doesn't work:

* can not be reached from other container by  `<hostname>.<domainname>`, i.e. ambari.dev

* can not be reached from ohter container by `<hostname>`, i.e. ambari

A work around is to use <hostname>.<domainname> as the service name but this brings out a whole bunch of other issues, e.g. this is not allowed in swarm-kit (docker/swarmkit#2437)