Swarmkit: Hostname is not resolvable if created by template in swarm compose

[m4r10k]

Given the following working swarm compose yml results in not resolvable hostnames on the overlay network using the swarm DNS service discovery as usual.

version: "3"

services:

  app:
    hostname: "{{.Service.Name}}-{{.Task.Slot}}"
    depends_on:
      - redis
      - minio
    networks:
      - net
    image: registry:latest
    volumes:
      - /home/mario/reg/registry-conf/config.yml:/etc/docker/registry/config.yml
    deploy:
      restart_policy:
        condition: any
      mode: replicated
      replicas: 3
      update_config:
        delay: 2s

  redis:
    networks:
      - net
    image: redis
    command: redis-server --appendonly yes
    volumes:
      - /home/mario/reg/redis-data:/data
    deploy:
      restart_policy:
        condition: any
      mode: replicated
      replicas: 1
      update_config:
        delay: 2s

  minio:
    ports:
            - 9000:9000
    image: minio/minio
    command: server /export
    networks:
      - net
    volumes:
      - /home/mario/reg/minio-data:/export
      - /home/mario/reg/minio-conf:/root/.minio
    environment:
      - MINIO_ACCESS_KEY=B2E0E27AAAAAAAAJATWZ
      - MINIO_SECRET_KEY=6f1yeVsuf4c9aaaaaaaaxgf3QBar8hrsKHj2ryRE
    deploy:
      restart_policy:
        condition: any
      mode: replicated
      replicas: 1
      update_config:
        delay: 2s

networks:
  net:
    external:
      name: registry_net

Here is the evidence:

[09:26 mario@ ~/reg]$ docker stack deploy -c registry-swarm.yml registry
Creating service registry_app
Creating service registry_redis
Creating service registry_minio
[09:26 mario@ ~/reg]$ docker exec registry_app.1.pf1ia54eyankjqo0y0egs43wh hostname
9483c153494b
[09:27 mario@ ~/reg]$ docker exec registry_app.2.t7lmvfgfyz9mtoinesawvnsad hostname
1c3b1054d421
[09:27 mario@ ~/reg]$ docker exec registry_app.1.pf1ia54eyankjqo0y0egs43wh ping 1c3b1054d421
PING 1c3b1054d421 (10.0.1.6): 56 data bytes
64 bytes from 10.0.1.6: seq=0 ttl=64 time=0.157 ms
64 bytes from 10.0.1.6: seq=1 ttl=64 time=0.137 ms
^C

changed the swarm compose to use hostname: "{{.Service.Name}}-{{.Task.Slot}}" for hostname:
hostname: "{{.Service.Name}}-{{.Task.Slot}}"

[09:27 mario@ ~/reg]$ docker stack deploy -c registry-swarm.yml registry
Updating service registry_app (id: js6hzq1zvtjzlgf01rnqc5u45)
Updating service registry_redis (id: luha9payd14qotjnk3r2eqbxv)
Updating service registry_minio (id: x15ubn9ctxjuo64j0qapfmyoh)
[09:27 mario@ ~/reg]$ docker exec registry_app.1.oc3eudiz86attr7kktjk0jjdl hostname
registry_app-1
[09:28 mario@ ~/reg]$ docker exec registry_app.2.w4mjpo46ikov2xkero552r6if hostname
registry_app-2
[09:28 mario@ ~/reg]$ docker exec registry_app.1.oc3eudiz86attr7kktjk0jjdl ping registry_app-2
ping: bad address 'registry_app-2'

Expected result:

The hostnames of the containers should be registered correctly in the swarm DNS service discovery during startup. It does also not work if I recreate the stack (rm, deploy)

Importance:

I guess that this is important because the correct behavior is needed for cluster services like Redis/Minio/etc which should be deployed through stack but which are depending to use stable DNS resolvable hostnames during the cluster configuration. IP-address are subject of change during redeployment's and these IP-addresses are not predictable beforehand - therefore ip-addresses are not usable in these scenarios (cluster configuration).

Docker version

Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:23:31 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:19:04 2017
 OS/Arch:      linux/amd64
 Experimental: false

[m4r10k]

Also setting the network alias does not change anything. No error, but no new alias. But I can ping the given one from any container.

    networks:
      net:
        aliases:
          - "{{.Service.Name}}-{{.Task.Slot}}"

            "Networks": {
                "registry_net": {
                    "IPAMConfig": {
                        "IPv4Address": "10.0.1.5"
                    },
                    "Links": null,
                    "Aliases": [
                        "c7d5b7a82724"
                    ],
                    "NetworkID": "uoufdo1bwnepruvuk594jmiun",
                    "EndpointID": "bfd860f6a532b5ee78d352edb280f4c049cf8464cd742122bdcf31e471376c40",
                    "Gateway": "",
                    "IPAddress": "10.0.1.5",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:0a:00:01:05"
                }

[10:09 mario@ ~/reg]$ docker exec registry_app.2.6lhya7ftxoq6lna7c672gx5ta ping c7d5b7a82724
PING c7d5b7a82724 (10.0.1.5): 56 data bytes
64 bytes from 10.0.1.5: seq=0 ttl=64 time=0.173 ms

[m4r10k]

Feature Request opened at moby/moby#34239

[nishanttotla]

@kleinsasserm since this issue is being tracked and discussed elsewhere, can we close this one?

[m4r10k]

@nishanttotla As this issue is covered as closes due commit docker/libnetwork@61f3d913, this one would be closed automatically if the PR is merged. My PR there is quite simple (two 2 liner) but very useful and it would be nice if you could support (+1) the merge there to get this done. This would be very helpful.

I've opened this issue here, because at first I thought that this is a swarm issue. Sometimes later I researched, that the source of this behavior is technically in libnetwork. I guess that others would also try to search here then elsewhere. Another thing is, that the default search view on Github covers only issues with status open. Therefore it is sometimes hard to see if something is merged or solved or wip for inexperienced users if issues are closed.

But if you think so you can close this one :)

[nishanttotla]

@kleinsasserm thank you for the detailed explanation and context. Since issues are searchable even when closed, let's close this one, and track any follow-up discussion in libnetwork.

[thaJeztah]

Linking https://github.com/docker/swarmkit/issues/1242, which is where the global design for a DNS schema is being discussed