Nomad: DNS Resolve is not working on exec driver + DNSStubListener

[kamijin-fanta]

Nomad version

Nomad v0.11.0

Operating system and Environment details

• Ubuntu 18.04.3 LTS
• systemd 237

Issue

• Cannot resolve the name from the exec driver.
• In systemd-resolved enabled environment, the symlink does not point to the correct path.

Reproduction steps

1. run systemd-resolved with StubListener
2. run nomad job
3. run dig inside the container

$ cat /etc/systemd/resolved.conf 

[Resolve]
NSStubListener=yes
# etc configs...

nomad job run dns-example-job.hcl

# nomad alloc exec ec3c343a bash

nobody@hostname:/$ dig example.com        

; <<>> DiG 9.11.3-1ubuntu1.9-Ubuntu <<>> example.com
;; global options: +cmd
;; connection timed out; no servers could be reached

nobody@hostname:/$ ls -l /etc/resolv.conf   
lrwxrwxrwx 1 root root 39 Apr 20 17:29 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

nobody@hostname:/$ cat /etc/resolv.conf 
cat: /etc/resolv.conf: No such file or directory

Job file (if appropriate)

job "dns-example" {
  group "example" {
    task "main" {
      driver = "exec"

      config {
        command = "sleep"
        args = [
          "1000000",
        ]
      }
    }
  }
}

Nomad Client logs (if appropriate)

n/a

Nomad Server logs (if appropriate)

n/a

[israellot]

Facing the same here.
My current workaround is to include a template
template { data = "nameserver 127.0.0.53" destination = "/etc/resolv.conf" }

[tgross]

So this is unfortunate:

nobody@hostname:/$ ls -l /etc/resolv.conf   
lrwxrwxrwx 1 root root 39 Apr 20 17:29 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

Looks like systemd-resolvd is symlinking that rather than writing it to disk, which means it can't get bind-mounted in the container.