Docker-alpine: Apparent DNS failure in Docker image alpine:3.8, nslookup: can't resolve '(null)'

[chrisinmtown]

Seeing odd DNS behavior in Docker image alpine:3.8. I'm baffled that nslookup complains yet finds the IP address. In comparison, ping works perfectly. See below. This little test runs a docker container to resolve the name of the host VM. If I can get that working the next test will be to have the docker container resolve the name of other running containers.

Traced this back from behavior in an OpenJDK image, in which Java cannot resolve host names. I'd really prefer to use an Alpine version of a Java/JRE image, it's half the size of a non-Alpine (debian) Java/JRE image, but this network glitch is kind of a killer.

So far I've run this test under a plain Ubuntu VM running docker 17.05.0-ce and under Kubernetes running docker version 18.09.1. Same behavior in both. I know there are many external variables that might affect this so it might not be an Alpine issue at all, altho issue #255 sure seems to be related.

Would someone possibly take a minute to explain please? Thanks in advance.

me@host-dev1-vm01-core:~$ docker run alpine:3.8 nslookup host-dev1-vm01-core

nslookup: can't resolve '(null)': Name does not resolve
Name:      host-dev1-vm01-core
Address 1: 10.1.0.6
me@host-dev1-vm01-core:~$ docker run alpine:3.8 ping host-dev1-vm01-core
PING host-dev1-vm01-core (10.1.0.6): 56 data bytes
64 bytes from 10.1.0.6: seq=0 ttl=64 time=0.061 ms
64 bytes from 10.1.0.6: seq=1 ttl=64 time=0.133 ms
^C
--- host-dev1-vm01-core ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.061/0.097/0.133 ms

[bboreham]

The BusyBox nslookup, which Alpine uses, does two lookups, one for the DNS server and one for the name you asked for. This can be seen here.

In your example nslookup did resolve the name host-dev1-vm01-core to the address 10.1.0.6.

The line can't resolve '(null)' says that, at that point, it didn't know what its DNS server was.
Looking at the code that might initialize it, we see why.
Sorry, I ran out of time chasing down the reason for that. Hope this is of some help.

[chrisinmtown]

Thanks @bboreham for the note, could this failure to resolve the DNS server cause a host-name resolution failure in a Java program? I might be asking the wrong questions, but I guess I'm grasping at straws, trying to explain why in my environment & tests the Java image openjdk:8-jre-alpine (derived from alpine:3.8 image) fails but the Java image openjdk:8-jre-slim (derived from debian) works just fine.

[bboreham]

It’s not really a failure at all; it’s just a program printing something out that isn’t helpful or interesting.

What Alpine nslookup does has no bearing on what a Java program does.

[chrisinmtown]

I ran nslookup to check whether the running container is able to resolve a name to an IP address. I hear you saying, nslookup is not a reliable indicator. Please suggest a better way.

[bboreham]

I'm not saying it is not a reliable indicator, I'm saying the line where it prints can't resolve '(null)' is not related to what you want to know.

Check the return code from nslookup; ignore that line.

[chrisinmtown]

I have not yet figured out the problem, pls see below for shortest possible Java debugging material, hope this will help other people.

file ResolveHostName.java

import java.net.InetAddress;
public class ResolveHostName {
    public static void main(String[] args) throws Exception {
        if (args.length != 1)
            throw new IllegalArgumentException("Usage: program host-name-to-resolve");
        System.out.println("Resolving " + args[0]);
        System.out.println(InetAddress.getByName(args[0]).toString());
    }
}

file Dockerfile-alpine

FROM openjdk:8-jre-alpine
COPY res-host-name.jar /
ENTRYPOINT ["java", "-jar", "res-host-name.jar"]

file build.sh

#!/bin/bash
set -e -x
javac ResolveHostName.java
jar cvfe res-host-name.jar ResolveHostName ResolveHostName.class
docker build -f Dockerfile-alpine  .

[jcperezamin]

Please, include this fix in all alpine versions https://github.com/kubernetes/kubernetes/issues/56903#issuecomment-409603030

[EmpireJones]

@jcperezamin - I think that's a different issue. Also, I think that'd just be hacking in a workaround for the underlying issue, which is a linux kernel bug to be fixed in v5.0 - https://github.com/torvalds/linux/commit/4e35c1cb9460240e983a01745b5f29fe3a4d8e39 . Also, that change would break some IPv6 support.

[inter169]

1. yes, you can ignore the alpine (busybox) nslookup outputting message "... can't resolve '(null)'", it's unnecessary, if you set the second param as a NS, the msg would be gone away:

# nslookup wx.qlogo.cn 100.100.2.136
Server:    100.100.2.136
Address 1: 100.100.2.136

Name:      wx.qlogo.cn
Address 1: 203.205.142.155
Address 2: 203.205.142.154

and the relative busybox codes (e.g. alpine3.8 is using http://busybox.net/downloads/busybox-1.28.4.tar.bz2 , networking/nslookup.c, function nslookup_main) and the debugging logs with strace showed that the program tried the DNS query for the given NS server ('forward' or 'reverse' DNS query, it depends on the second NS param you gave, if the NS param was some ns ip it did reverse query .aka PTR query) first of all, you got that msg because of the empty global NS server at the initialized time.

1. my codes implements (workaround https://github.com/kubernetes/kubernetes/issues/56903#issuecomment-409603030) removed the ipv6 from default behavior to decrease that dns timed out issue, at the docker image tier, please consider the pros and cons (it worked well for me, the nodejs/java/go/c++/python application that no needs ipv6), and the perfect solution to solve the dns timed out would be others (kernel conntrack racy fixes, local node dns cache ).

I have been running most of applications (java/go/nodejs/c/c++/python) based on my customized alpine docker for 1+ years.
and highly recommend running some debugging with strace, tcpdump on the docker, you would run docker on the privileged mode to get more system capability,
can you please post some strace / tcpdump logs here?

thanks,
harper

[xbmono]

I have similar issue in our Kubernetes cluster when I try to get the ip of Kubernetes DNS server. I'm using the this docker image: nginx:1.16.0-alpine

/ # nslookup kube-dns.kube-system.svc.cluster.local
nslookup: can't resolve '(null)': Name does not resolve

Name:      kube-dns.kube-system.svc.cluster.local
Address 1: 192.168.0.10 kube-dns.kube-system.svc.cluster.local

[WeihanLi]

fixed with config below with k8s deploy

spec:
  template:
    metadata:
      labels:
        app: activityreservation
    spec:
      dnsConfig:
        options:
          - name: ndots
            value: "1"

refer to https://github.com/WeihanLi/ActivityReservation/blob/d3e4de902af70ad1c85618db8a481f7fbfe1a964/k8s/reservation-deployment.yaml for details

[bboreham]

@xbmono if you are complaining about "can't resolve '(null)'" please read my explanation at https://github.com/gliderlabs/docker-alpine/issues/476#issuecomment-457124840. It is nothing to worry about. If something else, I suggest you open a new issue.

[Antiarchitect]

Having the same issue with Alpine 3.9 (not Kubernetes case). Is there any workaround for this?

[vedmant]

I have the same problem with alpine:3.10

[sashok2k]

I have the same problem with alpine:3.10

Same for me..

[sscholl]

same here node:alpine brings this problem

[WeihanLi]

any workaround with docker here?

[ncopa]

The BusyBox nslookup, which Alpine uses, does two lookups, one for the DNS server and one for the name you asked for. This can be seen here.

This was very useful. There are apparently a different nslookup implementation available in busybox which can be enabled with CONFIG_FEATURE_NSLOOKUP_BIG and the comments there says that it is compatible with musl. I wil enable that and see if i can backport it to alpine:3.11 at least.

BTW, the official docker image has moved to https://github.com/alpinelinux/docker-alpine. Since this was a config option in upstream alpine, it would have been good if it was reported upstream to
https://gitlab.alpinelinux.org/alpine/aports

[ncopa]

This should be fixed in alpinelinux/aports@e5c984f68aabb28de623a7e3ada5a223c2b66d77 and will be available next release (alpine:3.11.3). Meanwhile, you can get it with apk upgrade -U -a

[jlegido]

A temporary workaround which worked for me: try to create the docker container in host mode:

--network host