Openshift-ansible: Could not resolve host: github.com; Unknown error

[sozercan]

Description

Using Origin BYO playbook, I am getting this issue when trying to start a new build that clones a repo:

Cloning "https://github.com/sozercan/nodejs-ex.git" ...
--
  | error: build error: fatal: unable to access 'https://github.com/sozercan/nodejs-ex.git/': Could not resolve host: github.com; Unknown error

Looks like same issue as this: https://github.com/openshift/origin/issues/10139

Version

Please put the following version information in the code block
indicated below.

ansible 2.3.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/sozercan/openshift-ansible/library/']
  python version = 2.7.5 (default, Nov  6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]

[sozercan]

Looks like a name server issue. If I add --server 8.8.8.8 as upstream dns to dnsmasq.service, it works externally.
But when it is trying to deploy to docker registry I get

error: build error: Failed to push image: Get https://docker-registry.default.svc:5000/v1/_ping: dial tcp: lookup docker-registry.default.svc on 168.63.129.16:53: no such host

Interestingly, if I deploy busybox container I can use nslookup locally fine.

/ $ nslookup jenkins.test.svc.cluster.local
Server:    10.2.0.4
Address 1: 10.2.0.4

Name:      jenkins.test.svc.cluster.local
Address 1: 172.30.237.225 jenkins.test.svc.cluster.local

[mlbiam]

Any updates on this? I have the exact same issue on an oc cluster up on CentOS 7

[wenchma]

any idea ? I have the same issue:

# oc logs -f bc/nginx-example
Cloning "https://github.com/sclorg/nginx-ex.git" ...
error: fatal: unable to access 'https://github.com/sclorg/nginx-ex.git/': Could not resolve host: github.com; Unknown error

[orange00]

Try disabling UFW (uncomplicated firewall) if you are using it - solved my problems.
See /var/log/syslog file for firewall blocking network activity.

[michaelgugino]

I have seen this manifest in a few different ways.

Firstly, you need to have actual working DNS for the hostnames in your cluster. If you don't have this, and instead you're using /etc/hosts or some other method, you'll probably have some dns problems.

Secondly, some ISPs (this happened to me) will resolve bogus domains to some 'user-friendly' webpage. www.thisdefinitelydoesnotexistasdf1234notadomain.com will return some result to some bogus IP.

You can try setting 'bogus-nxdomain=some.ip.goes.here' in a file in /etc/NetworkManager/dnsmasq.d/
and reload NetworkManager. This IP will vary based on ISP or your local network settings.

The net effect of this is build pod tries to look up 'github.com', '.cluster.local' is append to it, dns query goes out for 'github.com.cluster.local' as first try, ISP responds with bogus IP, you are unable to actually connect to that IP because it's bogus, github.com proper is never tried.

[mlbiam]

So for me, this issue turned out to be related to my ISP redirecting all unknown dns requests to their search page. Once I changed my router DNS settings to go to a 3rd party dns this problem went away

[progsri]

This helped for me [htps://stackoverflow.com/questions/20430371/my-docker-container-has-no-internet/45644890#45644890

[kuzmanzs]

I have Ubuntu 18
I have changed symlink:
sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf

and I have added
Edit /etc/docker/daemon.json

{
"dns": ["10.1.2.3", "8.8.8.8"]
}

docker restarted

on host:
cat /etc/resolv.conf
nameserver 84.2.44.1
nameserver 84.2.46.1
nameserver 2001:4c48:1::1
Too many DNS servers configured, the following entries may be ignored.
nameserver 2001:4c48:2::1

on mysql-9-deploy pod:
sh-4.2$ cat /etc/resolv.conf
nameserver 172.30.0.2
search me-projects.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

It does,'t report same nameserver back and I have.

mysql-10-4rg2s
X Image Pull Back-off
0/1 0 a few seconds

and at wildfly examlple
Cloning "https://github.com/openshift/openshift-jee-sample.git
error: fatal: unable to access 'https://github.com/openshift/openshift-jee-sample.git/':
Could not resolve host: github.com; Unknown error

I have tested with enabled/disabled ufw- nochange.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[felipecruz91]

Same issue in CentOS 8.1

[narasimhamurthyk3]

same issue on Ubuntu 18.04 LTS

[dfrankes]

@felipecruz91 @narasimhamurthyk3 I have found a fix (working on CentOS, should also work on ubuntu)

Shutdown the cluster with: oc cluster down
Edit the file: openshift.local.clusterup/node/node-config.yml and set dnsIP: "" to 8.8.8.8
Edit the file openshift.local.clusterup/kubedns/resolv.conf and add

nameserver 8.8.8.8
nameserver 8.8.4.4

Also make sure you have the DNS options inside the docker config file

Edit /etc/docker/daemon.json and add "dns": ["8.8.8.8", "8.8.4.4"]

Then start your cluster with oc cluster up and now it should work fine.

[jsmiguel]

@felipecruz91 @narasimhamurthyk3 I have found a fix (working on CentOS, should also work on ubuntu)

Shutdown the cluster with: oc cluster down
Edit the file: openshift.local.clusterup/node/node-config.yml and set dnsIP: "" to 8.8.8.8
Edit the file openshift.local.clusterup/kubedns/resolv.conf and add

nameserver 8.8.8.8
nameserver 8.8.4.4

Also make sure you have the DNS options inside the docker config file

Edit /etc/docker/daemon.json and add "dns": ["8.8.8.8", "8.8.4.4"]

Then start your cluster with oc cluster up and now it should work fine.

It worked for me! Thanks! :raised_hands:

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.