Minikube: Access to external database?

[dstroot]

Retargeting my question:

Minikube sets up a local network on your host box so your dev box and minikube can talk to each other. If I run a container inside minikube that needs access to resources/services on the internet (not on the local machine) is there anything special that needs to be done? Or should it "just work"?

More Info:

Running busybox inside the cluster and using traceroute I see that it stops at 10.0.2.2. Now I just need to figure out what 10.0.2.2 is and why it stops there. ;)

/ # traceroute 23.99.34.75
traceroute to 23.99.34.75 (23.99.34.75), 30 hops max, 46 byte packets
 1  172.17.0.1 (172.17.0.1)  0.004 ms  0.004 ms  0.001 ms
 2  10.0.2.2 (10.0.2.2)  0.179 ms  0.217 ms  0.183 ms
 3  *  *  *
 4^C
/ #

Original Question:

Using Minikube v 6 on OSX. I have an already containerized app that talks to a database on Azure. I thought I would just be able to spin up the container inside K8S and have it work, but DNS would only have internal K8S resources, and even if I feed it the external IP it still seems that egress/ingress is causing problems because the app cannot authenticate to the external database. Can someone point me in the right direction?

I tried "Services without Selectors" (http://kubernetes.io/docs/user-guide/services/) but no joy...

[dlorenc]

Can you explain what you mean by DNS not working? What's the hostname you're trying to resolve?

[dstroot]

Thanks for the quick response. Functionally, I have a SQL database on Azure and a containerized app that updates it. The app reads environment variables to tell it where to point. Initially I tried just specifying the IP:

        - name: MSSQL_HOST
          value: "23.99.34.75"  #yms0wtdqkl.database.windows.net
        - name: MSSQL_PORT
          value: "1433"

Then I tried this approach:

apiVersion: v1
kind: Service
metadata:
  name: database
spec:
  ports:
  - port: 1433
    targetPort: 1433
    protocol: TCP
---
# Because this service has no selector, the corresponding Endpoints
# object will not be created. You can manually map the service to
# your own specific endpoints:
kind: Endpoints
apiVersion: v1
metadata:
  name: database
subsets:
  - addresses:
      - ip: "23.99.34.75"
    ports:
      - port: 1433

Environment vars used by my pod:

        - name: MSSQL_HOST
          value: database  
        - name: MSSQL_PORT
          value: "1433"

Whatever I tried so far the app inside K*S has not been able to connect to the database. I am stumped. A simple docker compose file that works looks like this:

version: '2'

services:
  web:
    image: dstroot/tpg-ttpweb
    container_name: "webserver"
    ports:
      - "80:8000"
    restart: never
    links:
      - redis
    environment:
      MSSQL_HOST: yms0wtdqkl.database.windows.net
      MSSQL_PORT: 1433
      MSSQL_USER: xxxxxxxxx
      MSSQL_PASSWORD: xxxxxxxxxx
      MSSQL_DATABASE: OLTP_SYS
      REDIS_URI: redis
      NODE_ENV: development
      PORT: 8000

  redis:
    image: redis
    container_name: "redis"
    cpu_shares: 50
    cpu_quota: 50000
    mem_limit: 256000000
    memswap_limit: 512000000
    restart: always

[jimmidyson]

Are you using virtualbox, xhyve or VMware fusion?

A few things to try to troubleshoot:

What do your application logs show?

Does kubectl endpoints database show up the correct endpoints?

If you run minikube ssh can you resolve any address from inside the vm?

If on the host you run dig @$(minikube ip) database.NAMESPACE.svc.cluster.local (replace namespace appropriately) does that resolve correctly (this will be a cluster internal address so not routable from your host)?

[dstroot]

Thanks for the reply!

I am using "standard" virtualbox. Don't even know how to switch...

Endpoints:

❯ kubectl get endpoints
NAME              ENDPOINTS          AGE
database          23.99.34.75:1433   1d      <---*
kubernetes        10.0.2.15:8443     3d
redis-commander   172.17.0.4:8081    2d
redis-master      172.17.0.2:6379    3d
ttp               172.17.0.5:3000    1d

DNS is working:

/ # dig @$(minikube ip) database.NAMESPACE.svc.cluster.local
sh: minikube: not found
sh: dig: not found
/ # nslookup database
Server:    10.0.0.10
Address 1: 10.0.0.10

Name:      database    <---*
Address 1: 10.0.0.111    <---*
/ #

Service:

❯ k describe service database
Name:           database
Namespace:      default
Labels:         <none>
Selector:       <none>
Type:           ClusterIP
IP:         10.0.0.111    <---*
Port:           <unset> 1433/TCP
Endpoints:      23.99.34.75:1433    <---*
Session Affinity:   None
No events.

Logs:

2016-07-17T16:39:49.739735824Z { [ConnectionError: Cannot open server "database" requested by the login.  The login failed.]
2016-07-17T16:39:49.739770380Z   name: 'ConnectionError',
2016-07-17T16:39:49.739774665Z   message: 'Cannot open server "database" requested by the login.  The login failed.',  <---*
2016-07-17T16:39:49.739784905Z   code: 'ELOGIN' }
2016-07-17T16:39:49.751651235Z 
2016-07-17T16:39:49.764836733Z npm info lifecycle [email protected]~start: Failed to exec start script
2016-07-17T16:39:49.765224916Z npm ERR! Linux 4.4.14-boot2docker
2016-07-17T16:39:49.765740576Z npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
2016-07-17T16:39:49.766093377Z npm ERR! node v5.12.0
2016-07-17T16:39:49.766387833Z npm ERR! npm  v3.8.6
2016-07-17T16:39:49.766837060Z npm ERR! code ELIFECYCLE
2016-07-17T16:39:49.767173007Z npm ERR! [email protected] start: `node ./server/bin/www`
2016-07-17T16:39:49.767312893Z npm ERR! Exit status 154
2016-07-17T16:39:49.767463076Z npm ERR! 

[jimmidyson]

The error code elogin looks like it can't authenticate to the mssql server, rather than DNS or network connection failing. Are you sure you have the right address & have specified the correct credentials?

This doesn't look like a problem in minikube, but rather an application configuration problem.

[dstroot]

Exactly. It can't reach MSSQL Server. However the same container, run with the same MSSQL environment variables/values, works just fine _outside_ of minikube/K8S. That's what is puzzling me. Should containers inside a K8S cluster just already have Internet egress/ingress? Or, do I have to do something to enable it?

[Draiken]

Having the same problem here.

Before using minikube I ran hyperkube directly through docker, and used the docker host machine IP to connect to my host machine's already existing database. It worked perfectly fine.

Now I can't connect to it through minikube.

Inside the machine, I can ping the host machine's docker IP, but can't connect to the database.
Here is on the host machine

telnet 172.17.0.1 3306
Trying 172.17.0.1...
Connected to 172.17.0.1.
Escape character is '^]'.
[
5.5.49-0ubuntu0.14.04.10&D`:@q:()1\\:MS8`Xxmysql_native_password

Here is on the minikube VM

docker@minikubeVM:~$ ping 172.17.0.1 -c 1
PING 172.17.0.1 (172.17.0.1): 56 data bytes
64 bytes from 172.17.0.1: seq=0 ttl=64 time=0.073 ms

--- 172.17.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.073/0.073/0.073 ms

docker@minikubeVM:~$ telnet 172.17.0.1 3306
telnet: can't connect to remote host (172.17.0.1): Connection refused

Seems like some sort of firewall. Since I can ping, I can telnet to 22 (ssh) for example, but not mysql.

I'm using the default virtualbox backend and default configurations

[dlorenc]

@Draiken can you explain what you mean by "docker host machine IP"? What exactly is that the IP of?

[Draiken]

When you install docker, it adds a docker interface. That's the IP I'm referring to. Before it was the IP I used to communicate between containers and the host to the docker engine.

Maybe I'm doing something completely wrong, but I don't believe the use-case is that uncommon. I have a local mysql running and I wanted to connect a pod running inside the minikube VM to it.

[dlorenc]

Ah ok, are you running on Linux? Minikube runs in a VM so I think that IP will point inside the VM, not to the host where your database might be running.

With Virtualbox you can usually connect to 10.0.2.2 to connect back to the host. Could you give that a try?

[Draiken]

Yes! It worked like a charm

Thanks for the help