Minikube: expose the kubernetes REST API using basic auth?

[jstrachan]

Being able to work with the kubernetes REST API from a web browser is really handy for developers learning how kubernetes works & developing against the kubernetes APIs.

Right now a bearer token needs to be passed in to use the REST API which is harder for developers on their laptops. While its still possible; its just not an awesome UX for a single VM on a developers laptop; where security isn't such a biggie.

So it'd be nice to pass in --basic-auth-file=SOMEFILE as we start the apiserver: http://kubernetes.io/docs/admin/authentication/ as we startup kubernetes

[pidah]

@jstrachan stumbled on this too, atm minikube uses client-cert authentication instead of Basic auth or Bearer token afaict.

On OS X, this is my config file:
$ cat ~/.kube/config
apiVersion: v1
clusters:

• cluster:
  certificate-authority: /Users/peteridah/.minikube/apiserver.crt
  server: https://192.168.99.100:443
  name: minikube
  contexts:
• context:
  cluster: minikube
  user: minikube
  name: minikube
  current-context: minikube
  kind: Config
  preferences: {}
  users:
• name: minikube
  user:
  client-certificate: /Users/peteridah/.minikube/apiserver.crt
  client-key: /Users/peteridah/.minikube/apiserver.key
  now grabbing those keys, you can now do the following:
  curl -v https://192.168.99.100:443 --cert ~/.minikube/apiserver.crt --key ~/.minikube/apiserver.key --cacert ~/.minikube/apiserver.crt
• Rebuilt URL to: https://192.168.99.100:443/
• Trying 192.168.99.100...
• Connected to 192.168.99.100 (192.168.99.100) port 443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
• successfully set certificate verify locations:
• CAfile: /Users/peteridah/.minikube/apiserver.crt
  CApath: none
• TLSv1.2 (OUT), TLS header, Certificate Status (22):
• TLSv1.2 (OUT), TLS handshake, Client hello (1):
• TLSv1.2 (IN), TLS handshake, Server hello (2):
• TLSv1.2 (IN), TLS handshake, Certificate (11):
• TLSv1.2 (IN), TLS handshake, Server key exchange (12):
• TLSv1.2 (IN), TLS handshake, Request CERT (13):
• TLSv1.2 (IN), TLS handshake, Server finished (14):
• TLSv1.2 (OUT), TLS handshake, Certificate (11):
• TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
• TLSv1.2 (OUT), TLS handshake, CERT verify (15):
• TLSv1.2 (OUT), TLS change cipher, Client hello (1):
• TLSv1.2 (OUT), TLS handshake, Finished (20):
• TLSv1.2 (IN), TLS change cipher, Client hello (1):
• TLSv1.2 (IN), TLS handshake, Finished (20):
• SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
• ALPN, server accepted to use h2
• Server certificate:
• subject: CN=minikube
• start date: Jul 11 16:59:44 2016 GMT
• expire date: Jul 11 16:59:44 2017 GMT
• subjectAltName: host "192.168.99.100" matched cert's IP address!
• issuer: CN=minikube
• SSL certificate verify ok.
• Using HTTP2, server supports multi-use
• Connection state changed (HTTP/2 confirmed)
• TCP_NODELAY set
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• Using Stream ID: 1 (easy handle 0x7ffcac00c400)

GET / HTTP/1.1
Host: 192.168.99.100
User-Agent: curl/7.49.1
Accept: _/_

• Connection state changed (MAX_CONCURRENT_STREAMS updated)!
• HTTP 1.0, assume close after body
  < HTTP/2 200
  < content-type: application/json
  < content-length: 533
  < date: Mon, 11 Jul 2016 17:26:09 GMT
  <
  {
  "paths": [
  "/api",
  "/api/v1",
  "/apis",
  "/apis/apps",
  "/apis/apps/v1alpha1",
  "/apis/autoscaling",
  "/apis/autoscaling/v1",
  "/apis/batch",
  "/apis/batch/v1",
  "/apis/batch/v2alpha1",
  "/apis/extensions",
  "/apis/extensions/v1beta1",
  "/apis/policy",
  "/apis/policy/v1alpha1",
  "/apis/rbac.authorization.k8s.io",
  "/apis/rbac.authorization.k8s.io/v1alpha1",
  "/healthz",
  "/healthz/ping",
  "/logs/",
  "/metrics",
  "/swaggerapi/",
  "/ui/",
  "/version"
  ]
• Closing connection 0
• TLSv1.2 (OUT), TLS alert, Client hello (1):

Then you can convert the key apiserver.key to .p12 format and import it along with apiserver.crt into your system keychain (OS X), then you can hit the API from any browser.

Not Ideal.

[jezell]

We ran into a similar problem since our code was using token auth against hyperkube / gke.

If you are using Go, Kubernetes officially supported client can be used inside minikube, which is how we worked around the problem. Here's a code sample from the Dashboard that works with minikube for auto configuring the client from inside a kube cluster:

https://github.com/kubernetes/dashboard/blob/469903ca89f5870cfe4d0a45a84c47d1ce54f9c6/src/app/backend/client/apiserverclient.go

From what I gathered while looking into this, doesn't look like it would be that hard to get bearer token support in minikube itself. It's just a matter of generating a token file like hyperkube does (https://github.com/kubernetes/kubernetes/blob/master/cluster/images/hyperkube/setup-files.sh) and passing it in the constructor here:

https://github.com/kubernetes/minikube/blob/master/pkg/localkube/apiserver.go

See param TokenAuthFile:

https://github.com/kubernetes/kubernetes/blob/master/pkg/genericapiserver/options/server_run_options.go

[bgandon]

Alternative workaround: here is a Bash snippet to enble the 8080 insecure port of the localkubedaemon inside the boot2docker VM that runs minikube.

minikube_ip=$($minikube ip)
function ssh_command() {
    ssh \
        -i "$HOME/.minikube/machines/minikube/id_rsa" \
        "docker@$minikube_ip" \
        "$@"
}
ssh_command 'sudo killall localkube && rm -f /var/run/localkube.pid'
ssh_command \
    "sudo sh -c 'PATH=/usr/local/sbin:\$PATH nohup /usr/local/bin/localkube \
        --apiserver-insecure-address=0.0.0.0 \
        --generate-certs=false --logtostderr=true --node-ip=$minikube_ip \
            > /var/lib/localkube/localkube.out \
            2> /var/lib/localkube/localkube.err \
            < /dev/null \
            & echo \$! > /var/run/localkube.pid &'"

This works great when run right after a minikube start and enables the Ansible kubernetes module to manage minikube resources (specifying api_endpoint: <Minikube-IP>:8080 and insecure: yes as task settings).

[r2d4]

You should be able to pass in arbitrary flags to the different kubernetes components once #529 is merged in

[jimmidyson]

Just be aware that any insecure port or default username/password for basic auth will open you up to CSRF attacks - @dlorenc mentioned this previously IIRC in another issue.

[bgandon]

Looking forward to #529 being shipped, then.

Indeed for security. This is meant to be used for local development and prototyping only.

[dlorenc]

529 should be shipped now. You should be able to set arbitrary values on Kubernetes components, allowing this to work. Let me know if you have trouble figuring out the right flags!

[joshdover]

I haven't been able to figure this one out, starting minikube with the following command:

minikube start --extra-config apiserver.InsecureBindAddress=0.0.0.0

And then running this command (on my local machine, not the VM) does not seem to work, though it seems like it should:

curl $(minikube ip):8080

When looking at the minikube logs, I get this error: Error: Unable to set type 127.0.0.1.

[dlorenc]

Hey,

I think you're close. Could you try InsecureBindAddress? This system works off reflection, so you'll need to use the go struct naming.

[joshdover]

Tried that too, that's when I got the Unable to set type error I posted above (I figured editing the message was better than sending everyone another notification).

Think the coercion into other types fails on some strings (like 127.0.0.1). Seems like a better default would to cast as a string rather than just skipping.

[r2d4]

This is a known bug in the configurator. It only works on basic field types for now
https://github.com/kubernetes/minikube/blob/master/pkg/util/config.go#L46

We should add support for more types, such as net.IP

We could reopen this bug or create a new one. This behavior is not obvious from the docs and it would be good to have it documented in an issue

[sbourlon]

Another solution is to use SSH port forwarding:

$ curl $(minikube ip):8080
curl: (7) Failed to connect to 192.168.95.158 port 8080: Connection refused

$ ssh -f -i $HOME/.minikube/machines/minikube/id_rsa docker@$(minikube ip) -L 8080:localhost:8080 -N

$ curl localhost:8080/apis
{
  "paths": [
    "/api",
    "/api/v1",
    "/apis",
[...]