Skipper: Kubernetes service ExternalName should be possible to target from ingress

[szuecs]

As a user, I want to specify an external proxy target within the ingress object.

In Kubernetes this could be done with service type ExternalName, but the skipper kubernetes dataclient does not support this, yet.

[daviddyball]

Hey @szuecs, did you ever find a way around this? We're running Skipper and I want to do exactly this, but I'm getting error during backend roundtrip: kube xxxxxxxxxxxxxxxx service: dial tcp :443: getsockopt: connection refused

[szuecs]

I am not sure if your question is related to svc externalname.
Right now you would need in skipper a kube-svc target which is running and to route the traffic somewhere else you can use custom-routes https://opensource.zalando.com/skipper/kubernetes/ingress-usage/#custom-routes. The example in the link shows how to response without proxy call directly from skipper, but instead of status(200) -> <shunt> you can also use (please test before production):

annotations:
  zalando.org/skipper-routes: |
    Method("GET") && Host("your-host-header") && Path("/") ->
    optional filters as you like ->
    "https://my.example.com/foo";

If you think I don't understand your point it might be useful to post some ingress.yaml and svc.yaml to show what you are doing. :)

[daviddyball]

Ahhh right, this bypasses my need to use ExternalName completely 😛

Currently I have this (with your additions):

apiVersion: v1
kind: Service
metadata:
  name: logs
spec:
  type: ExternalName
  externalName: logs.internal
  ports:
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
  selector: {}
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: logs
  annotations:
    kubernetes.io/ingress.class: skipper
    zalando.org/skipper-routes: |
      Host("logs.example.com") -> "https://logs.internal";
spec:
  rules:
  - host: logs.example.com
    http:
      paths:
      - backend:
          serviceName: logs
          servicePort: https

When I apply it I get this in the skipper logs:

[APP]time="2018-05-17T10:14:29Z" level=info msg="route settings, update, route: kube_default__logs__logs_example_com____logs: Host(/^logs[.]example[.]com$/) -> \"http://:443\""

And this is what I get when I try to access:

[APP]time="2018-05-17T10:15:28Z" level=error msg="error during backend roundtrip: kube_default__logs__logs_example_com____logs: dial tcp :443: getsockopt: connection refused"
[APP]time="2018-05-17T10:15:28Z" level=error msg="error while proxying, route kube_default__logs__logs_example_com____logs, status code 503: dial tcp :443: getsockopt: connection refused"
81.139.52.202 - - [17/May/2018:10:15:28 +0000] "GET /_plugin/kibana/ HTTP/1.1" 503 20 "" "curl/7.59.0" 0 logs.example.com

Any ideas?

[szuecs]

skipper can't validate the certificate for hostname logs.internal. You might have to pass your custom CA to skipper to make it possible to validate your issued certificates using your custom CA.
This will also show the failure, I guess:

% openssl s_client -connect logs.internal:443

[daviddyball]

There's no option to ignore certificate validation errors?

[szuecs]

-insecure :)

[daviddyball]

ahhh right... all of our certificate validation happens on the ELB's in AWS, so I'm not too worried about
SSL validation for proxy traffic. I'll try with that and see how I get on.

Thank you for your help. I really appreciate it 👍

[daviddyball]

Hi @szuecs I've applied -insecure and it doesn't appear to be an SSL issue. I think the issue might be that there is no destination being set on the route that gets created.

The logs are showing status code 503: dial tcp :443: getsockopt: connection refused... but shouldn't it be dial tcp <address>:443:?

[szuecs]

Can you exec into the skipper pod and list all routes:

wget -O/dev/stdout localhost:9911/routes

[daviddyball]

Sure here's the output

kube_default__logs__logs_example_com____logs: Host(/^logs[.]example[.]com$/)
  -> "http://:443";

There were a few others relating to services on-cluster, but I cut them out.

[szuecs]

@daviddyball Can you post an updated version of your current ingress object?

[daviddyball]

@szuecs here you go.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: skipper
    zalando.org/skipper-routes: |
      Host("logs.example.com") -> "https://logs.internal:443"
  generation: 1
  name: logs
  namespace: default
spec:
  rules:
  - host: logs.example.com
    http:
      paths:
      - backend:
          serviceName: logs
          servicePort: https
status:
  loadBalancer: {}

[szuecs]

@daviddyball can you please change the port in the routes annotation to 444, to see what skipper shows in the routing table (wget -O/dev/stdout localhost:9911/routes )?
It looks really strange and please also check that you did not replace some names in a wrong way. The k8s object to the route looks strange to me.

[daviddyball]

@szuecs Using this config (updated with service port to 444)

apiVersion: v1
kind: Service
metadata:
  name: logs
  namespace: operations
spec:
  type: ExternalName
  externalName: logs.internal
  ports:
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
  selector: {}
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: logs
  namespace: operations
  annotations:
    kubernetes.io/ingress.class: admin
    zalando.org/skipper-routes: |
      Host("logs.example.com") -> "https://logs.internal:443"
spec:
  rules:
  - host: logs.example.com
    http:
      paths:
      - backend:
          serviceName: logs
          servicePort: 444

I get this route

kube_operations__logs__logs_example_com____logs: Host(/^logs[.]example[.]com$/)
  -> "http://:443";

[szuecs]

Are you sure that you get only one route?
The Host() predicate is only a regexp if it's not using the annotation.

Additionally you could try to target any kind of other service type ClusterIP in your cluster, because I never tried externalName and am not sure how our code behaves. I guess the route is without IP, because ExternalName has no IP in the runtime object of endpoints or service.

[daviddyball]

Oh my god @szuecs .... I've just realised I'm running v0.9.*, so figured I'd upgrade to v0.10.7 and it's all suddenly started working 🙄 It looks like the routes only respected the very first filter declaration when scanning Ingress objects and then ignored any set on other resources.

I'm so sorry for wasting your time, but I really appreciate your assistance throughout the day

[szuecs]

@daviddyball good to know that the current version works as expected. Thanks for responding. :)

[HerrmannHinz]

hey i was running into the same issue. would love to see this fixed.
for me updating to the latest skipper simply just don't solves it.

here is my use case:

• create service for external url/ip/service (this external service is may just an ip - hard to read or recognize, maybe also comes without https)
• attach an ingress to this so it can be served via a FQDN plus SSL cert.

basically i want to use an externalName'd service + skipper as proxy for external services.

cc @szuecs

[daviddyball]

We just create [Ingress] -> [Service] -> [Remote] where Service is type ExternalName. It's working fine for us, so we didn't end up needing direct Ingress-to-ExternalName in the end

[szuecs]

@daviddyball I am not sure if you pass the externalName service, if you do:

    zalando.org/skipper-routes: |
      Host("logs.example.com") -> "https://logs.internal:443"

This is basically creating a second route from the ingress which has higher priority than the default route and the backend is a network backend to https://logs.internal:443