jx should annotate the deployed Ingress with `kubernetes.io/ingress.class: "nginx"` when the nginx ingresscontroller is used

[jody-frankowski]

The gce ingress controller only supports NodePort services, however there is also a bug in the gce ingress controller which makes it stop evaluating Ingress objects when it fails for one.

Since jx doesn't annotate its Ingress object with kubernetes.io/ingress.class: "nginx", and the backend services are of ClusterIP type, this effectively forbids us to use the gce ingress controller.

Note in the following example that the service is of type NodePort and should work with the gce ingress controller (the following works when I set kubernetes.io/ingress.class: "nginx" on all the ingress deployed by jenkins-x):

 ❯ kubectl get svc
NAME           TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
hello-server   NodePort   10.15.242.10   <none>        8080:30489/TCP   3h

 ❯ kubectl get ingress hello-server -o yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: gce
[...]
spec:
  rules:
  - host: hello-server.jx.[...]
    http:
      paths:
      - backend:
          serviceName: hello-server
          servicePort: 8080

 ❯ kubectl describe ing hello-server
[...]
Events:
  Type     Reason  Age   From                     Message
  ----     ------  ----  ----                     -------
  Warning  GCE     4m    loadbalancer-controller  googleapi: Error 400: Invalid value for field 'namedPorts[0].port': '0'. Must be greater than or equal to 1, invalid

The annotation should also probably be set by default for any ingress deployed by jenkins-x when the nginx ingress controller is installed.

Ping @vgallissot

[rawlingsj]

@jody-frankowski we have a controller called exposecontroller which runs as a helm hook and generates the ingress rules. I guess this should take some config, maybe defaulting to nginx that indicates the kubernetes.io/ingress.class annotation to add to all ingress rules that it creates. Would that work?

[jody-frankowski]

Yes, I think that's the proper solution.

We just need to be mindful of two things:

• Only annotate when the user has elected to install the nginx controller
• What about people that modify the chart to configure their own ingress (but one that still relies on the nginx controller)? Would that be a possible use case? If so the annotation would need to be documented somewhere.

[randymay]

Is there a workaround for this until it is fixed?

[chrisferry]

Any news on this? We use multiple ingress controllers so we NEED that annotation.

[rawlingsj]

Sorry this fell off my radar. I'll get that annotation kubernetes.io/ingress.class: "nginx" added.

[rawlingsj]

I forgot to reference the full repo for this issue in the commits so linking the fixes here

https://github.com/jenkins-x/draft-packs/pull/35
https://github.com/jenkins-x/jenkins-x-platform/commit/9c7a501bba96f0a46e39e5fb12e9da370462caf6

We'll need to wait for the BDD tests to run and pass before the new release is available. That will be triggered now and will likely release tomorrow providing all things are well.

[rawlingsj]

I'll update and close this issue once the new release is available.

[rawlingsj]

The release with this fix is now available.

[JacobHeidelbachDFDS]

Can the annotation be overwritten, in case you have another Ingress Controller configured? (Eg. Traefik)
Update: Added custom myvalues.yaml file for environment which fixed it :)

Would it make sense to parameterize this ingress.class through jx?

[christianmue1980a]

@JacobHeidelbachDFDS , what is the syntax to override the "kubernetes.io/ingress.class" annotation via myvalues.yaml?