Generator-jhipster: K8s using ingress is not working on GKE - gateway keeps unhealthy

[marcelinobadin]

Overview of the issue

My project is built using v4.6.2, microservices architecture and deployed using Kubernetes on Google Cloud (Container Engine - GKE).

GKE already has a ingress controller so there is no need to deploy a new one.

The problem is, the ingress configuration keep saying me that the gateway (microservice) is 'unhealthy' and because of that I keep getting 502 error when I try to access the application.

Motivation for or Use Case

Reproduce the error

Create an application foo/bar like using microservices, deploy it using kubernetes on a cluster running on Google Cloud (use Container Engine) using Ingress option.

You will not going to be able to connect on your application. You will get 502. This happens because the service is getting noticed that it is unhealthy.

Probably is something with the liveness and readiness probes but I could not figure out how to solve it yet.

(also, I had to manually adjust jhipster-registry to work - but there is a pull request ready to solve it)

• [x] Checking this box is mandatory (this is just to show you read everything)

[xetys]

Hey,

depending on the cluster size, it might be needed to increase the initialDelaySeconds from the liveness probe.

You should check the health of the gateway pods by looking into logs and registry registers.

For understanding, both probes have a default of 10s frequency and failure threshold of 3. So Your startup time must be less then initialDelaySeconds + 30s. Otherwise k8s will kill the pod before it boots up.

[marcelinobadin]

Thank you, I did it. I had to change the JHIPSTER_SLEEP to all the microservices came up without killing my cluster and because of that I did the setup accordingly, without it the liveness probe would kill everything (I have another issue with the horizontal pod autoscaling but it is for another time).

Apparently the problem is on the health check that ingress creates to the L7 load balancer( (it creates two health checks, one is ok, the one that depends on the gateway check, the other which relies on L7 is the problem). I tried to change its configuration but nothing works. My only option by now is to use LoadBalancer but Ingress is a better option for my use cases.

And of course, my lack of experience (with ingress) probably is a major factor on this.

[marcelinobadin]

I found a solution! On the gateway service, we are using type: ClusterIP, I had changed it to NodePort and it worked fine (after 3 days digging on this!).

[pascalgrimaud]

@marcelinobadin @xetys : is it resolved here ? Or is there a need to PR ?

[marcelinobadin]

Needs a PR but before should be nice to hear from someone who has more knowledge on Kubernetes and tell that it is ok to move from ClusterIP to NodePort.

[PierreBesson]

Yes Marcelino, it seems using a nodePort is required to use the GKE ingress : https://stackoverflow.com/questions/37299646/are-ingress-working-with-clusterip-services
We should document this.

[marcelinobadin]

Great @PierreBesson, that's the answer, so no need for a PR. And to not let it pass, I saw you working on Kubernetes generator since the beginning and it become a great job! Congratulations! Nice work you have done.

[PierreBesson]

Thank you, but really all the credit for getting k8s out of beta should go to @xetys and @srinivasa-vasu.
I'm closing this issue and will await your PR to the docs.

[marcelinobadin]

Thank you @xetys and @srinivasa-vasu for your excellent work on the kubernetes generator!

My ingress configuration also had to add:

annotations:
ingress.kubernetes.io/add-base-url: /

without it I just got 404 from all app/* files.

I will try to create a PR for documentation.

[PierreBesson]

@marcelinobadin I think deployment should work in GKE out of the box. Could you PR directly to the generator instead.