Spring-cloud-netflix: Eureka Client Instance Health Check is failing.

[harishkadamudi]

Please find the source code for the Issue I am going to describe.
Source Code - https://github.com/harishkadamudi/eurekaissue/tree/eurekaissue/complete

I have a Eureka Server and a Client, Client was successfully registered with Server, Intentionally I have set healthcheck url different in my eureka.client.healthcheck properties.
Because my assumption here is, If Client healthcheck is failing for any reason, Eureka client should evict itself or Eureka Server should remove such Instance. But I do not see that is happening, is this a expected behaviour ?, If so then why are we are setting healthcheck in the eureka instance. Any another way to achieve this, removing unhealthy instance?

Thanks.

[harishkadamudi]

Looks like application status is not propagating to Eureka correctly.

[harishkadamudi]

Even when application is not responding, we have observed that Eureka doesn't remove its Instance.

[harishkadamudi]

Hello, any lights on this?

[ryanjbaxter]

No yet, havent had a chance to look at it. There has been lots going on with people on the team, we have been busy with the Edgware release and preparing for an upcoming conference, we will try to get to it soon, thanks for your patience.

[fahimfarookme]

You have set eureka.instance.health-check-url, instead you have to configure eureka.instance.health-check-url-path if you want to change /heath endpoint. Note the -path in the key.

[harishkadamudi]

@ryanjbaxter , Thanks for the reply.

[harishkadamudi]

@fahimfarookme , I don't want to change the health endpoint, my question is little different. I have intentionally configured incorrect, thinking eureka instance would fail, which is not happening. That the case I am bringing here.

[fahimfarookme]

Okay, eureka.instance.health-check-url property is not being used by Eureka client to determine its health / status. Consider a scenario where you implement a custom HealthCheckHandler - then you'll have to configure the relative (.health-check-url-path) or absolute (.health-check-url) path of your custom health endpoint. Likewise it's only used if a microservice wants to publish a different health URL to its clients through Eureka Server. i.e.
_/eureka/apps_

<homePageUrl>http://instance-host:port/</homePageUrl>
<statusPageUrl>http://instance-host:port/info</statusPageUrl>
<healthCheckUrl>http://instance-host:port/heathToExaternal</healthCheckUrl>

Probably the healthCheckUrl could be used by load balancers or monitoring systems. Asgard uses healthCheckUrl to make deployment decisions.

However Spring Cloud is configured with EurekaHealthCheckHandler where it determines the health of eureka instances by aggregating status' from multiple HealthIndicators (i.e. not invoking the .health-check-url) and propagates to Eureka Server through heartbeats. That's the reason that your changing of .health-check-url did't have any impact on Eureka server dashboard.

Even when application is not responding, we have observed that Eureka doesn't remove its Instance.

Here if you mean that when your app is DOWN, Eureka server doesn't remove the instances; then it could be due to 2 reasons.

1. Server response cache and heart-beat interval. So it may take some time to reflect the instance eviction.

1. Self-preservation. So you may turn it off and try.

[harishkadamudi]

I don't think so, if those are not related, then setting status url wouldn't have been a property of eureka instance. You consider this scenario, /health is not returning anything, because of this LB is not delegating the request to that instance, However eureka dashboard is showing this instance as UP. definitely you don't want see this happening.

[harishkadamudi]

I mentioned, application is not responding, that means, application is hanged and what we have observed is eureka instance is UP. I think it shouldn't be the case if /health is resolved currently.

[harishkadamudi]

If you look at eureka health handler, there is an effort made to propagate application health to eureka instance.

[spencergibb]

Eureka doesn't make calls to /health. @fahimfarookme has described accurately how health can be propagated via the heartbeat.

[harishkadamudi]

Closing the issue based on above comments.