Nomad: Improve logging for unhealthy allocations

[publysher]

Nomad 0.6

While trying out the new deployment features, I'm running into failures due to unhealthy allocations. However, when I query Consul myself, it reports all services as healthy.

Turning on debug logging for both the Nomad client and the server is not helping me much. I'm getting a lot of

client.alloc_watcher: continuing since all checks (want 1; got 0) haven't been registered for alloc "e82c4dec-64a7-231e-d829-74026965f4ec"

messages in my Nomad client, and that's it.

It would be nice to know

• the IDs of the failing health checks
• the reason this check is regarded as failing
• in case of HTTP endpoints: the exact endpoint that is being called

[dadgar]

Hey,

That message means that the check hasn't even been registered in Consul yet. You should take a look at nomad alloc-status to see if the task has been started? You may want to consider increasing the health deadline.

[aroca]

Hey Armon!

I kinda had this same issue. I had a typo in the service check port so the check was unhealthy. I fixed that and make a plan and then a run. As you mentioned, the allocation never happened but the service check in Consul was registered and Healthy.
The only thing i saw was

```Latest Deployment
ID = cdfd4c05
Status = failed
Description = Failed due to unhealthy allocations

```Deployed
Task Group  Desired  Placed  Healthy  Unhealthy
search      1        1       0        1

I stopped the Job.. then run it, and it was allocated perfect.

Cheers!

[filip-vt]

improved visibility on _why_ an allocation gets "Failed due to unhealthy allocations" would be really helpful

[stale[bot]]

This issue will be auto-closed because there hasn't been any activity for a few months. Feel free to open a new one if you still experience this problem :+1:

[ao10]

improved visibility on _why_ an allocation gets "Failed due to unhealthy allocations" would be really helpful

Still would like to see this