After releasing Consul 1.4.1 we discovered that there were problems with Nomad's integration with Consul. After further digging we discovered a couple problems.
The symptom of the problem first manifest themselves with the api.Agent.Checks function of previous versions of the API client being unable to decode the JSON responses coming back from Consul. The root cause of this was that in 1.4.1 Consul was now sending back some empty strings for the time.Duration fields in the HealthCheckDefinition struct. Previously if the fields were empty they were not omitted. On the client side, the empty strings were unable to be decoded into the time.Duration values.
The fix for the first problem will be to omit those empty string fields just as we did before.
As we looked into this issue we found a secondary issue which was that the types of the fields in the 1.4.1 API client had been changed from an api.ReadableDuration to a raw time.Duration. As this could cause issues with consumers of the API client, the change will be reverted to maintain compatibility with older versions.
I am literally half way thru the upgrade to 1.4.1 and we use nomad as well. Should I roll back to 1.4.0?
@memelet Yes, you should stay on 1.4.0. There are some bugs fixed in 1.4.0 but they may or may not affect you. If you were already successfully running 1.4.0 I'd recommend that. If you were running 1.3.x I would recommend staying on that version until 1.4.2 next week.
I was coming up from 1.2.0. I've already downgraded one cluster to 1.4.0. Doing the second now. The production cluster can remain at 1.2.0 until next week. Thanks!!
This is fixed in https://github.com/hashicorp/consul/pull/5276 and is part of 1.4.2, which has been released.
This fix worked for me. Thank you for the rapid turn around!