Marathon: TCP/HTTP health checks not resistant to network partition

Created on 11 Apr 2016  路  5Comments  路  Source: mesosphere/marathon

Current implementation of health checks does not perform optimally during a network partition. It marks task to be killed and once the network partition heals it kills all of the tasks on the far side of the partition. Can Marathon use TASK_LOST status from Mesos to pause health checks?

Most helpful comment

@gkleiman In a perfect world, it seems like this and this are actually better. Doing it this way, would allow not having to have these sort of hacks in marathon proper, and everyone is better off!

What do you think?

All 5 comments

Marathon treats a TASK_LOST update as a task failure and will launch a new the task, so it wouldn't make sense to pause health checks.

The proper fix would be to add support for Mesos HTTP/TCP health checks once https://issues.apache.org/jira/browse/MESOS-2533 is merged.

Fixed in v0.15.x by e8ace92168573927326f64963b73bb6df88dbdd3, we still have to port this fix to 1.x

cc/ @aquamatthias

@gkleiman In a perfect world, it seems like this and this are actually better. Doing it this way, would allow not having to have these sort of hacks in marathon proper, and everyone is better off!

What do you think?

Support for Mesos health checks is in marathon now.

What version will this be in?

Was this page helpful?
0 / 5 - 0 ratings