Amazon-ecs-agent: Tune SIGKILL timeout on a per ECS Task/Container Definition basis

[CpuID]

Summary

Tune SIGKILL timeout on a per ECS Task/Container Definition basis, as opposed to Container Instance wide.

Description

Right now you can use an environment variable on the ECS Agent to tune the SIGKILL timeout sent for docker stop operations under the hood. This is ECS Agent wide, it would be extremely nice to be able to do this on a per Task or Container Definition basis.

See previous request with upvotes from others: https://github.com/aws/amazon-ecs-agent/issues/126#issuecomment-255335035

[adnxn]

@CpuID thanks for bringing this to our attention. I'm tagging this as a feature request for us to evaluate.

[CpuID]

thx @adnxn any idea if this is on the nearish term roadmap...? (2018~)

[ellenthsu]

@CpuID - we need to evaluate the priority of this feature request, at this point in time, we cannot commit to a specific timeline. We'll update this thread once we prioritize this request. Thanks for understanding!

[CpuID]

thx @ellenthsu

[themsaid]

With the EC2 launch type this can be configured using an environment variable, however with the fargate launch type there's no way of configuring this which was shocking given that fargate seems like the perfect choice for running a queue worker container, but the limit being too low may kill a long running job forcefully.

[CpuID]

With the EC2 launch type this can be configured using an environment variable, however with the fargate launch type there's no way of configuring this which was shocking given that fargate seems like the perfect choice for running a queue worker container, but the limit being too low may kill a long running job forcefully.

Note, this issue is specifically about more granular tuning (vs global) on the EC2 launch type. I would assume adding support for SIGKILL tuning for Fargate might deserve it's own GitHub issue? Sounds like a valid feature request though :)

[coultn]

For everyone interested in this issue, I wanted to let you know that the ECS team is aware of it, and that it is under active consideration. +1's and additional details on use cases are always appreciated and will help inform our work moving forward.

[cha7ri]

Any updates on this issue? we have a use case where we need to stop gracefully a task but it needs more 30s to finish its work!
I know I can use the placement strategy to launch this service in a specific instances, but this will add more complexity and will not allow us to optimize the resources usage.
It will be really nice to be able to use the task definition to configure the stop timeout.

[adnxn]

@cha7ri: we're planning to implement this as part of the container ordering. here's the proposal https://github.com/aws/containers-roadmap/issues/123

[adnxn]

closing issue, changes have been merged to dev with #1849 and expected to be released with agent v1.26.0