Amazon-ecs-agent: Support for --stop-timeout

[simplesteph]

Now that the main ECS Linux has been updated to Docker 17.03, we have the capability of specifying a timeout per task using:

docker run -d --stop-timeout=20 alpine sleep 500
time docker stop 187626bd862f
187626bd862f
docker stop 187626bd862f  0.01s user 0.01s system 0% cpu 20.711 total

Can ECS please support that?

[adnxn]

Hi @simplesteph, thanks for bringing this to our attention. This will require work on the agent codebase, as well the as the ECS API's task definition. Unfortunately we can't commit to a delivery date right now, but we will continue to track it as a feature request. Thanks.

[TheTweak]

Seems to be supported now through ECS_CONTAINER_STOP_TIMEOUT http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html

[jahkeup]

Hey @TheTweak! Thanks for the pointing out that configuration environment variable. ECS_CONTAINER_STOP_TIMEOUT does cause the Agent to kill containers after timing out, but this configuration setting is global and cannot be set or overridden on a per-task basis. If I understand @simplesteph correctly, it would be desirable to configure each Task independently to give containers more or less time to stop.

[coen-hyde]

Yes please. A per Task parameter for the stop timeout would be useful.

[Halama]

Per Task parameter would be useful especially with Fargate where you don't have a control over environment.

[coultn]

I believe this is the same issue as https://github.com/aws/amazon-ecs-agent/issues/1241 is it not? Either way, I wanted to let everyone on this thread know that we on the ECS team are aware of the issue and that it is under active consideration. If you have additional details/use cases to share, they are always appreciated and will help inform our work.

[deeptechs]

I guess there is a misunderstanding. @simplesteph described the timeout for a task even if it is running. I need also such this feature. I know this feature is strange regarding ecs logic.. Just I want to limit a task life.

ECS_CONTAINER_STOP_TIMEOUT is yes global but also it is used for stopped task and defining the time period from a stopped task to a starting container kill.

[samuelkarp]

@deeptechs The --stop-timeout parameter that @simplesteph referenced is the same thing as ECS_CONTAINER_STOP_TIMEOUT but at a per-container level instead of global. It sounds like you're describing a maximum-lifetime for a container, which would be a separate feature request.

[deeptechs]

@samuelkarp Thanks for the clarification, you are right this can be a separate feature which we shouldn't request since we can manage any task via api. We are now handling a running-task life in our app side. By the way we will try to handle Sigterm signal during ECS_CONTAINER_STOP_TIMEOUT to stop docker containers gracefully.

[adnxn]

im closing this in favor of https://github.com/aws/amazon-ecs-agent/issues/1241. both issues are tracking granular per container stop timeouts. given the current proposal, we would be able to set "timeout per task" by configuring the each container in the task to have the same timeout.

please add any feedback to the other issue tracking this feature request.