Amazon-ecs-agent: Restart container when source image changes

[radenui]

Hi,

Today, when a Docker image changes, we need to either create a new task definition with the same parameters and perform a rolling update, either restart each task one by one.

I would like to be able to automatically perform the rolling update when a newer version of the source image is found.

Suggested behavior (on services):

• Each ecs-agent running a container would monitor the corresponding docker images
• If a newer version of the image is found, It would send this information back to ECS server so it can perform a rolling update using the same task definition with the newer Docker image.

That would be awesome!
Thanks,

Arthur

[euank]

The sort of rolling update you're describing sounds a lot like what would typically done by a build pipeline, such as Jenkins or AWS CodePipeline.

We have a blog post talking about such a pipeline for ECS with Jenkins.

To me, it makes more sense to get the sort of behavior you describe by having a new Docker image trigger an event in a build pipeline leading to a new deployment with a new task definition. By moving it up a layer, each instance doesn't have to constantly poll for new images.

One of the purposes of being able to do a rolling update is to handle failures — to be able to roll back. If you publish a new task definition with a new revision and image tag for each update (see the above build pipeline post for an example there), you retain the ability to roll back if a new revision has issues. If you also try to rely on image changes for updates/versioning, you can't go back. There's a little more discussion around this in this forum thread

Ultimately, that behavior sounds like it fits in a build pipeline that lives outside ECS.

I'd be happy to discuss further of course.
Best,
Euan

[radenui]

Hey Euan,

Thanks for your answer!
Right, it makes a lot of sense.
I was thinking about this to have a dev shortcut, but killing the tasks is enough in a dev environment.
I think this issue can be closed,

Thanks again,

Arthur

[hueniverse]

It would be nice if there was a way to automatically generate new task definitions when a new image tag is pushed. This way, at least that part would be automated and the CI system just needs to switch the service to the new revision.

[tjwebb]

Agree with @hueniverse, here, but maybe this is an issue for ecs-cli. It's practical, and also it keeps the CI service from needing to _know_ in detail the innards of ECS Tasks and how to generate one to replace the old one. A redeploy or some similar command for ecs-cli would be immensely useful.

[rajasekaran07]

@samuelkarp @radenui @tjwebb @hueniverse

It is not working for me. I am using build pipeline in AWS ECS and jenkins. When build is completed my service will not take new task definition. it is still running with old task. PFB the logs

Note: I am using one EC2-container instance in my ECS cluster. Will it works on single container instance?

{
"message": "(service my-test-node-service) was unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster. For more information, see the Troubleshooting section of the Amazon ECS Developer Guide.",
"id": "4c55cfa0-f9e9-42d0-b29c-6925f281ba4d",
"createdAt": 1482870012.703
},
{
"message": "(service my-test-node-service) was unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster. For more information, see the Troubleshooting section of the Amazon ECS Developer Guide.",
"id": "f130fc6c-03f5-499e-b1f5-1b981d0d6520",
"createdAt": 1482848396.3
},
{
"message": "(service my-test-node-service) was unable to place a task because no container instance met all of its requirements. The closest matching (container-instance f491214e-685b-46e9-8741-4072c222e3e0) is already using a port required by your task. For more information, see the Troubleshooting section of the Amazon ECS Developer Guide.",
"id": "36a29480-5fb4-4cea-9f58-c087f545b05f",
"createdAt": 1482847266.292
}
],
"runningCount": 1
}
}

Regards,
Raja

[javavenkats5]

@rajasekaran07 Any update to your issue. I have same issue. Its not killing running task. Just updating revision but old task is still running so new code is not effective.
???