Amazon-ecs-agent: Cannot update ECS Agent

[bendavies]

Summary

Cannot update ECS agent to latest version.

Description

We're currently running v1.32.1 on our cluster. ECS tells us a update is available:

Trying to update to 1.33 via the UI and API both error:

An error occurred (NoUpdateAvailableException) when calling the UpdateContainerAgent operation: There is no update available for your container agent.

Any ideas what could be wrong here?

Thanks!

[fenxiong]

Hi,
Are you using ECS Optimized AMI? If so what's the image id that you are using?

[fenxiong]

Not sure if it's your case, but one potential reason to get this is that you updated Docker on the instance to from 18.06 to 18.09 (latest available from Amazon Linux). In that case you will get this error, until we update the Docker version on the AMI to 18.09 as well.

[tedWexus]

I had a different problem, but the root of the problem was updating Docker to v18.09.9-ce in my EC2 instance. The following line in my auto scaling launch configuration User Data was responsible for the update:

yum update -y

I created a new launch configuration without that line in User Data, created the Auto Scaling Group, and the EC2 instance that was automatically created had Docker v18.06.1-ce installed. ECS Agent v1.33.0 was also installed! I am using ECS Optimized AMI.

[bendavies]

Indeed, we are using the ECS optimised AMI, but have updated docker with yum update -y too.

docker -v
Docker version 18.09.9-ce, build 039a7df

[bendavies]

We were able to rebuild our custom AMI from the ECS Optimized AMI which came with v1.33 of the ECS agent, and then update docker to 18.09.

The error is strange though. We've been on 18.09 for ages and have updated the agent via the UI without issue up until now.

[petderek]

This issue should be resolved now, as of our latest AMI update. Please let us know if you are still seeing issues.

Spoke too soon; I'm still observing this.

Mitigations are to use Docker 18.06 or use the latest ECS optimized AMI.

Latest AMI is available in SSM:

aws ssm get-parameter --name /aws/service/ecs/optimized-ami/amazon-linux/recommended/image_id --output text --query Parameter.Value

[petderek]

Ok, we've put out a fix and I'm no longer able to repro this condition. 😄

We released agent 1.35.0, and the AMIs now ship with Docker 18.09 by default.