Amazon-ecs-agent: "Network Bindings - not configured" although there are port mappings in the container.

[jmetzmeier]

This is a duplicate of #1057, #1063, and possibly #1064. #1063 was closed as fixed in version 1.15.1, however, I had a chance to test today and the problem is still present in that version.

[nmeyerhans]

Can you provide me with ecs-agent agent logs and a task ID that saw impact? Are you using the ECS Optimized AMI? The logs would be in /var/log/ecs/. Feel free to send them to me via email at [email protected] rather than posting them publicly here.

[jmetzmeier]

I sent the logs and task information to you via email. I am building an AMI using the ECS Optimized AMI as a base. Our only changes to the AMI is some basic user creation and the addition of a monitoring daemon (built via packer). I tested 1.5.0 with the vanilla AMI and still encountered the issue, but did not test vanilla with 1.5.1. The version we were using before I upgraded the AMI was 1.14.3, which continues to work properly.

[nmeyerhans]

@jmetzmeier Thanks for reporting this. We were able to confirm the issue and reproduce it here, and you can see @richardpen has just proposed a fix. I don't have an ETA for including this in a release at this time, but stay tuned for updates...

[bndw]

I believe I'm running into this as well with agent version 1.15.1.

• I have portMappings defined for my container
• They're being mapped by Docker
  0.0.0.0:32771->80/tcp
• describe-tasks does not show them (note no networkBindings attr):
  json { "containerArn": "arn:aws:ecs:us-east-2:xxx:container/b63f46d8-b45c-461b-aa56-958737b2a5c9", "taskArn": "arn:aws:ecs:us-east-2:xxx:task/982d52c6-8401-4fbc-b18e-76eaa09cf353", "lastStatus": "RUNNING", "name": "streamline-sidecar" }

This appears to be an intermittent issue, as sometimes the networkBindings show up after killing the task a few times.

EDIT: The severity of this bug is extremely high IMHO. We depend on this information for service discovery and without it our services are unreachable.

[nmeyerhans]

@bndw Yes, it certainly sounds like it. We're releasing 1.15.2 today with a complete fix. Apologies for the inconvenience.

[bndw]

Thanks for the fast response @nmeyerhans

[nmeyerhans]

Agent 1.15.2 is available now. You can get it from Docker Hub. It's also on the latest ECS optimized AMI, 2017.09.c. The list of regional AMI IDs is available in our docs, or you can use the following command to determine the AMI ID for a given region:

aws ec2 describe-images --owner 591542846629 --region us-west-2 --filter=Name=name,Values=amzn-ami-2017.09.c-amazon-ecs-optimized

[ctaggart]

Any idea if this bug still affects ECS Fargate?
https://forums.aws.amazon.com/thread.jspa?threadID=270940&tstart=0

[jhaynes]

@ctaggart You shouldn't be running into this bug on Fargate. If you are, please feel free to contact support. We can potentially try to debug here, but support will probably be a better option.

[guizmaii]

I confirm that this bug is always present in Fargate.