Amazon-ecs-agent: Support for pulling ECR images from another AWS account?

[jhovell]

I'm not sure if this is supposed to work or if I just have something misconfigured. I have set up some cross-account policies and roles which seem to be working based on a few boto and CLI test scripts. But it seems the ECS agent would need to be made aware of the cross account role and use sts assume role to be able to download from another account.

The error I get in ECS and Docker is misleading (404 not 403 or 401). Perhaps that is a sign this is actually supposed to work but Googling I can only find how to set up cross account access in general but nothing about ECS agent support

http://docs.aws.amazon.com/AmazonECR/latest/userguide/RepositoryPolicyExamples.html#IAM_allow_other_accounts

ECS agent log
```2016-02-10T06:56:03Z [INFO] Adding event module="eventhandler" change="ContainerChange: arn:aws:ecs:us-west-2:111111111111:task/example-1379-4d99-ae58-a2130c1dacd7 my-app -> STOPPED, Reason CannotPullContainerError: Error: image my-app:1.0 not found, Known Sent: NONE"

Docker log

time="2016-02-10T07:27:39.052084026Z" level=error msg="HTTP Error" err="No such image: 111111111111.dkr.ecr.us-east-1.amazonaws.com/my-app:1.0 (tag: 1.0)" statusCode=404

[dangrd]

Make sure you are using the latest version of the agent to ensure you have a version built with Amazon ECR support. Support was added in late December 2015 which handles Authentication to ECR.

After that the easiest way to do cross-account permissions with ECR is using repository policies following example from the document you linked above. This policy document is applied directly to an Amazon ECR Repository with no need for additional roles or temporary tokens. The Amazon ECS Agent using standard configuration uses the EC2 Instance Role to make calls to AWS. This should work fine to access cross-account registries. Make sure this role has the AmazonEC2ContainerServiceforEC2Role managed policy applied so that the agent has permissions to the Amazon ECR APIs.

I believe the 404 error here is a bit of a red herring. If the Docker client fails the attempt to pull the image the client falls back to the V1 API. Since Amazon ECR does not support Docker Registry V1 this reports a 404. I believe this behavior has been changed in Docker 1.10.

[jhovell]

thank you @dangrd :+1: ... Adding a repository policy was what I needed to do. I didn't realize this couldn't be accomplished through cross account roles. Nice that this can be specified at the repository level as well.

In case it helps anyone my policy allows pulling from another account:

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "CrossAccountPull",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111111111111:root"
            },
            "Action": [
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:BatchCheckLayerAvailability"
            ]
        }
    ]
}

[hamstah]

@jhovell Did you have to setup anything else to get it to work?

I'm having issue with my cross account setup too.

docker.errors.APIError: 500 Server Error: Internal Server Error ("denied: User: arn:aws:sts::XXXX:assumed-role/blah/i-bleh is not authorized to perform: ecr:BatchGetImage on resource: arn:aws:ecr:eu-west-1:YYYY:repository/ZZZZ")

I have the same policy as the one you have for the repo ZZZZ in account YYYY, with XXXX as my principal.

Not sure what the issue is

[karelbemelmans]

@hamstah I'm having the same issue as you had, did you find a solution for it?

[picwelltimjones]

Having trouble with cross-account pulls was resolved for one of our users once we had the user properly log in. If cross-account, the user needs to set the registry id to the id of the target account:

$(aws ecr get-login –registry-ids  <accountID_of_repository> --region <region>)

[AlexShuraits]

@karelbemelmans , @hamstah Same issue here. Did you succeed?

[hamstah]

Hey,

The following policy works for me, sorry I missed your comment @karelbemelmans

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "CrossAccountPull",
            "Effect": "Allow",
            "Principal": {
                "AWS": "${cross_account_id}"
            },
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:BatchCheckLayerAvailability"
            ]
        }
    ]
}

Note that the user needs to login with

aws ecr get-login

With both accounts after assuming the right credentials.
(I do it in boto too)
Let me know if that works, I have it working so it's definitely possible but might be an issue with auth.

@AlexShuraits

[AlexShuraits]

@hamstah Thanks for quick reply.
Is there a way to do "aws ecr get-login" on ESC service?
I am getting this error inside the service, I am not trying to pull the image on my local/remote machine.
I tried to connect to the machine where the service is trying to pull the image and the aws cli is even not installed there. I am using ECS-optimized AMI provided by Amazon.
I tried to allow every account with "Principal : *" and all the ecr actions inside, and I am still getting this error.

[samuelkarp]

@AlexShuraits The ECS agent will perform the necessary steps to obtain an authorization token from ECR. Please ensure that you have the permissions necessary in both the repository policy of the account that owns the repository and in the IAM profile on your EC2 instance. Please see the documentation on repository policies and the documentation on permissions necessary for your EC2 IAM profile. If you're continuing to experience difficulties, please feel free to open a new issue.

[hamstah]

Ah yes, I replied assumed general pull not from.ecs sorry.

I also didn't include the repo policy, will do tomorrow

On 18 Jan 2017 6:30 pm, "Samuel Karp" [e-mail] wrote:

@AlexShuraits https://github.com/AlexShuraits The ECS agent will
perform the necessary steps to obtain an authorization token from ECR.
Please ensure that you have the permissions necessary in both the
repository policy of the account that owns the repository and in the IAM
profile on your EC2 instance. Please see the documentation on repository
policies
http://docs.aws.amazon.com/AmazonECR/latest/userguide/RepositoryPolicyExamples.html#IAM_allow_other_accounts
and the documentation on permissions necessary for your EC2 IAM profile
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_managed_policies.html#AmazonEC2ContainerServiceforEC2Role.
If you're continuing to experience difficulties, please feel free to open a
new issue.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws/amazon-ecs-agent/issues/308#issuecomment-273559371,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALfjvfHqPgXIsTY57e90L-Zv8dkYAuRks5rTlougaJpZM4HXEOJ
.

[AlexShuraits]

@samuelkarp @hamstah I have created new issue #675