Kaniko: Support native AWS IAM integration with updated docker-credential-ecr-login

[siwyd]

There is now native integration with AWS IAM using Kubernetes ServiceAccounts: https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/. This requires all SDK's used to interact with AWS to support this however. As we use kaniko to build images on AWS Kubernetes clusters, it would be great if kaniko shipped with a docker-credential-ecr-login binary that supports this native IAM.

I've opened a PR in the amazon-ecr-credential-helper repo to update the SDK: https://github.com/awslabs/amazon-ecr-credential-helper/pull/183.

I'm opening this issue so the kaniko Docker image can get updated with a new binary when a new amazon-ecr-credential-helper gets released.

[siwyd]

@tejal29 After the PR merge (https://github.com/awslabs/amazon-ecr-credential-helper/pull/183), amazon-ecr-credential-helper master branch now supports native Kubernetes AWS IAM integration. Would it be possible perhaps to build a new kaniko release so this is supported in the upstream kaniko container image?

[patstrom]

Is this solved with the new release?

[antiqe]

Can someone confirm that the 0.15.0 is compatible with IRSA ?

[dlaidlaw]

The image at gcr.io/kaniko-project/executor:debug-v0.16.0 does contain the latest release of docker-credential-ecr-login: 0.4.0, from Jan 6th and containing the IRSA feature. However, I still cannot get kaniko to retrieve the correct credentials. I always end up with the Instance Profile credentials instead of those from the Kubernetes service account.

I have verified that the correct AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE values are set according to IRSA. I don't see why I get the instance credentials.

[diist]

It works by setting the following environment variable: AWS_SDK_LOAD_CONFIG=true

[dlaidlaw]

If running on an EC2 instance with an instance profile, you may also need to set: AWS_EC2_METADATA_DISABLED=true In my case kaniko kept getting the EC2 instance profile role credentials instead. This disables the instance profile role credentials and kaniko then gets the expected credentials.

To be clear, set both AWS_EC2_METADATA_DISABLED and AWS_SDK_LOAD_CONFIG as mentioned above to true on EC2 instances with instance profiles.

[gsf]

In my case, running the debug image on kubernetes, I had to overwrite the docker config first:

echo '{"credsStore":"ecr-login"}' > /kaniko/.docker/config.json
/kaniko/executor --context `pwd` --destination xxxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/app:latest