Aws-cdk: [ecr] CDK DockerBundling should use the underlying default image from ECR

[srinivasreddych]

As per the latest DockerHub rate limit release, the activity of running cdk bundling is causing us rate limit issues. It would be great if CDK would switch the default dockerimage from “amazon/aws-sam-cli-build-image-” https://hub.docker.com/u/amazon to an image hosted in AWS ECR.

Use Case

Our build systems run cdk deploy which will bundle assets for lambda functions. For bundling the assets, cdk would pull amazon/aws-sam-cli-build-image and perform the activity. It would be great to have CDK rely on an image hosted outside of DockerHub.

Error seen

Unable to find image 'amazon/aws-sam-cli-build-image-python3.6:latest' locally
docker: Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit.

[AndyEfaa]

I can confirm the issue. This is causing all of our pipelines to fail.
In our case, the image amazon/aws-sam-cli-build-image-python3.7 is build in the context of PythonFunction(https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-lambda-python.PythonFunction.html).

[TheRealAmazonKendra]

My team is also having this issue with the image amazon/aws-sam-cli-build-image-nodejs12.x. It is currently causing our CI builds to fail.

[koups496]

My pipeline is also failing on amazon/aws-sam-cli-build-image-python3.7 for the same reason

I can confirm the issue. This is causing all of our pipelines to fail.
In our case, the image amazon/aws-sam-cli-build-image-python3.7 is build in the context of PythonFunction(https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-lambda-python.PythonFunction.html).

[erudisch]

We have the same issue with PythonFunction

A way to set the docker login would be great to use the images hosted on ecr.

see: https://github.com/aws/aws-cdk/issues/11544

[AndyEfaa]

Assuming that you build your CDK apps in a pipeline, the issue can be fixed by including a docker login in the buildspec.yaml file as follows:

version: 0.2 
env:
  secrets-manager:
    DOCKERHUB_USERNAME: "dockerhub-access-token:dockerUser"
    DOCKERHUB_ACCESS_TOKEN: "dockerhub-access-token:dockerAccessToken"
phases: 
  install: 
    runtime-versions: 
        python: 3.7 
    commands:
      - echo "Start Build..."
  build: 
    commands: 
      - nohup /usr/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2 &
      - timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
      - docker login -u ${DOCKERHUB_USERNAME} -p ${DOCKERHUB_ACCESS_TOKEN}
      - echo "Do rest of cdk build..."
  post_build:
    commands:
      - echo "Do post build..."
artifacts: 
  files: '**/*'

Works great for us.

Thx to @skinny85 for the hint.

[nbaillie]

With the release of Amazon ECR Public Gallery
could it be moved to use : public.ecr.aws/amazonlinux/amazonlinux:latest

Im guessing however aws-sam-cli-build* might also move there at some point.

[srinivasreddych]

Hello @MrArnoldPalmer Any estimate timeline around when CDK would start leveraging base images from AWS Public ECR? For now, we are still stuck in doing docker login -u username -p pwd. Thanks!

[MrArnoldPalmer]

@srinivasreddych no estimate on this right now, though I understand the pain this is causing as pretty much every developer using docker as started running into these new rate limits. docker login is the workaround I recommend for now.

I need to look around a bit for other usage of dockerhub images in the cdk codebase that affect user's synth/deploy. I'll provide an update when I'm able to do that.

[outdoorpet]

We are running into this issue using CDK Python lambda asset bundling as described here. It isn't clear to me how I can fix this with supplying docker credentials as a work around...

[zxkane]

Same here. It’s better moving the image to https://gallery.ecr.aws/ .