Containers-roadmap: [ECR] [Remote Docker Repositories]: Pull through cache

[wayne-folkes]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
I would like to be able to store docker images that are usually hosted on third party registries in ECR.

Which service(s) is this request for?
ECR

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Our organization would like to be able to avoid being affected if/when those registries go down by having a copy of certain images cached in ECR. Today if Quay.io or some other public registry goes down we may not be able to scale up a cluster.

Some secondary benefits would be being able to limit which images can be used and also saving on network costs as we would not need every service pulling images from the internet when they can be pulled from ECR via private link.

I image this would work similar to CodeArtifact where you can have service pull libraries from upstream as needed.

Are you currently working around this issue?
How are you currently solving this problem?
Today we have to pull a list of images from our many k8s clusters and run a Codebuild job to pull those images and push them into ECR.

Additional context
Anything else we should know?

Attachments
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[DJMatus23]

We use Artifactory to perform this role, and it has many security, availability and performance benefits.

[wayne-folkes]

@DJMatus23 We are using Artifactory today but would like to get out of having to run that ourselves.

[stijndehaes]

With the recent changes to docker hub with rate limiting this has become more important then ever:

https://docs.docker.com/docker-hub/download-rate-limit/

[pgarbe]

Would love to see this feature in ECR, or even better have ECR integrated in CodeArtifactory (which supports already pull-through-cache)

I build an CDK Construct that syncs specific images from DockerHub to ECR (https://github.com/pgarbe/cdk-ecr-sync). Might be useful until this feature is implemented.

[omieomye]

Thanks for raising this issue. We're looking at what this will take to implement. A couple of questions for the community:

1. How do you see authentication working between an ECR registry and an upstream one? Is authentication even necessary to use?
2. Is it more important to cache a publicly shared image or your organization's images in ECR?

[connorearl]

For our use case, all our private images are in ECR, so upstream authentication wouldn't be needed, The main reason we want this is to allow us to restrict our cluster to pull from one source, and only have approved images in ECR.

Also with the new limits for the docker hub and interruptions to quay.io a few months ago, being able to cache public images is important for availability.

[wayne-folkes]

Our use case is similar. Our private images are in Artifactory with plans to move them to ECR. The images we are thinking about are public images that do not require auth to pull.

Being able to cache public images is more important.

[dresnick-sf]

Same for us too. We'd like this feature to ensure we can get public images. No need for authentication with an upstream registry.

[RichiCoder1]

Same as above. Caching Docker/Quay/GitHub. About the only auth'd images we might cache are from other ECRs.

[jkroepke]

Please use reactions instead wrote same here like the entry post describe it.

Got tons of mails.

[blockjon]

According to the DockerHub announcement I think most of us are now in big trouble because there's only 1 month left before we go from unlimited DockerHub pulls to just 16 pulls per hour which is slated to start on November 1, 2020. That is a big problem, mostly for our automation mechanisms (CICD) which make the assumptions that DockerHub image pulls work. I am encouraged an AWS ECR employee said they would look into solutions, but I think for all intents and purposes that everybody who is in this situation is now scrambling for solutions and we should not wait and hope AWS solves this anytime soon. Even if they roll out a full solution in the next few days we would still need time to adapt to using it. I am now going to look into how to solve this with Artifactory.

[jkroepke]

I don't know if the ECR will helps here. I guess the outgoing IPs of the ECR will be always on limit.

I do some research for my company. I start to investigate how to setup a caching registry for our CI and our Kubernetes Platform (not EKS).

I decide to buy a 5$/month user which has no rate-limit on DockerHub. I don't now if it's cheaper as a caching ECR, but it may cheaper and easier then setup an high availability caching docker registry. Since the caching docker registry will affected by rate-limits, too. (no matter using a own IP or a shared IP by ECR), a caching registry may not help.

[eugenestarchenko]

Would love to see this feature in ECR, or even better have ECR integrated in CodeArtifactory (which supports already pull-through-cache)

I build an CDK Construct that syncs specific images from DockerHub to ECR (pgarbe/cdk-ecr-sync). Might be useful until this feature is implemented.

Google wants to transform its Container Registry to https://cloud.google.com/artifact-registry
Note: Artifact Registry is currently in beta as the evolution of Container Registry, it supports multiple artifact formats, regional repositories, and more granular access control. After it becomes generally available, Artifact Registry will replace Container Registry.

It feels like something similar should occur with AWS CodeArtifact and AWS ECR in near future. Good time to ask AWS about RoadMap on that with all these multi-accounts and docker hub limits.