Moby: Allow configuration of additional registries
Partially superseds issue #8329. It will be superseded completely with upcoming, consequent proposals based on this.
Together with extensions it obsoletes PR #10411 but is still compatible with it.
This is a root proposal for issues concerning usage and configuration of additional registries. It outlines:
1. configuration file paths
2. their format
3. their structure
Use case
As an enterprise user interacting with multiple private registries and having little or no interest in public one, I'd like to be able to pull from them without the need to specify them on command line. I'd like to specify which registries can be pushed to, which registries can be pulled from. I'd like to be able to prevent our developers from interacting with public registry or at least let Docker ask them for confirmation before a push.
In order to do this, Docker has to provide some way of configuration which would be extensible, legible and easy to edit.
Requirements
- Allow user to configure a list of additional registries specifying at least:
- index name
- index aliases
- associated mirrors
Proposal
Make Docker daemon read registry configuration file e.g. /etc/docker/registries.yml in YAML format.
Example
# Special object grouping auxiliary settings concerning registry configuration.
general:
# Optional directory relative to this config file with additional
# registries. Defaults to empty string.
load_dir: /etc/docker/registries.d
# Special object with default properties for all registries (even those not
# configured).
defaults:
public:
index: docker.io # this is redundant
aliases: [index.docker.io] # this as well
mirrors: [127.0.0.1:5000, docker-mirror.domain.ltd]
# Each following entry represents additional registry if it has an `index`.
redhat:
index: registry.access.redhat.com
mirrors: [127.0.0.1:5001]
public registry would stay hardcoded in docker for cases where registry configuration is absent or public entry is missing.
The same applies to default values. Right now there aren't any properties we could put there - they'll be specified in extension proposals.
Proposed Extensions
- #11816 pull from default registries
- #11817 push to default registry
- #11818 search default registries
- #11819 block registries
- #11820 authentication support
miminar
All 19 comments
Hi @miminar ,
Like was said earlier, "distribution and registry" topics are discussed in the open on the distribution repo (https://github.com/docker/distribution/), on irc (#docker-distribution), and during the group meetings.
There you will find work in progress proposals to address some of your issues here, specifically about namespace and discovery:
I strongly encourage you to participate in these discussions to save efforts and make sure things are happening. I'm not saying proposals and ideas here are not examined, just that being aware of (conflicting/concurrent) work in progress that is in the making is definitely useful for you.
Finally, the distribution working group does already have two persons from RedHat (@vbatts and @ncdc) that are actively participating in the discussions and design making for the registry communication pipeline. Synching/reaching out to them is probably a good idea that can help you get up to speed and figure out the best way to contribute to the community effort on these topics.
Nevertheless, about this proposal here:
As an enterprise user interacting with multiple private registries and having little or no interest in public one, I'd like to be able to pull from them without the need to specify them on command line.
If the solution to this would lead to a situation where docker pull ubuntu could mean two different things for two different persons/installs, that would very bad for the user.
It would lead to a fragmentation of the ecosystem and break the community.
This was discussed to death already, and such a solution would simply be refused.
Saving a few additional keystrokes (eg: docker pull myregistry:5000/ubuntu) is not a strong enough argument to warrant breaking the ecosystem.
Now, ability to add flexibility into image distribution (eg: uncouple image names and transport) is definitely something everybody wants, and that is being worked on: it does require "trust" so that image names can be signed to guarantee namespace consistency, though.
I'd like to specify which registries can be pushed to, which registries can be pulled from. I'd like to be able to prevent our developers from interacting with public registry or at least let Docker ask them for confirmation before a push.
From an ops perspective, if you are worried about communication to specific servers, there are simpler ways to prevent that (firewall, dns, etcetera).
Nevertheless, submitting the idea and use case to https://github.com/docker/distribution is certainly good as well.
Overall, your proposal is interesting, but:
- requires image signing so that we can guarantee the namespace is not endangered
- definitely intersect with discovery
Looking forward to talk to you more about these.
dmp42
on 26 Mar 2015
@dmp42 Got it. I'll join the discussions there.
miminar
on 27 Mar 2015
Thanks @miminar
Let's catchup on irc.
dmp42
on 27 Mar 2015
Discussion for distribution repo.
LK4D4
on 15 Sep 2016
Hi Gentlemen
This could be quite easy to fix with an environment/config variable. Would be nice to control default registry from outside the Dockerfile.
fenchu
on 29 Sep 2016
Is this proposal totally dead?
chrrrles
on 23 Oct 2016
@dmp42
If the solution to this would lead to a situation where docker pull ubuntu could mean two different things for two different persons/installs, that would very bad for the user.
This argument just does not make sense which has been seen many times with apt, Maven, NPM, Bower and other similar packages.
"docker pull ubuntu" _should_ mean different things when ran off user's personal laptop compared to running off corporate hardened SOEs.
"gradle dependency org.apache.lang:commons-lang" _should_ and _does_ mean different things when ran off user's personal laptop (just download whatever sits on maven-central) versus running off lock-down enterprise environment (usually Nexus repository manager locked down to specific / signed JAR files).
At this point we are using Nexus as a source of truth - no direct access to _anything_ public internet from our CI build agents is the policy. This works fine with every other piece of the jigsaw puzzle, except Docker - which is extremely disappointing.
ghost
on 24 Oct 2016
"docker pull ubuntu" should mean different things when ran off user's personal laptop compared to running off corporate hardened SOEs.
Can you elaborate on why that should be so?
What would be in the ubuntu dev version that would not be in the hardened node? (or conversely)
On the contrary, as a dev, I would love to walk out of the "works locally / doesn't work in prod" mess, and code against exactly what's running in prod.
At this point we are using Nexus as a source of truth
Are you concerned about tempered content from public sources?
Or about developers using libraries you have not vetted? (and below for more questions in that line)
This argument just does not make sense which has been seen many times with apt, Maven, NPM, Bower and other similar packages.
If my code depends on library foobar in version VER, then the only way to make this reproducible is if package-manager install foobar@VER refers to the same thing everywhere you are.
If you want to fork / patch foobar, that's fine, just change your dependency and point to the fork. eg: package-manager install alternative:foobar@VER
If you want to just control distribution of the exact same thing, then it's worth asking:
- is this for "security"? and then what's wrong with content-adressability? how do you protect your local cache from poisoning? are you using TUF/DCT?
- is this for availability? then you can do that already with the registry mirroring capabilities
- is this for another reason?
Happy to chat more about this, and figuring out clear use cases (and reasons to do so).
dmp42
on 25 Oct 2016
How about you just implement this and let US (users) worry about the impact of configuring alternative repositories? We should be able to define a hierarchy of where to pull images from. For example:
- internal registry
- public registry
Whatever argument you (the Docker team) provide against doing this is not relevant to users who want this feature. Give us the flexibility to do what we want. Let us worry about the consequences of doing so. You can't protect us from shooting ourselves in the foot so stop trying.
This statement:
It would lead to a fragmentation of the ecosystem and break the community.
I have no idea how that would be an issue because company A wants to run an internal private registry for resolving references to internal images. Unless you can give a concrete example where providing this feature would somehow lead to a broken Docker hub, I don't see how this is a true statement.
et304383
on 14 Dec 2016
The Java community has been doing this for 10+ years with its Maven repositories that hold Java JAR and WAR files -- with much success. I think it is a perfect metaphor for this situation.
hdave
on 29 Dec 2016
Trusted/signed images are an option now, so you can open this ticket back up now.
We own a licensed copy of Docker Trusted Registry running on-prem, but our hordes of developers aren't about to change all their muscle-memory and Dockerfiles to start using it overnight.
Can we pretty please have a configurable default registry for push/pull yet???
Dockerfiles are currently not portable between dev/staging/production environments specifically because of this issue.
RedHat's functional implementation of '--add-registry' and '--block-registry' means the community is extremely broken due to the availability of the exact solution that they've been begging for for years being present but only in a disparate fork of the docker engine/client. Is this the goal of Docker Incorporated developers? Doesn't this "hurt the community pretty badly"?
We just want to use docker with our own registry as a default when none are specified (kind of like the "library" repository is default, yet it may be overridden). We don't want to resort to DNS spoofing or blocking the public registry via our firewall or proxies - that would hurt the community pretty badly because we actually want our developers to have the option of pushing/pulling to/from the Docker Hub when explicitly specified.
There are YEARS worth of pull requests and forum posts and serverfault/stack exchange solutions (which break with every update to docker). There are too many rejected PRs to keep track of.
If somewhere in the "Distribution" section, this request/requirement has not been summarily dismissed, can someone please point out where the horde of wanting enterprise/government admins must now beg for this feature to be genuinely considered?
There is a solution that Red Hat implemented. It works flawlessly. You have the PR. Please consider it. I don't want to be stuck on their version of 1.10. That hurts the community.
jsengla
on 9 Jan 2017
As a developer with 30 years experience, I see no issue with fragmentation. The average community developer, working on his or her own, is going to use docker.io by default. They have neither the need nor the desire to set up their own registry, therefore they will not experience fragmentation.
The average corporate developer, precisely where you want to maintain traction for your product, is going to set up their own registry, and it will make life vastly easier for everyone involved in the project at all levels if the default registry were configurable. A few examples:
- Developers wouldn't need to worry about remembering to use the company registry.
- People involved in IA and security would not have to worry about inadvertent use of unapproved images.
- Legal wouldn't have to worry about inadvertent pushes of proprietary images to the docker.io registry.
Without the ability to set the default registry, all of the above and more are possible. Sure, a firewall could be set up to block access to the docker.io registry, but special exceptions would have to be made in that firewall to allow pulling desired public images for placement on the local registry. Yet more complexity that could be avoided if setting the default registry were possible.
With a configurable default registry, the exception case becomes pulling from the docker.io registry, and it is clear in the Dockerfile when this is being done. This makes it much easier for everyone involved to ensure that the correct images are being used.
The argument that it will fragment the community completely ignores the needs of a significant portion of the community -- commercial users -- in favor of another portion which wouldn't even be affected by the capability, because they'd never use it.
KedneckInc
on 2 Mar 2017
Can only agree with the last couple of statements. It is kind of ridiculous that a product tries to tell its users what features they need.
Shouldn't it be the other way around? I mean look at all the approvals of these comments asking for this -already implemented- feature.
I have never belonged to the "JAVA" community, but I still need this. It would make my life much, much easier.
So, instead of trying to tell us that "you really don't need this" maybe you should just give it to us and then _don't use it on your end_ if _you_ don't need it.
Ps: the situation with "registry acting as a pull-through cache of a private registry" is the exact same one. Kind of disappointing. Hope the situation/mentality of the community changes in the future
Levovar
on 17 May 2017
Having the ability to configure the default registry seems like a huge win IMHO.
diclophis
on 23 Jan 2018
If the solution to this would lead to a situation where docker pull ubuntu could mean two different things for two different persons/installs, that would very bad for the user.
To control the community and tell us the community whats bad/good for us is the wrong state of mind IMHO, it's our responsibility weather changing the default docker behavior or not.
We're not asking for the DOCKER TEAM to change the default behavior of Docker out of the box. we're simply asking for an option to change the default registry as ENV var or any other solutions already suggested in the distribution discussions.
If a user changes his default registry to something else, obviously
docker pull ubuntu would be different for him but it's under user's responsibility
It's a simple feature that has been asked from your own community,
which have been already implemented in a fork and obviously did NOT tear the community apart.
Chrislevi
on 8 Apr 2018
@dmp42 I finally had the time to go over and look at the two threads you recommended. I'm sorry to say that I found them quite confusing, and that I don't see how/where they answer the problem that I have. To wit, for security purposes, we need to be certain that developers are, by default, (a) not pulling from a repo that has not been vetted, and (b) are not inadvertently pushing our work to a public repo.
Can you explain how any of the existing PR would cover those two issues?
The use case that you see for fragmentation does not occur in our case because our work for paying customers will never be in a public repo, so the fact that docker pull ubuntu doesn't necessarily do the same thing in this private environment that it would in the public environment is simply not a valid issue. The two environments are entirely separate.
The customers I work for have very stringent requirements for vetting before any public resource is used in the project, and would absolutely refuse to automatically use whatever new version of ubuntu came from the public repo. As such, we've had to carefully instruct our developers regarding the issues, and make sure that they never pull or push from the public repo. This would be vastly simpler if we could just set a system wide variable that set the default docker repo to our private repo. Any developer who overrides that setting would know that they are violating our security requirements.
KedneckInc
on 1 May 2018
@KedneckInc see https://github.com/moby/moby/pull/34319, for which the design is currently being finalized
thaJeztah
on 1 May 2018
@thaJeztah @dmp42 god, this is a sad read given that Docker Inc is essentially telling us that docker pull someimage will now break based on rate limiting image manifest GET's (which, if they were not limited, would at least allow us to retain the global namespace, while then pulling the identified layer from some other source).
SvenDowideit
on 29 Oct 2020
Docker supports _registry mirrors_ (also known as "Run a Registry as a pull-through cache").
https://docs.docker.com/registry/recipes/mirror/#configure-the-docker-daemon
finkr
on 25 Nov 2020
Related issues
programmerq
路
3Comments
anshumanbh
路
3Comments
lvthillo
路
3Comments
karellm
路
3Comments
nisanharamati
路
3Comments
Most helpful comment
As a developer with 30 years experience, I see no issue with fragmentation. The average community developer, working on his or her own, is going to use docker.io by default. They have neither the need nor the desire to set up their own registry, therefore they will not experience fragmentation.
The average corporate developer, precisely where you want to maintain traction for your product, is going to set up their own registry, and it will make life vastly easier for everyone involved in the project at all levels if the default registry were configurable. A few examples:
Without the ability to set the default registry, all of the above and more are possible. Sure, a firewall could be set up to block access to the docker.io registry, but special exceptions would have to be made in that firewall to allow pulling desired public images for placement on the local registry. Yet more complexity that could be avoided if setting the default registry were possible.
With a configurable default registry, the exception case becomes pulling from the docker.io registry, and it is clear in the Dockerfile when this is being done. This makes it much easier for everyone involved to ensure that the correct images are being used.
The argument that it will fragment the community completely ignores the needs of a significant portion of the community -- commercial users -- in favor of another portion which wouldn't even be affected by the capability, because they'd never use it.