Cli: Allow `docker push' to push multiple/a subset of tags

[ebd2]

Allow the following usage:

docker push org/image-a:some_tag org/image-b:other_tag

This would make docker push simpler to use in systems that need to build and push multiple images to dockerhub by obviating the need to wrap docker push in a loop.

[glyif]

@dnephin, super cool tool! imma try it out. for my educational purposes, i'm going to implement this feature. :)

[iBobik]

Also it can save resources while building - the same image does not have to be uploaded twice.

[thaJeztah]

Prior discussions on this feature can be found in https://github.com/moby/moby/issues/7336, https://github.com/moby/moby/pull/11682, https://github.com/moby/moby/issues/16106, and related, https://github.com/moby/moby/issues/23383

Note that adding support for this on the command-line will not provide enhancements other than not having to "loop" through tags for pushing (see https://github.com/docker/cli/pull/458#pullrequestreview-57642310)

[SeidChr]

I wonder how that ticket is still open. it started in 2014.. was a quest to finally find the last instance here :)
Anyway. Im in for it.

[elboletaire]

I wonder how that ticket is still open. it started in 2014.. was a quest to finally find the last instance here :) Anyway. Im in for it.

+1

It's curious/weird how you can tag multiple tags at once with docker build -t, but you can't publish them all at once...

[Hronom]

Guys what the progress on this issue? Still need docker push with multiple tags!

[Nowaker]

This, please!

[smuryginim]

+1

[XBeg9]

+1

[pappnase99]

How is this still a thing?

[elboletaire]

There's currently the PR https://github.com/docker/cli/pull/458#pullrequestreview-57642310 which is under review. AFAIK for now it only adds a loop for the docker push instead of trying to do it in parallel, which is how it works in docker build; that's why it isn't still accepted.

[ggalihpp]

Oh its open. I though its already implemented :(

[AntonKrug]

Ignoring the performance, the options either to push image with all tags or single tag seems bit odd.

[javatechy]

You can install dokr . and use dokr dock --push image_name. This will push your multiple images

[Hronom]

Please do this at least, so many time since this started, I was thinking it's already implemented.

[aramhakobyan]

+1

[achillesrasquinha]

This is more of a design problem. I'd blindly assumed that this would have worked for me

$ docker push $(docker images --format "{{.Repository}}" | grep "foobar")

only to realize it takes just one argument. I'd say the same for docker pull. There are just too many great use cases for a multi-push. Plus, it's pretty neat instead of a

$ for i in `docker images --format "{{.Repository}}" | grep "foobar"; do docker push $i; done

Not very command-like.

[lagden]

@achillesrasquinha

Is it not necessary pass the tag?

Example:

docker images --format "{{.Repository}}:{{.Tag}}" | grep "foobar"

lagden/foobar:6.1.0
lagden/foobar:latest

[purplesoda]

I was able to do this using AWS ECR for our docker image hosting. We're using jenkins for CI to push the image up to ECR once the build is finished. Then you can use aws cli to re-tag the image with whatever you want, including variables.

In this example, push your initial image up with the latest tag, then specify whatever additional tag you'd like in the second command behind --image-tag

MANIFEST=\$(aws ecr batch-get-image --repository-name yeet --image-ids imageTag=latest --query 'images[].imageManifest' --output text)

aws ecr put-image --repository-name yeet --image-tag v_$BUILD_NUMBER --image-manifest "\$MANIFEST"

We're using ECS for hosting so this actually fit nicely into our pipe. You can still use amazon ECR if you're hosting elsewhere though.

[thiagolsfortunato]

Any new?!

[davidtabernerom]

I achieve this by using pipes with echo and xargs, but it really needs to be implemented, this only automate it, but the command is summoned each time.

If anyone wondering:
echo 'repo/flavor:tag' 'repo/flavor:tag' 'repo/flavor:tag' | xargs -n 1 docker push

[saravana1992]

Any news on this?

[sj26]

Abandon hope all ye who enter here.

I've had great success using skopeo which allows flexible manipulation of remote docker image registries, like re-tagging images on remotes without needing to push/pull locally, or pushing/pulling multiple images in parallel, etc.

[FearlessHyena]

@davidtabernerom 's workaround fixes it :)
But hopefully this gets implemented in future

[sj26]

The problem with the workaround is you can't say "push exactly this version of this image to these tags on the remote" — if you have multiple processes or threads on a machine then they can stomp on the local tags before, while, or after each push to the remote, so you may get tags on the remote which point to different images. :-(

[adampl]

To avoid unnecessary repetitions, use this Bash expansion syntax:

echo repo/flavor:{1.0.0,1.2.3} | xargs -n 1 docker push
# or
echo repo/{foo,bar}:1.2.3 | xargs -n 1 docker push
# or even
echo repo/{foo,bar}:{1.0.0,1.2.3} | xargs -n 1 docker push

[msbenjamin12]

I solved the problem by the following:
docker push [REPOSITORY]

example:
localhost:5000/someRepo:tag1
localhost:5000/someRepo:tag2
localhost:5000/someRepo:tag3

The following will push all the above example images to the REPOSITORY
docker push localhost:5000/someRepo

As a result tag1, tag2, and tag3 will be pushed to the REPO "localhost:5000/someRepo" recursively.

[japanfred]

I solved the problem by the following:
docker push [REPOSITORY]

example:
localhost:5000/someRepo:tag1
localhost:5000/someRepo:tag2
localhost:5000/someRepo:tag3

The following will push all the above example images to the REPOSITORY
docker push localhost:5000/someRepo

As a result tag1, tag2, and tag3 will be pushed to the REPO "localhost:5000/someRepo" recursively.

Thanks! This works perfectly in our build environment.

[WebSpider]

With the solution of @msbenjamin12 this issue seems resolved.

[superstator]

No, that pushes ALL tags for a repository. For us at least we want to be able to push multiple specific tags, but not just blanket push the entire repo.

[cpuguy83]

Why is this desirable? You can do this with the shell already as mentioned in multiple places above.

I do not see a technical advantage to supporting this.
The syntax prevents us from using the command to do other things (such as supporting remotes, ala git).
It also causes problems like figuring out errors on push (what tag had the error?).

Why are we trying to avoid a for loop or xargs?

[sj26]

The main advantage of docker push with multiple arguments for me is that docker can optimise the pushes, and also do them in parallel, similar to the way buildkit optimises pulling images in parallel.

The other is that it guarantees all pushed images are resolved at the same time, which given docker has global, mutable state could reduce fragility introduced by highly concurrent environments. For this latter reason I'd also really like the ability to docker push <exact-image-sha>:<remote-target> similar to git push <remote> <commit-sha>:<remote-ref>, but that's a separate issue which I'm currently using skopeo to work around as mentioned before.

[cpuguy83]

For this latter reason I'd also really like the ability to docker push :

There lies the contention with multiple image push.
We'd like to do something like docker push <remote> <image>.
I think overloading : for this is not desirable and the above syntax is more inline with git.

docker can optimise the pushes

Docker can already optimize this without having to change the API by de-duping layer uploads. For that matter this can (should?) be resolved on the registry side, assuming it's not already.

[superstator]

To me it was mostly a consistency thing. I can build a single image and apply multiple tags with

docker build . -t some.io/myimage:1.0 -t some.io/myimage:latest

So it seems logical I'd be able to then push those tags similarly with

docker push some.io/myimage:1.0 some.io/myimage:latest

Granted it's a much bigger deal to loop with docker build since there's all the context hashing happening there, but it also seems like an easy win. Maybe as a compromise push could either take a single identifier as it does now, or an optional -t arg (to match build) to supply multiple tags?

[fabiocarneiro]

The expansion is not a good enough option as it still requires to send the data again. Multiple tags on the same push allow the transfer to happen only once.

[cpuguy83]

Send what data?

[Nowaker]

I do not see a technical advantage to supporting this.

The problem is, Docker users do see a technical advantage in this, as evidenced by the fact it's the TOP1 most upvoted ticket in this repo. https://github.com/docker/cli/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

[solvaholic]

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push ~multiple~ _a subset of_ tags

---

I'm trying to automate pushing of Docker images to Docker hub and to GitHub's package registry, and I'd like the latest release in a series to be reachable a couple different ways. For example when I first publish a new release with tag :2.165.2 I want these to be equivalent:

docker pull solvaholic/actions-runner:2.165.2
docker pull solvaholic/actions-runner:2
docker pull solvaholic/actions-runner:latest

So I want to push those three tags all at once. Looking for a way to do that led me to this issue, where I was reminded how to achieve it:

From @AntonKrug,

Ignoring the performance, the options either to push image with all tags or single tag seems bit odd.

@msbenjamin12,

I solved the problem by the following:
docker push [REPOSITORY]

example:
localhost:5000/someRepo:tag1
localhost:5000/someRepo:tag2
localhost:5000/someRepo:tag3

The following will push all the above example images to the REPOSITORY
docker push localhost:5000/someRepo

As a result tag1, tag2, and tag3 will be pushed to the REPO "localhost:5000/someRepo" recursively.

and @superstator:

No, that pushes ALL tags for a repository. For us at least we want to be able to push multiple specific tags, but not just blanket push the entire repo.

For me the solution is to manage my image tags such that this syntax will push all, and only, the tags I'd like to push:

docker push [registry/]owner/repository

docker push already knows how to push one or all local tags for a repository. The request in this issue is to be able to push _some_ tags for a repository.

[alitoufighi]

About what @msbenjamin12 said in https://github.com/docker/cli/issues/267#issuecomment-574420403, is there any specific implementation when tag is not set? (The one we all are here to achieve: The parallelism)
I'm thinking is it better to remove excess old tags before pushing and use this approach to push all of them at once, or running push multiple times?

EDIT: I tested the xargs approach mentioned above and it prints out the dialog The push refers to repository ... for the number of tags (as expected), but pushing without tags prints it only once. So I'm suspecting this to be an specific implementation or sth which can speed up pushing.

EDIT 2: Reading this method, I see without specifying tags (options.All), the whole repository (that I think is the name authors gave to "without-tag-path" of an image) is pushed all in one API call (while it's not released yet).

[ebd2]

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push ~multiple~ _a subset of_ tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

[AlphaQx]

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push ~multiple~ _a subset of_ tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

This is really funny comment

[ebd2]

As this has rather unexpectedly gone viral and hit the top of HackerNews (ensuing discussion here), I'd like to apologize for any headaches this causes for the docker maintainers. Thank you for your hard work, and I appreciate that fulfilling any feature request (even one as apparently popular as this one?) is a balance of trade-offs.

Btw, best of luck for your future endeavors @ebd2

Thank you @habibiefaried!

If anybody is wondering what I've been doing in the several years since I opened this issue, I commented on HN here.

[hasokeric]

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push ~multiple~ _a subset of_ tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

He will be back! Soon.

[seanmcoleman]

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push ~multiple~ _a subset of_ tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

May I recommend the SawStop?

[htoooth]

sound good

[warent]

@docyx Cool your jets. If you disagree with a comment, you can also use a 👎 or 😕 reaction instead of filling up feeds with off-topic verbal abuse. @AlphaQx did not deserve that.

[navilan]

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push ~multiple~ _a subset of_ tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

This is really funny comment

If it's a funny comment, use the 😄 reaction, don't fill up my fucking feed with this bullshit

Fuck you

This is a really funny comment :)

[Oymate]

@ devs limit discussion

[Fasani]

@solvaholic do you at least sell it online? 😂

[maxsatula]

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

He will be back! Soon.

If that really happens, hope he wasn't an Emacs user. As it strictly requires all the 10 fingers (sometimes even more), you know...

[ebd2]

If that really happens, hope he wasn't an Emacs user. As it strictly requires all the 10 fingers (sometimes even more), you know...

At the risk of taking us even further off topic (and further vexing the Docker maintainers), I broke my right wrist almost 20 years ago, and I find Emacs outrageously uncomfortable to use with a nomal-ish keyboard layout (QWERTY with a couple of remaps to make it closer to the old-school Sun layout). As I don't have this issue with vi/vim, that's my preferred editor.

And yes, I still futz around with code from time to time, just not for money. It's a lot more fun that way.

[osher]

@cpuguy83 - I'm not afraid of loops. This is not my problem.

The use-case is to avoid instability where some tags have been published and some have not.
Looping makes a separate API call per tag, and the danger is a network hickup in the middle of this loop and a careless CI developer somewhere out there.

What I expect from such API is to apply ALL tags provided and exit well, or NONE of them and exit with an error.

Use-Case:
Consider a docker-based containerized micro-services mesh.
Consider a CI of a mono-repo that builds it all and rolls out all applicable changes
Consider a strategy that relays on tag latest.

• sounds familiar, or not common enough? even without the latest tag it's still a thing.

If following a build only a part of the images that should have been tagged latest have in deed accepted the tag - we're in an unstable state which I hoped to avoid.

The solution now is a whole bunch of scriptology that handles many concerns and more, where the true concerns of my code as a user are naming the images and sending success/fail notifications from the CI.

Perhaps this requires evolution also on the registry side - I'm not involved enough to know.
But if we mean to provide a good service to our users - this should be the mindset.

[antonu17]

but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

The best description for a Debezium Kafka Connector ever

[ferrywlto]

@cpuguy83 - I'm not afraid of loops. This is not my problem.

The use-case is to avoid instability where some tags have been published and some have not.
Looping makes a separate API call per tag, and the danger is a network hickup in the middle of this loop and a careless CI developer somewhere out there.

What I expect from such API is to apply ALL tags provided and exit well, or NONE of them and exit with an error.

Use-Case:
Consider a docker-based containerized micro-services mesh.
Consider a CI of a mono-repo that builds it all and rolls out all applicable changes
Consider a strategy that relays on tag latest.

* sounds familiar, or not common enough? even without the `latest` tag it's still a thing.

If following a build only a part of the images that should have been tagged latest have in deed accepted the tag - we're in an unstable state which I hoped to avoid.

The solution now is a whole bunch of scriptology that handles many concerns and more, where the true concerns of my code as a user are naming the images and sending success/fail notifications from the CI.

Perhaps this requires evolution also on the registry side - I'm not involved enough to know.
But if we mean to provide a good service to our users - this should be the mindset.

In addition, executing the command multiple times for the same image with different tags could be a problem to some CI/CD pipeline that cost network traffic.

[cpuguy83]

@osher Atomic push of multiple tags is indeed something that would need to live within the registry specification. This sounds like a different case for sure.

@ferrywlto

In addition, executing the command multiple times for the same image with different tags could be a problem to some CI/CD pipeline that cost network traffic.

I'm not sure I understand this.