Packer: docker-push support for multiple tags

Created on 29 Apr 2020  ·  11Comments  ·  Source: hashicorp/packer

I've reviewed the documentation and poured through the issues here, and I just can't seem to make docker-push behave in a way that makes sense to me. This might legitimately be a bug, but figured since the docker-push post-processor hasn't been touched in six years, it's considered "good enough" as-is and any additional functionality would be considered an enhancement.

Feature Description

I want to be able to have a docker-tag definition with multiple tags defined, and have all the tags pushed by the docker-push post-processor. Here's a naive example:

"post-processors":
  [
    [
      {
        "only": ["bionic"],
        "type": "docker-tag",
        "repository": "terradatum/systemd",
        "tag": "bionic,ubuntu-bionic,ubuntu-18.04"
      },
      {
        "type": "docker-push",
        "login": true,
        "login_username": "{{user `docker_username`}}",
        "login_password": "{{user `docker_password`}}"
      }
    ]
  ]

You can see that with the above, all three tags are applied, but only the last is pushed:

```shell script
2020-04-28T18:31:33-07:00: ==> bionic: Committing the container
2020-04-28T18:31:35-07:00: bionic: Image ID: sha256:e64e33a222142e6a51dc62788232e4d8e81c70c3347373be36bbfa638e2c1183
2020-04-28T18:31:35-07:00: ==> bionic: Killing the container: f143f41a61c2dabd87fa825d3c009fc8a1d121951a720a68cea8029bf2ad2bc6
2020-04-28T18:31:40-07:00: ==> bionic: Running post-processor: docker-tag
2020-04-28T18:31:40-07:00: bionic (docker-tag): Tagging image: sha256:e64e33a222142e6a51dc62788232e4d8e81c70c3347373be36bbfa638e2c1183
2020-04-28T18:31:40-07:00: bionic (docker-tag): Repository: terradatum/systemd:bionic
2020-04-28T18:31:40-07:00: ==> bionic: Running post-processor: docker-tag
2020-04-28T18:31:40-07:00: bionic (docker-tag): Tagging image: terradatum/systemd:bionic
2020-04-28T18:31:40-07:00: bionic (docker-tag): Repository: terradatum/systemd:ubuntu-bionic
2020-04-28T18:31:40-07:00: ==> bionic: Running post-processor: docker-tag
2020-04-28T18:31:40-07:00: bionic (docker-tag): Tagging image: terradatum/systemd:ubuntu-bionic
2020-04-28T18:31:40-07:00: bionic (docker-tag): Repository: terradatum/systemd:ubuntu-18.04
2020-04-28T18:31:40-07:00: ==> bionic: Running post-processor: docker-push
2020-04-28T18:31:40-07:00: bionic (docker-push): Logging in...
2020-04-28T18:31:41-07:00: bionic (docker-push): WARNING! Your password will be stored unencrypted in /home/rbellamy/.docker/config.json.
2020-04-28T18:31:41-07:00: bionic (docker-push): Login Succeeded
2020-04-28T18:31:41-07:00: bionic (docker-push): Configure a credential helper to remove this warning. See
2020-04-28T18:31:41-07:00: bionic (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2020-04-28T18:31:41-07:00: bionic (docker-push): Pushing: terradatum/systemd:ubuntu-18.04
2020-04-28T18:31:41-07:00: bionic (docker-push): The push refers to repository [docker.io/terradatum/systemd]
2020-04-28T18:31:42-07:00: bionic (docker-push): d3536ca2000e: Preparing
2020-04-28T18:31:42-07:00: bionic (docker-push): 28ba7458d04b: Preparing
2020-04-28T18:31:42-07:00: bionic (docker-push): 838a37a24627: Preparing
2020-04-28T18:31:42-07:00: bionic (docker-push): a6ebef4a95c3: Preparing
2020-04-28T18:31:42-07:00: bionic (docker-push): b7f7d2967507: Preparing
2020-04-28T18:31:42-07:00: bionic (docker-push): 838a37a24627: Layer already exists
2020-04-28T18:31:42-07:00: bionic (docker-push): 28ba7458d04b: Layer already exists
2020-04-28T18:31:42-07:00: bionic (docker-push): b7f7d2967507: Layer already exists
2020-04-28T18:31:43-07:00: bionic (docker-push): a6ebef4a95c3: Layer already exists
2020-04-28T18:32:07-07:00: bionic (docker-push): d3536ca2000e: Pushed
2020-04-28T18:32:12-07:00: bionic (docker-push): ubuntu-18.04: digest: sha256:5935f2fc152e81d849ce02b4e32feda2677e7a6aa58589dfcaa6d687dbe89fc8 size: 1364
2020-04-28T18:32:12-07:00: bionic (docker-push): Logging out...
2020-04-28T18:32:12-07:00: bionic (docker-push): Removing login credentials for https://index.docker.io/v1/
2020-04-28T18:32:12-07:00: Build 'bionic' finished.

Instead, I have to do something like this to push multiple tags:

```yaml
  "post-processors": [
    [
      {
        "type": "docker-tag",
        "repository": "terradatum/systemd",
        "tag": "bionic",
        "only": [
          "bionic"
        ]
      },
      {
        "type": "docker-push",
        "login": true,
        "login_username": "{{user `docker_username`}}",
        "login_password": "{{user `docker_password`}}",
        "only": [
          "bionic"
        ]
      }
    ],
    [
      {
        "type": "docker-tag",
        "repository": "terradatum/systemd",
        "tag": "ubuntu-bionic",
        "only": [
          "bionic"
        ]
      },
      {
        "type": "docker-push",
        "login": true,
        "login_username": "{{user `docker_username`}}",
        "login_password": "{{user `docker_password`}}",
        "only": [
          "bionic"
        ]
      }
    ],
    [
      {
        "type": "docker-tag",
        "repository": "terradatum/systemd",
        "tag": "ubuntu-18.04",
        "only": [
          "bionic"
        ]
      },
      {
        "type": "docker-push",
        "login": true,
        "login_username": "{{user `docker_username`}}",
        "login_password": "{{user `docker_password`}}",
        "only": [
          "bionic"
        ]
      }
    ]
  ]

Use Case(s)

Docker supports multiple tags, so does the docker-tag post-processor. It seems reasonable to me that my first example would "just work":tm:.

enhancement post-processodocker
Source
picture rbellamy

Most helpful comment

As a side note, it would make more sense to use tags keyword now in place of tag to avoid confusion.

picture nqb on 6 May 2020
👍3

All 11 comments

Oops, looks like while this was implemented in https://github.com/hashicorp/packer/pull/8392, we never updated the docs. I'll make a PR to fix that now. Try supplying the tags as a []string rather than a single string. e.g.: "tag": ["bionic", "ubuntu-bionic", "ubuntu-18.04"]

SwampDragons picture SwampDragons on 29 Apr 2020

SO SIMPLE, yet so important! I can't believe I didn't try that! Gonna go give that a shot right now.

rbellamy picture rbellamy on 30 Apr 2020

I'm afraid that didn't do it for me. This has the effect of only pushing the last tag.

 "post-processors": [
    [
      {
        "type": "docker-tag",
        "repository": "terradatum/systemd",
        "tag": ["bionic","ubuntu-bionic","ubuntu-18.04"],
        "only": [
          "bionic"
        ]
      },
      {
        "type": "docker-push",
        "login": true,
        "login_username": "{{user `docker_username`}}",
        "login_password": "{{user `docker_password`}}",
        "only": [
          "bionic"
        ]
      }
    ]
  ]

2020-04-29T17:24:51-07:00: ==> bionic: Committing the container
2020-04-29T17:24:54-07:00:     bionic: Image ID: sha256:d69252de476c9e6d89344a9016209599d0ce114bbc6cf3e7ac9342cd6fc2f6c4
2020-04-29T17:24:54-07:00: ==> bionic: Killing the container: b109b937b5ff9319fb718ce31f5c70999965cce4dd0e70d8f8029e7cfb6fb042
2020-04-29T17:24:57-07:00: ==> bionic: Running post-processor: docker-tag
2020-04-29T17:24:57-07:00:     bionic (docker-tag): Tagging image: sha256:d69252de476c9e6d89344a9016209599d0ce114bbc6cf3e7ac9342cd6fc2f6c4
2020-04-29T17:24:57-07:00:     bionic (docker-tag): Repository: terradatum/systemd:bionic
2020-04-29T17:24:57-07:00:     bionic (docker-tag): Tagging image: sha256:d69252de476c9e6d89344a9016209599d0ce114bbc6cf3e7ac9342cd6fc2f6c4
2020-04-29T17:24:57-07:00:     bionic (docker-tag): Repository: terradatum/systemd:ubuntu-bionic
2020-04-29T17:24:58-07:00:     bionic (docker-tag): Tagging image: sha256:d69252de476c9e6d89344a9016209599d0ce114bbc6cf3e7ac9342cd6fc2f6c4
2020-04-29T17:24:58-07:00:     bionic (docker-tag): Repository: terradatum/systemd:ubuntu-18.04
2020-04-29T17:24:58-07:00: ==> bionic: Running post-processor: docker-push
2020-04-29T17:24:58-07:00:     bionic (docker-push): Logging in...
2020-04-29T17:24:59-07:00:     bionic (docker-push): WARNING! Your password will be stored unencrypted in /home/rbellamy/.docker/config.json.
2020-04-29T17:24:59-07:00:     bionic (docker-push): Configure a credential helper to remove this warning. See
2020-04-29T17:24:59-07:00:     bionic (docker-push): Login Succeeded
2020-04-29T17:24:59-07:00:     bionic (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2020-04-29T17:24:59-07:00:     bionic (docker-push): Pushing: terradatum/systemd:ubuntu-18.04
2020-04-29T17:24:59-07:00:     bionic (docker-push): The push refers to repository [docker.io/terradatum/systemd]
2020-04-29T17:24:59-07:00:     bionic (docker-push): a018cc02e65f: Preparing
2020-04-29T17:24:59-07:00:     bionic (docker-push): 28ba7458d04b: Preparing
2020-04-29T17:24:59-07:00:     bionic (docker-push): 838a37a24627: Preparing
2020-04-29T17:24:59-07:00:     bionic (docker-push): a6ebef4a95c3: Preparing
2020-04-29T17:24:59-07:00:     bionic (docker-push): b7f7d2967507: Preparing
2020-04-29T17:25:00-07:00:     bionic (docker-push): 838a37a24627: Layer already exists
2020-04-29T17:25:01-07:00:     bionic (docker-push): b7f7d2967507: Layer already exists
2020-04-29T17:25:01-07:00:     bionic (docker-push): 28ba7458d04b: Layer already exists
2020-04-29T17:25:01-07:00:     bionic (docker-push): a6ebef4a95c3: Layer already exists
2020-04-29T17:25:26-07:00:     bionic (docker-push): a018cc02e65f: Pushed
2020-04-29T17:25:30-07:00:     bionic (docker-push): ubuntu-18.04: digest: sha256:fea8afd291213f85f97a7989b9dae38bfd2906ab2cf09f07a5c4121f02aef4c8 size: 1364
2020-04-29T17:25:30-07:00:     bionic (docker-push): Logging out...
2020-04-29T17:25:30-07:00:     bionic (docker-push): Removing login credentials for https://index.docker.io/v1/
2020-04-29T17:25:30-07:00: Build 'bionic' finished.

==> Builds finished. The artifacts of successful builds are:
--> bionic: Imported Docker image: sha256:d69252de476c9e6d89344a9016209599d0ce114bbc6cf3e7ac9342cd6fc2f6c4
--> bionic: Imported Docker image: terradatum/systemd:ubuntu-18.04
--> bionic: Imported Docker image: terradatum/systemd:ubuntu-18.04
rbellamy picture rbellamy on 30 Apr 2020

For completeness, #8392 was for the docker-tag post-processor, which you can see above IS doing its job by tagging the image three times. I think this issue should be re-opened.

rbellamy picture rbellamy on 30 Apr 2020

@SwampDragons could you re-open this issue, please?

rbellamy picture rbellamy on 1 May 2020

@rbellamy, you don't need to use a JSON list for tags, only a comma-separated list

nqb picture nqb on 1 May 2020

So documentation need to be adapt and issue re-opened.

nqb picture nqb on 1 May 2020

In fact, it sounds more like a bug because if we expected tag to be an array of strings.

I opened a new PR but it doesn't fix the bug, only example.

What do you want to do @SwampDragons ?

nqb picture nqb on 1 May 2020

@rbellamy @nqb thanks for following up here. I reopened the issue so that we can take a look at the updated information and figure out what's be going on.

nywilken picture nywilken on 1 May 2020
👍1

As a side note, it would make more sense to use tags keyword now in place of tag to avoid confusion.

nqb picture nqb on 6 May 2020
👍3

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashibot[bot] picture hashibot[bot] on 7 Jun 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mvermaes picture mvermaes  ·  3Comments
DanielBo picture DanielBo  ·  3Comments
PartyImp picture PartyImp  ·  3Comments
wduncanfraser picture wduncanfraser  ·  3Comments
mushon4 picture mushon4  ·  3Comments