Packer: Post-processor docker-tag or docker-push don't work after shell-local

[liwo]

Overview of the Issue

I want to run tests on a created docker image before pushing to the registry. As I don't want any tests to influence the resulting image I want to run the tests as a post processor.

I created a post-processor definition like the following:

    "post-processors": [
        [
            {
                "type": "docker-tag",
                "repository": "my.registry/{{user `imageName`}}",
                "tags": ["{{user `imageVersion`}}"]
            },
            {
                "type": "shell-local",
                "environment_vars": ["IMAGE_ID=my.registry/{{user `imageName`}}:{{user `imageVersion`}}"],
                "inline": ["./bin/serverspec.sh spec/image"]
            },
            {
                "type": "docker-tag",
                "repository": "my.registry/{{user `imageName`}}",
                "tags": ["latest"]
            },
            {
                "type": "docker-push",
            }
        ]
    ]

According to documentation, shell-local post processor should pass the given artifact as is, therefore I expect this to work. But I get the following error message in packer 1.6.2:

==> docker: Running post-processor: docker-tag
Build 'docker' errored after 37 seconds 117 milliseconds: 1 error(s) occurred:

* Post-processor failed: Unknown artifact type: 
Can only tag from Docker builder artifacts.

Reproduction Steps

{
    "builders": [
        {
            "type": "docker",
            "image": "ubuntu:20.04",
            "commit": true
        }
    ],
    "provisioners": [
        {
            "type": "shell",
            "inline": [
                "true"
            ]
        }
    ],
    "post-processors": [
        [
            {
                "type": "docker-tag",
                "repository": "my.registry/test",
                "tags": ["1.0"]
            },
            {
                "type": "shell-local",
                "inline": ["true"]
            },
            {
                "type": "docker-tag",
                "repository": "my.registry/test",
                "tags": ["latest"]
            },
            {
                "type": "docker-push"
            }
        ]
    ]
}

Packer version

1.6.2

Operating system and Environment details

Docker version 18.06.3-ce, build d7080c1
Host: ubuntu 18.04 amd64

Log Fragments and crash.log files

==> docker: Committing the container
2020/09/25 10:32:09 packer-bin plugin: Committing container with args: [commit 3ac627794b4413f65a0b07b86c52122f2d67e5fb7a9dd847fb28278f50aec981]
    docker: Image ID: sha256:323f6f9495a26fce5007157fe2b7b525c1cfddcf761e1e4f27998562cdfc7e84
==> docker: Killing the container: 3ac627794b4413f65a0b07b86c52122f2d67e5fb7a9dd847fb28278f50aec981
2020/09/25 10:32:10 [INFO] (telemetry) ending docker
==> docker: Running post-processor: docker-tag
2020/09/25 10:32:10 [INFO] (telemetry) Starting post-processor docker-tag
    docker (docker-tag): Tagging image: sha256:323f6f9495a26fce5007157fe2b7b525c1cfddcf761e1e4f27998562cdfc7e84
    docker (docker-tag): Repository: my.registry/test:1.0
2020/09/25 10:32:10 [INFO] (telemetry) ending docker-tag
==> docker: Running post-processor: shell-local
2020/09/25 10:32:10 Flagging to keep original artifact from post-processor 'docker-tag'
2020/09/25 10:32:10 [INFO] (telemetry) Starting post-processor shell-local
2020/09/25 10:32:10 packer-bin plugin: [INFO] (shell-local): Prepending inline script with #!/bin/sh -e
==> docker (shell-local): Running local shell script: /tmp/packer-shell842372160
2020/09/25 10:32:10 packer-bin plugin: [INFO] (shell-local): starting local command: /bin/sh -c PACKER_BUILDER_TYPE='docker' PACKER_BUILD_NAME='docker'  /tmp/packer-shell842372160
2020/09/25 10:32:10 packer-bin plugin: [INFO] (shell-local communicator): Executing local shell command [/bin/sh -c PACKER_BUILDER_TYPE='docker' PACKER_BUILD_NAME='docker'  /tmp/packer-shell842372160]
2020/09/25 10:32:10 [INFO] (telemetry) ending shell-local
2020/09/25 10:32:10 [INFO] (telemetry) Starting post-processor docker-tag
==> docker: Running post-processor: docker-tag
2020/09/25 10:32:10 [INFO] (telemetry) ending docker-tag

* Post-processor failed: Unknown artifact type: 
Can only tag from Docker builder artifacts.
Build 'docker' errored after 4 seconds 853 milliseconds: 1 error(s) occurred:
==> Wait completed after 4 seconds 853 milliseconds
2020/09/25 10:32:10 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2020/09/25 10:32:10 machine readable: docker,error []string{"1 error(s) occurred:\n\n* Post-processor failed: Unknown artifact type: \nCan only tag from Docker builder artifacts."}

* Post-processor failed: Unknown artifact type: 
Can only tag from Docker builder artifacts.
==> Builds finished but no artifacts were created.
2020/09/25 10:32:10 [INFO] (telemetry) Finalizing.

* Post-processor failed: Unknown artifact type: 
Can only tag from Docker builder artifacts.

==> Wait completed after 4 seconds 853 milliseconds

==> Some builds didn't complete successfully and had errors:
--> docker: 1 error(s) occurred:

* Post-processor failed: Unknown artifact type: 
Can only tag from Docker builder artifacts.

==> Builds finished but no artifacts were created.

[sylviamoss]

Hey there, I implemented a possible solution for this. Could you try out the binaries and let me know if it works for you?
https://app.circleci.com/pipelines/github/hashicorp/packer/7325/workflows/9d990fb5-b067-46b2-8424-61e70948aff3/jobs/83406/artifacts

[liwo]

Hi Sylvia,

thank you for your very quick response. I can confirm that the shell-local post-processor now passes the artifact correctly and the docker-tag post-processor after it receives the artifact and continues on. Unfortunately I now get an error message I can't explain:

* Post-processor failed: unexpected EOF

This happens with above minimal example as well as my real template. If I remove the "docker-push" step from the template the error disappears.

Full build log from above minimal example: https://gist.github.com/liwo/da9535a59daea599e24288495865cdb0

Can you please have another look?

[sylviamoss]

Thanks for testing! While writing the fix I changed a docker-push line of code which caused the error, but now is fixed. Could you try out the new binaries, please? Here they are https://app.circleci.com/pipelines/github/hashicorp/packer/7339/workflows/ba8b6550-1b1c-4721-a1d8-a4002a9346e5/jobs/83592/artifacts

[liwo]

Hi Sylvia,

thanks for taking care again! The minimal example now errors out at docker push because "my.registry" is obviously not a real registry, but that's totally fine. My real template now runs through without error and works as expected.

Thank you for your time and your really fast fix!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.