Packer: digitalocean-import post-processor is ... broken?

[danryu]

Overview of the Issue

I'm trying to create Custom Images in digitalocean using Packer but the post-processor always fails with

* Post-processor failed: unexpected EOF

I am using as generic as possible a template, and the post-processor config is taken directly from
https://www.packer.io/docs/post-processors/digitalocean-import#basic-example
and I am using all the required options.
Can you please tell me what is wrong at least? The docs are very light and the log output isn't very helpful.

Reproduction Steps

Run Packer build on the below buildfile.

Packer version

1.6.2

Simplified Packer Buildfile

{
    "variables": {
      "do_api_token": "aabbcc",
      "do_spaces_key": "ddeeffgg",
      "do_spaces_secret": "gghhiioo"
    },
    "builders": [
        {
            "type": "digitalocean",
            "api_token": "{{user `do_api_token`}}",
            "image": "ubuntu-20-04-x64",
            "region": "ams3",
            "size": "c-2",
            "ssh_username": "root"
        }
    ],
    "post-processors": [
        {
            "type": "digitalocean-import",
            "api_token": "{{user `do_api_token`}}",
            "spaces_key": "{{user `do_spaces_key`}}",
            "spaces_secret": "{{user `do_spaces_secret`}}",
            "spaces_region": "ams3",
            "space_name": "space1",
            "image_name": "somename1",
            "image_description": "Packer import {{timestamp}}",
            "image_regions": ["nyc3", "nyc2"],
            "image_tags": ["custom", "packer"]
        }
    ]
}

Operating system and Environment details

Kubuntu 20.04
Packer installed from download at packer.io.
DigitalOcean account, working Space at "space1" in region "ams3".

Log Fragments and crash.log files

2020/08/30 12:33:46 packer-builder-digitalocean plugin: Looking up snapshot ID for snapshot: packer-1598783449
2020/08/30 12:33:47 packer-builder-digitalocean plugin: Snapshot image ID: 69266684
==> digitalocean: Destroying droplet...
==> digitalocean: Deleting temporary ssh key...
2020/08/30 12:33:50 [INFO] (telemetry) ending digitalocean
2020/08/30 12:33:50 [INFO] (telemetry) Starting post-processor digitalocean-import
==> digitalocean: Running post-processor: digitalocean-import
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: Rendered space_object_name as packer-import-1598783449
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: Looking for image in artifact
Build 'digitalocean' errored after 3 minutes 2 seconds: 1 error(s) occurred:
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: panic: runtime error: index out of range [0] with length 0
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: 
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: goroutine 43 [running]:
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: github.com/hashicorp/packer/post-processor/digitalocean-import.(*PostProcessor).PostProcess(0xc000cb6000, 0x53feac0, 0xc0000d1fc0, 0x541e720, 0xc0002b5440, 0x541e780, 0xc0002f2100, 0x1, 0x203000, 0x28, ...)
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin:   /home/circleci/project/packer/post-processor/digitalocean-import/post-processor.go:183 +0x164e
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: github.com/hashicorp/packer/packer/rpc.(*PostProcessorServer).PostProcess(0xc00030a8c0, 0x1, 0xc000c820f0, 0x0, 0x0)
2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin:   /home/circleci/project/packer/packer/rpc/post_processor.go:109 +0x277

2020/08/30 12:33:50 packer-post-processor-digitalocean-import plugin: reflect.Value.call(0xc0002fc420, 0xc0005c15d8, 0x13, 0x4bbd471, 0x4, 0xc000c63f18, 0x3, 0x3, 0x0, 0x405400, ...)
* Post-processor failed: unexpected EOF

[sylviamoss]

Hey there! The digitalocean-import post-processor is used to import images created with other builders and importing the artifact to DigitalOcean. If you are creating a custom image with DigitalOcean builder, then this builder will create this custom image snapshot in your DigitalOcean account and the post-processor is not necessary.
Is there any specific reason you added the digitalocean-import post-processor to your template with the digitalocean builder?

[nywilken]

Unrelated to @sylviamoss question about the reasoning for trying to use the digitalocean-import. It looks like you found a bug with the code that causes Packer to crash. I will open a PR to fix the panic.

[danryu]

@sylviamoss
I wasn't previously aware that I could use snapshots in all regions as I expected to with custom images - am now using snapshots fine.
It would be good to make this clear in the docs. Thanks for clarifying though.

[nywilken]

@danryu thanks for the follow up. I went ahead and opened a PR to handle the panic. But I would like to include any documentation you think would've made things a littler clearer in this case. Can you add your thoughts to this thread?

Thanks!

[danryu]

Hey @nywilken
Really, just what @sylviamoss said: _"The digitalocean-import post-processor is used to import images created with other builders and importing the artifact to DigitalOcean. If you are creating a custom image with DigitalOcean builder, then this builder will create this custom image snapshot in your DigitalOcean account and the post-processor is not necessary."_
Or something along those lines!

I think my problem was that I (vaguely) remember a time when Snapshots were "second-class citizens" to Custom Images on DigitalOcean - I've been using DO a long time. I seem to remember that Droplets instantiated from Snapshots took a lot longer than from Custom Images, and had more limited regional availability - however, this is a vague memory and I may have it wrong anyway.

I may be in a very small minority of people who might get confused over this :)
Nonetheless, what @sylviamoss said cleared it up instantly for me, so I think it worth including as a note in the docs.
After all, creating a Custom Image (as opposed to a Snapshot) in DO using Packer seems like a sensible enough objective - I had just assumed that the Snapshot was being created as a transitional phase towards creating the Custom Image.
Now I see that the Snapshot has feature-parity with the Custom Image, I'm good to go :)

[nywilken]

Thanks for the added context @danryu. If it's confusing to one then there's room for improvement :wink: I added a small note to the docs in a separate HCL2 docs PR #9865.

Thanks again for opening up this issue, and for your time to provide the additional feedback. Cheers!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.