Packer: openstack builder hangs when using use_blockstorage_volume if volume size is too small

[puetzk]

From https://github.com/hashicorp/packer/issues/6956#issuecomment-435854310

If the value determined by GetVolumeSize is actually too low, perhaps an image with no value set for MinDiskGigabytes, in a format (e.g. qcow2) where the file is considerably smaller than the minimum volume that can unpack it, the volume is created but the status shown in cinder list becomes "error". WaitForVolume doesn't seem to recognize this case, and just keeps looping (and waiting) indefinitely), since volumes.Get is succeeding (the GUID does exist and has a status to read), and "error" != "available".

[puetzk]

Note by hangs, I just mean that it reports:

Waiting for volume packer_<...> (volume id: <...) to become available...

and then keeps polling it indefinitely (for hours at least). If I manually remove the status="error" volume, it notices the polling failure and exits. So it's not really 'hung' so much as just failing to make progress or give up.

==> openstack: Error waiting for volume: Resource not found
Build 'openstack' errored: Error waiting for volume: Resource not found

==> Some builds didn't complete successfully and had errors:
[focus-cloud-esf | ubuntu-xenial] --> openstack: Error waiting for volume: Resource not found

[jo]

Hi, we also have the problem - is there a possibility to set the size of the volume? Or another workaround?

[SwampDragons]

I don't believe we currently have that feature, but PRs are always welcome :)

[jo]

Hi @SwampDragons, sadly I don't have Go skills yet :)

This problem makes it impossible to create an image other than the default RAW, which results in very large volumes (25GB in our case vs ~500MB for QCOW2).

This is our build definition:

{
  "builders": [{
    "type": "openstack",
    "ssh_username": "root",
    "image_name": "poc-symphony-api-server",
    "source_image_name": "ubuntu-16.04-server-cloudimg-amd64",
    "image_disk_format": "qcow2",
    "use_blockstorage_volume": true,
    "flavor": "m1.small",
    "networks": ["03d67faf-be0d-4d27-9044-a544a42c217d"],
    "security_groups": ["ssh"]
  }]
}

When we run it:

==> openstack: Loading flavor: m1.small
    openstack: Verified flavor. ID: f885c36d-d5d2-4421-a1f4-b0a68716ede7
==> openstack: Creating temporary keypair: packer_5c18d1de-d57f-9027-7772-cd93d33cfca1 ...
==> openstack: Created temporary keypair: packer_5c18d1de-d57f-9027-7772-cd93d33cfca1
    openstack: Found Image ID: 07fc4691-ee35-4394-887a-4850f5ba1300
==> openstack: Creating volume...
==> openstack: Waiting for volume packer_5c18d1de-f16a-becd-f126-62d32c79244e (volume id: 15602ad4-5420-451b-8998-738e43742a77) to become available...

now it hangs. OpenStack console shows for the volume:

Name: packer_5c18d1de-f16a-becd-f126-62d32c79244e
ID: 15602ad4-5420-451b-8998-738e43742a77
Status: Error 
Size: 1 GiB

Any hints of how we can create different volumes than RAW would be highly appreciated.

[SwampDragons]

This is a community-supported plugin, which means HashiCorp's devs aren't going to prioritize adding new features to it, but a size option should be fairly straightforward to add; if you add a VolumeSize field to the step here: https://github.com/hashicorp/packer/blob/master/builder/openstack/step_create_volume.go then you can do a check for whether VolumeSize is > 0 and put the GetVolumeSize code into a conditional.

I'm happy to review PRs for this.

[jo]

Thank you @SwampDragons, you really helped me getting started!

I made the pull request, it works for me and fixes our problem. I'd be happy if you could have a look.

[rickard-von-essen]

Fixed in #7130

[puetzk]

as far as I can see #7130 doesn't actually fix #6957 (WaitForVolume looks like it would still go into an endless loop if the volume.Status is "error" due to insufficient size and never reaches "available".

It does provide a workaround (since you could now raise volume_size to avoid the condition that hangs), but there were two issues and only #6956 is directly fixed.

[rickard-von-essen]

@puetzk thanks for the clarification!