Packer: OpenStack delete old image after new one is created

[nickcarenza]

Builder: OpenStack
Provisioner: Custom (salt-minion)
Intended Issue Type: Enhancement

Summary: Need a way to delete old image with same after new image is successfully built.

I have a packer template that builds a base image that many other templates use to build their images. A new base image build is triggered whenever changes are made to the common Salt configs. A build of the base image then triggers rebuilds of the other images so that they are up to date as well.

The first problem was finding a way to pass the source_image id of the latest common image to the other builders. I solved this by setting a required variables in the template and using the nova client to get the image ID by name. It's not pretty but it works.

packer build -var "source_image=`nova image-list | grep "Ubuntu 14.04 Common" | awk '{print $2}' | head -n1`" dev.json

The second problem I have is that I can't find a way with packer to _replace_ the old image after the new one is built. The output from the OpenStack nova cli isn't detailed enough to show which image is newer.

So after multiple builds, I end up with multiple images in OpenStack with the same name.

~  nova image-list
+--------------------------------------+---------------------------------+
| ID                                   | Name                            |
+--------------------------------------+---------------------------------+
| 4867d6d2-2c9d-4f42-bf62-74cdcbf71da1 | Public Records App              |
| d8898af1-abf0-41b7-981d-229b8368d40f | Public Records App              |
| 6bc15dc5-81b4-4d08-8aea-4e3914902ffd | Ubuntu 14.04 (Trusty)           |
| bf239971-e888-4efc-b3d9-04b40a40df09 | Ubuntu 14.04 Common             |
+--------------------------------------+---------------------------------+

My current solution is to make a custom post-processor that gets more detailed information from the OpenStack API and deletes the older of the two images with the same name.

Thanks for your time.

[nickcarenza]

In case anyone is looking for a way to do this, here is the command I am using at the moment:

export OLD_IMAGE_ID=`curl -s -X GET $OS_GLANCE_URL/images -H "X-Auth-Token: $OS_TOKEN" -H "Content-Type: application/json" | jq '.images | map(select(.name=="You Image Name Here")) | min_by(.created_at).id' | sed s/\"//g`
glance image-delete $OLD_IMAGE_ID

_Assumes you've set the appropriate environment variables to use the glance cli and have exported OS_TOKEN from an authorization request._

This command gets a list of images as json, filters them by name then gets the oldest one based on created_at date. Then passes that id to the glance cli and deletes that image.

[petehayes]

My workaround is to create the new image suffixed with a timestamp and if packer build completes succesfully, delete the original image and rename the suffixed image to the original image name.

[mitchellh]

If Packer successfully creates an image, it is not Packer's responsibility to clean that image up. Packer is not an image management system, it is just a creation tool. In your case where you're putting it into automation, I can see how it would be nice for Packer to be able to remove them, but it dramatically simplifies the scope and goals of this project to focus on creation only.

[hco]

Hey @mitchellh,

is this still how you see things?

As a user of packer & the openstack builder, I would love if packer would be able to "replace" an image within openstack.

[thejmazz]

FWIW, I came up with a script that cleans duplicate images (by name) from openstack, keeping only the most recent of each:

#!/usr/bin/env bash

# This script deletes openstack images which have the same name,
# keeping only the most recent one.

function get_dup_images () {
    # since uniq prefixes with spaces, the sed step cleans that up

    openstack image list -f value | awk '{print $2}' | sort | uniq -c | \
    sed 's/^ *\([0-9]*\) /\1 /' | \
    grep -v ^1 | awk '{print $2}'
}

function delete_all_but_latest_image () {
    # we explicity specify columns so that we know that last one is Name
    # so that it is safe to use grep $image_name$

    openstack image list -f value -c ID -c Name | \
    grep "$1$" | awk '{print $1}' | \
    xargs -I % bash -c 'echo % `openstack image show -f value -c created_at -c ID %`' | \
    sort -k2 --reverse | tail -n +2 | awk '{print $1}' | \
    # xargs -I % bash -c 'echo "Gonna delete image %"'
    xargs -I % bash -c 'openstack image delete % && echo "Deleted image %"'
}

for image in `get_dup_images`; do
    delete_all_but_latest_image $image
done

[vertrost]

If Packer successfully creates an image, it is not Packer's responsibility to clean that image up. Packer is not an image management system, it is just a creation tool. In your case where you're putting it into automation, I can see how it would be nice for Packer to be able to remove them, but it dramatically simplifies the scope and goals of this project to focus on creation only.

Well, then I wonder how you can explain, that for AWS packer can replace images with "force_deregister" option, and for Openstack there is no such option?

[azr]

Hey @vertrost; in the case you are mentioning; the image is deregistered when the name of the image is the same (so the image already exists) and force_deregister is true, which is technically simple to achieve, packer checks if a similarly named image exists and if so delete it.

I think it would be totally fine to open a PR that does the same for OpenStack. Please remember that the open stack builder is community maintained and that we won't allocate time to it. We will review PRs joyfully though :)

[rickard-von-essen]

@vertrost there is a very important difference. For AWS the AMI name have to be unique, for OpenStack multiple images can have the same name. With that in mind I would advise against deregistring images by name in OpenStack.

[vertrost]

@vertrost there is a very important difference. For AWS the AMI name have to be unique, for OpenStack multiple images can have the same name. With that in mind I would advise against deregistring images by name in OpenStack.

Why? To have a possibility to collect all images with specified name and deregister them, and after that register a new one - I see it as a useful feature, instead of having self-written workarounds.
Also, the question, what is the case for having images with the same name? Because I don't see any reason to have such. Different images - different names, it should be clear, no?

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.