Terraform-provider-google: Create a data source for "latest" disk images

[catsby]

Create a data.disk_image (naming is hard) data source so users can easily get the full self_link for a given disk image from a family:

• https://cloud.google.com/compute/docs/reference/latest/images#resource
• https://cloud.google.com/compute/docs/reference/latest/images/getFromFamily

Should have similar features as data.aws_ami:

• https://www.terraform.io/docs/providers/aws/d/ami.html

Example proposed usage:

data "google_disk_image" "latest_debian" {
  most_recent = true
  family = "debian-8"
}

resource "google_compute_disk" "default" {
  name  = "test-disk"
  type  = "pd-ssd"
  zone  = "us-central1-a"
  image = "${data.google_disk_image.latest_debian.self_link}"
}

We've removed the shorthand support from google_compute_disk, so we should provide a data source to allow users to use it indirectly via a data source.

/cc @danawillow related to #1

[danawillow]

fyi @paddycarver because you were saying this might not actually be necessary, we might just be able to fix the shorthand.

[paddycarver]

So I talked with @catsby about this yesterday and think I've found the bottom of this particular issue. The long and short of it is I need to read more closely.

Instance _templates_ can use family names, and that's an important feature. Instances and disks _cannot_. The API doesn't accept it.

So the solution is to make sure only instance template use the family specification. That's a small modification to resolveImage.

There are actually a few features at play here:

1. Using shorthands, which is important both for UX and for matching API inputs.
2. Using families (which is what #1 was fixing) which is important for instance templates, but _cannot_ be used with disks and instances.
3. Obtaining the latest image in a family, which is useful for instances and disks, and can be used to pin instance templates at the latest image instead of having them detect the latest image at runtime, enforcing homogeneity.

This issue just addresses that third point, and I think is useful. The first two points I'll address in a quick update to resolveImage and the documentation.

Sorry for all the confusion on this one, was entirely caused by me not looking closely enough.

[danawillow]

Coming back around to this- do you have documentation for that? I've definitely created disks on instances with just a family name lately and it created just fine. I think the problem isn't that the API doesn't accept it, but that when it's returned from the API it has a full name, not just the family name we specified.

[visar]

What's the status on this? How can Terraform (terraform plan/apply) detect the latest image from an image family?

[galdor]

Anything new ? We have this exact problem here, i.e. we would like to use terraform to detect new images in a family and use them.

[pdilyard]

For anyone who needs this feature and can't wait for it to be added, I hacked together this:

1. Create a module called "describe_image_family" for example
2. The contents of the module's main.tf:

variable "family" {}

data "external" "image_name" {
  program = ["sh", "${path.module}/script.sh"]

  query = {
    family = "${var.family}"
  }
}

output "latest" {
  value = "${data.external.image_name.result.name}"
}

1. Create a file called script.sh in your module folder, its contents:

# Exit if any steps fail
set -e

# Parse out the "family" variable from the JSON input
eval "$(jq -r '@sh "FAMILY=\(.family)"')"

# Execute the gcloud command needed to pull the latest family image
NAME=$(gcloud compute images describe-from-family $FAMILY | grep name | sed -e 's/name: //')

# Output JSON
jq -n --arg name "$NAME" '{"name":$name}'

1. Use your new module like this:

module "nginx" {
  source = "./modules/describe_image_family"
  family = "nginx"
}

resource "google_compute_instance" "test-instance" {
  ...
  boot_disk {
    initialize_params {
      image = "${module.nginx.latest}"
    }
  }
  ...
}

This uses the gcloud command line tool to describe the image family, so be sure you're authenticated with the tool. It also uses jq to parse JSON, so that needs to be installed or you need to replace the shell script with something that parses in another way. To debug, try running this from your shell:

echo '{"family":"nginx"}' | sh ./modules/describe_image_family/script.sh

That should output JSON with the name of your latest image.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!