Terraform: Capability control number of backends attached dynamically for GCP google_compute_backend_service

[universalvishwa]

Current Terraform Version

$ terraform -v
Terraform v0.11.10
+ provider.google v1.19.1

Use-cases

Pleas refer to the code snippet below.

# TCP load balancer Backend service
resource "google_compute_backend_service" "default" {
  name          = "tcplb-backend"
  description   = "TCP proxy load balancer backend config"
  port_name     = "dns"
  protocol      = "TCP"
  timeout_sec   = 30
  project       = "test-prohect-1649"

  backend {
    group = "${google_compute_instance_group.us.self_link}"
  }
  backend {
    group = "${google_compute_instance_group.eu.self_link}"
  }
  backend {
    group = "${google_compute_region_instance_group_manager.us-grp.instance_group}"
  }

  health_checks = ["${google_compute_health_check.default.self_link}"]
}

In this resource, the 'backend' is very clunky. The only way I can add another backend is by copying another block in to the resource even with similar config. It is true that, this requires a map to pass in. However, it is still not efficient to deal with this.

Attempted Solutions

In the documentation it says it accepts a list of backends. I tried passing in a list in the form of,

backend {
  group = ["${google_compute_instance_group.*.self_link}"]
}

But, it doesn't work. Somehow, what I would like to have is to be able to pass in a list of backends (for similar resources) and control the linkage more easily and dynamically.

Proposal

Either add the capability to support a list. separate the 'backend' into it's own resource and add the capability to add it to backend_sevice resource.

References

[apparentlymart]

Hi @universalvishwa!

I think what you have here is a different way of stating the problem covered over in #7034: in current versions of Terraform, it's not possible to dynamically generate nested block structures like backend.

As I posted over there, the forthcoming v0.12.0 release of Terraform will include a solution for this, with a new dynamic block construct which in your case would look something like this:

resource "google_compute_backend_service" "default" {
  name          = "tcplb-backend"
  description   = "TCP proxy load balancer backend config"
  port_name     = "dns"
  protocol      = "TCP"
  timeout_sec   = 30
  project       = "test-prohect-1649"

  dynamic "backend" {
    for_each = [
      google_compute_instance_group.us.self_link,
      google_compute_instance_group.eu.self_link
    ]
    content {
      group = backend.value
    }
  }

  backend {
    group = "${google_compute_region_instance_group_manager.us-grp.instance_group}"
  }

  health_checks = ["${google_compute_health_check.default.self_link}"]
}

I notice in your example you were also trying to pattern-match over all of the google_compute_instance_group resources in your configuration. That isn't possible and won't ever be possible because Terraform relies on the explicit references between resources to build its dependency graph. The closest thing would be to have a single google_compute_instance_group resource with count = 2 and then use the instances of that resource:

  dynamic "backend" {
    for_each = google_compute_instance_group.regional.*.self_link,
    content {
      group = backend.value
    }
  }

This will become more intuitive to express in a later release when for_each will also be supported at the resource level, as discussed in #17179, allowing resource instances to be identified by meaningful string keys rather than by numeric indices.

Since the dynamic block feature is already merged in master ready for the next release and we already have #17179 open for the for_each block, I'm going to close this out. Thanks for sharing this use case!

[universalvishwa]

Hi @apparentlymart,

Thank you very much. I'm looking forward to get my hands the new release soon.

[bukzor]

For any passers-by, I've come up with this workaround for the issue, below. It uses null_resource as a standin for the missing google_compute_backend resources, and "splat" syntax to attach those backends to the backend_service.

resource "null_resource" backends {
  count = "${length(split(" ", module.continent--us.ingress_instance_groups))}"

  triggers = {
    balancing_mode  = "RATE"
    capacity_scaler = "1"
    group           = "${element(split(" ", module.continent--us.ingress_instance_groups), count.index)}"

    max_rate_per_instance = "1"
  }
}

resource "google_compute_backend_service" "hoo" {
  name                            = "k8s-be"
  connection_draining_timeout_sec = "0"

  health_checks = ["${google_compute_health_check._.self_link}"]

  backend = ["${null_resource.backends.*.triggers}"]
}

[dz-pyps]

Thanks to @bukzor I've managed to achieve the expected result with terraform 0.11.13.

First, you need to define a null resource

resource "null_resource" backends {
  count = "${var.some_variable["instance_count"]}"

  triggers = {
    group = "${element(google_compute_instance_group.some_instancegroup_name.*.self_link, count.index)}"
  }
}

and then use backend

backend = ["${null_resource.backends.*.triggers}"]

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.