Terraform: Conditionally Omitted Blocks

[prehor]

Current Terraform Version

Terraform v0.12.0-alpha4 (2c36829d3265661d8edbd5014de8090ea7e2a076)

Use-cases

Terraform 0.12 introduced Conditionally Omitted Arguments. For modules, it would be nice to have Conditionally Omitted Blocks, so we can create universal modules that can use all resource parameters including optional block parameters and the user can specify only some of them.

Proposal

An example of an Azure Storage Account module that manages a storage account with globally unique name and can have optional custom_domain block:

# azurerm_storage_account_module/main.tf
variable "resource_group_name" {
  type = string
}

variable "location" {
  type = string
}

variable "custom_domain" {
  default = null

  # Define the block structure (or we can use object() instead of block())
  type = block({
    name = string
    use_subdomain = bool
  })
}

resource "random_uuid" "name" {
  keepers = {
    resource_group_name = var.resource_group_name
  }
}

resource "azurerm_storage_account" "sa" {
  name = random_uuid.name.result
  location = var.location
  resource_group_name = var.resource_group_name

  # Use a block from the variable or omit the block if the variable is null
  custom_domain = var.custom_domain
}

output "name" {
  value = azurerm_storage_account.sa.name
}

Use of Conditionally Omitted Block:

# main.tf
variable "resource_group_name" {}
variable "location" {}

resource "azurerm_resource_group" "rg" {
  name = var.resource_group_name
  location = var.location
}

# Manage a storage account with random name
module "sa1" {
  source = "./azurerm_storage_account_module"
  resource_group_name = azurerm_resource_group.rg.name
  location = azurerm_resource_group.rg.location
}

# Manage another storage account with random name and custom domain
module "sa2" {
  source = "./azurerm_storage_account_module"
  resource_group_name = azurerm_resource_group.rg.name
  location = azurerm_resource_group.rg.location

  # Specify an optional block (or we can use '=' for assignment)
  custom_domain {
    name = "example.com"
    use_subdomain = true
  }
}

[prehor]

Current Terraform Version

Terraform v0.12.0-alpha4 (2c36829d3265661d8edbd5014de8090ea7e2a076)

Additional Context

OK. I found this article about Dynamic Nested Blocks and tried to use it for Azure Storage Account custom_domains:

# azurerm_storage_account_module/main.tf
variable "resource_group_name" {
  type = string
}

variable "location" {
  type = string
}

variable "account_replication_type" {
  default = "LRS"
}

variable "account_tier" {
  default = "standard"
}

variable "custom_domain" {
  default = null

  type = object({
    name = string
    use_subdomain = bool
  })
}

resource "random_string" "name" {
  length  = 24
  upper   = false
  lower   = true
  number  = true
  special = false

  keepers = {
    resource_group_name = var.resource_group_name
  }
}

resource "azurerm_storage_account" "sa" {
  name = random_string.name.result
  location = var.location
  resource_group_name = var.resource_group_name
  account_replication_type = var.account_replication_type
  account_tier = var.account_tier

  dynamic "custom_domain" {
    for_each = var.custom_domain == null ? [] : list(var.custom_domain)

    content {
      name = custom_domain.name
      use_subdomain = custom_domain.use_subdomain
    }
  }
}

output "name" {
  value = azurerm_storage_account.sa.name
}

Use the module:

# main.tf
variable "resource_group_name" {}
variable "location" {}

resource "azurerm_resource_group" "rg" {
  name = var.resource_group_name
  location = var.location
}

# Manage a storage account with random name
module "sa1" {
  source = "./azurerm_storage_account_module"
  resource_group_name = azurerm_resource_group.rg.name
  location = azurerm_resource_group.rg.location
  account_replication_type = "LRS"
  account_tier = "standard"
}

# Manage another storage account with random name and custom domain
module "sa2" {
  source = "./azurerm_storage_account_module"
  resource_group_name = azurerm_resource_group.rg.name
  location = azurerm_resource_group.rg.location
  account_replication_type = "LRS"
  account_tier = "standard"

  # Specify an optional block
  custom_domain = {
    name = "example.com"
    use_subdomain = false
  }
}

It ends with an error:

$ /usr/local/Caskroom/terraform-0.12.0-alpha4/0.12.0-alpha4/terraform apply

Error: Unsupported attribute

  on azurerm_storage_account_module/main.tf line 50, in resource "azurerm_storage_account" "sa":
  50:       name = custom_domain.name

This object does not have an attribute named "name".

Error: Unsupported attribute

  on azurerm_storage_account_module/main.tf line 51, in resource "azurerm_storage_account" "sa":
  51:       use_subdomain = custom_domain.use_subdomain

This object does not have an attribute named "use_subdomain".

Both errors have an underlined attribute name after custom_domain.

Related

I tried the example from Dynamic Nested Blocks:

resource "random_uuid" "resource_group_name" {
  keepers = {
    keep = "always"
  }
}

resource "azurerm_resource_group" "rg" {
  name = random_uuid.resource_group_name.result
  location = "eastus"
}

variable "subnets" {
  default = [
    {
      name   = "a"
      number = 1
    },
    # {
    #   name   = "b"
    #   number = 2
    # },
    # {
    #   name   = "c"
    #   number = 3
    # },
  ]
}

locals {
  base_cidr_block = "10.0.0.0/16"
}

resource "azurerm_virtual_network" "example" {
  name                = "example-network"
  resource_group_name = azurerm_resource_group.rg.name
  address_space       = [local.base_cidr_block]
  location            = azurerm_resource_group.rg.location

  dynamic "subnet" {
    for_each = [for s in var.subnets: {
      name   = s.name
      prefix = cidrsubnet(local.base_cidr_block, 4, s.number)
    }]

    content {
      name           = subnet.name
      address_prefix = subnet.prefix
    }
  }
}

and it ends with the same errors:

$ /usr/local/Caskroom/terraform-0.12.0-alpha4/0.12.0-alpha4/terraform apply

Error: Unsupported attribute

  on /Users/reho/OneDrive/GitHub/sicz/terraform-azure/storage/xxx/main.tf line 46, in resource "azurerm_virtual_network" "example":
  46:       name           = subnet.name

This object does not have an attribute named "name".


Error: Unsupported attribute

  on /Users/reho/OneDrive/GitHub/sicz/terraform-azure/storage/xxx/main.tf line 47, in resource "azurerm_virtual_network" "example":
  47:       address_prefix = subnet.prefix

This object does not have an attribute named "prefix".

It looks like the Dynamic Nested Blocks in Terraform v0.12.0-alpha4 are broken.

[apparentlymart]

Hi @prehor,

It looks like the examples in the article are outdated or incorrect. Sorry about that! The problem is that the iterator object has attributes key and value, and so on your example you should write subnet.value.name to access the name attribute of each object in the for_each collection.

There is a bug with dynamic blocks nested inside one another in alpha4 which has since been fixed, but the simple case of a single block should be working.

Sorry for the incorrect article content! In the mean time we have a draft of the updated docs for this feature which include more details than the article did.

[prehor]

Hi @apparentlymart ,

thanks for your help, Dynamic Blocks are already working for me:

resource "azurerm_storage_account" "sa" {
  name = random_string.name.result
  location = var.location
  resource_group_name = var.resource_group_name
  account_replication_type = var.account_replication_type
  account_tier = var.account_tier

  dynamic "custom_domain" {
    for_each = var.custom_domain == null ? [] : list(var.custom_domain)

    content {
      name          = custom_domain.value.name
      use_subdomain = custom_domain.value.use_subdomain
    }
  }
}

Please use more examples in the updated docs for this feature because it's not too clear for me:

• Define the variable that is used in for_each to clarify its structure
• More examples for iterating list(string), list(map), map(string) and map(map)
• An example from Terraform 0.12 Preview with for inside for_each
• An examples which use labels and iterator arguments

[sepulworld]

Im running into a similar issue where I believe the docs (and blog post) are leading me in the wrong direction.

https://github.com/terraform-providers/terraform-provider-aws/issues/8260

variable "global_secondary_indexes" {
  default = [
    {
      name   = "index1"
      hash_key = "foo1"
      range_key = "bar1"
      projection_type = "ALL"
      key_type = "S"
    },
    {
      name   = "index2"
      hash_key = "foo2"
      range_key = "bar2"
      projection_type = ""
      key_type = "S"
    }
  ]
}

resource "aws_dynamodb_table" "dynamodb_table" {
  name         = "zane-test-inno"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "rage"


  dynamic "global_secondary_index" {
    for_each = [for g in var.global_secondary_indexes: {
      name            = g.name
      hash_key        = g.hash_key
      range_key       = g.range_key
      projection_type = g.projection_type
    }]

    content {
      name            = global_secondary_index.name
      hash_key        = global_secondary_index.hash_key
      range_key       = global_secondary_index.range_key
      projection_type = global_secondary_index.projection_type
    }
  }

  dynamic "attribute" {
    for_each = [for g in var.global_secondary_indexes: {
      name = g.name
      type = g.key_type
    }]

    content {
      name = global_secondary_index.name
      type = global_secondary_index.type
    }
  }

  attribute {
    name = "rage"
    type = "S"
  }
}

generate errors like the following saying there isn't an attribute for that dynamic block content...

Error: Unsupported attribute

  on test.tf line 58, in resource "aws_dynamodb_table" "dynamodb_table":
  58:       range_key       = global_secondary_index.range_key

This object does not have an attribute named "range_key".

I'm assuming things have changed and / or Im trying to do something that isn't supported

[sepulworld]

I think I may have found the issue

    content {
      name            = global_secondary_index.name
      hash_key        = global_secondary_index.hash_key
      range_key       = global_secondary_index.range_key
      projection_type = global_secondary_index.projection_type
    }

Change to ...

    content {
      name            = global_secondary_index.value.name
      hash_key        = global_secondary_index.value.hash_key
      range_key       = global_secondary_index.value.range_key
      projection_type = global_secondary_index.value.projection_type
    }

[petrokashlikov]

I agree that having more clear documentation will be super useful. I'm looking to use template gcs_storage_bucket resource with may not have lifecycle_rule block, have 1 or have few of them.
If I use syntax like @sepulworld suggested, everything works as expected, terraform iterates through variable and create necessary count of blocks, except it can't not to create it at all if I don't pass variable or set it to null.
Getting error "A null value cannot be used as the collection in a 'for' expression."

variable "rules" {
  default = [
    {
      type   = "SetStorageClass"
      storage_class = "NEARLINE"
      age                   = 60
      created_before        = "2017-06-13"
      is_live               = false
      matches_storage_class = ["REGIONAL"]
      num_newer_versions    = 10
    },
    {
      type   = "SetStorageClass"
      storage_class = "COLDLINE"
      age                   = 50
      created_before        = "2017-06-13"
      is_live               = false
      matches_storage_class = ["NEARLINE"]
      num_newer_versions    = 10
    }
  ]
}
resource "google_storage_bucket" "default" {
  count         = "${var.gcs_enabled == "true" ? 1 : 0}"
  name          = "${var.gcs_storage_bucket_name}"
  location      = "${var.gcs_region}"
  project       = "${var.project}"
  storage_class = "${var.gcs_storage_class}"
  force_destroy = "${var.gcs_force_destroy}"

  dynamic "lifecycle_rule" {
    for_each = [for g in var.rules: {
      type = g.type
      storage_class = g.storage_class
      age = g.age
      created_before = g.created_before
      is_live = g.is_live
      matches_storage_class = g.matches_storage_class
      num_newer_versions = g.num_newer_versions

    }]
    content {
       action {
         type          = lifecycle_rule.value.type
         storage_class = lifecycle_rule.value.storage_class
       }
      condition {
        age                   = lifecycle_rule.value.age
        created_before        = lifecycle_rule.value.created_before
        is_live               = lifecycle_rule.value.is_live
        matches_storage_class = lifecycle_rule.value.matches_storage_class
        num_newer_versions    = lifecycle_rule.value.num_newer_versions
      }
    }
  }

  versioning {
    enabled = "${var.gcs_versioning_enabled}"
  }
}

If I use syntax suggested by @prehor , then it can omit block or create 1, but not clear how in that case iterate to create more then 1

[balan2010]

Hey Guys, How do I conditionally omit a block based on a value in type list(objects)

Resource Block

resource "google_compute_subnetwork" "network_ip_ranges" {
  count = length(var.subnetworks)

  name                     = var.subnetworks[count.index].subnet_name
  ip_cidr_range            = var.subnetworks[count.index].cidr_range
  region                   = var.subnetworks[count.index].region
  enable_flow_logs         = var.subnetworks[count.index].enable_flow_logs
  private_ip_google_access = var.subnetworks[count.index].private_ip_google_access
  network                  = var.network_self_link
  project                  = var.project_id

  dynamic "secondary_ip_range" {
    for_each = length(var.subnetworks[count.index].secondary_ip_ranges) == 0 ? [] : [ for n in var.subnetworks[count.index].secondary_ip_ranges: {
                range_name = n.range_name
                ip_cidr_range = n.cidr_range
    }]

    content {
      range_name = secondary_ip_range.value.range_name
      ip_cidr_range = secondary_ip_range.value.ip_cidr_range
    }
  }
} 

Variable Type

variable "subnetworks" {
    type = list(object({
        subnet_name              = string
        cidr_range               = string
        region                   = string
        enable_flow_logs         = string
        private_ip_google_access = string
        secondary_ip_ranges      = list(object({
            range_name = string
            cidr_range = string
        }))
    }))
}

Example Values:

subnetworks = [
        {
          subnet_name = "test1-sbn"
          cidr_range = "10.56.0.0/20"
          region = "australia-southeast1"
          enable_flow_logs = "false"
          private_ip_google_access = "false"
          secondary_ip_ranges = [{
                range_name = "services-secondary-range"
                cidr_range = "10.54.254.0/24"
          },
          {
                range_name = "cluster-secondary-range"
                cidr_range = "10.55.252.0/22"
          },          
          ]
        },
        {
          subnet_name = "test3-sbn"
          cidr_range = "10.96.0.0/20"
          region = "australia-southeast1"
          enable_flow_logs = "false"
          private_ip_google_access = "false"
          secondary_ip_ranges = []
        }       
  ]

Expected Result

Secondary Ip ranges dynamic block should be omitted when secondary_ip_range is set to []

Actual Result

Error: Unsupported block type

  on .terraform/modules/subnets/gcp/subnets/main.tf line 16, in resource "google_compute_subnetwork" "network_ip_ranges":
  16:   dynamic "secondary_ip_range" {

Blocks of type "secondary_ip_range" are not expected here.

Is there anyway I could achieve expected behavior without creating another resource block excluding nested block ?

[balan2010]

The issue was fixed in https://github.com/hashicorp/terraform/pull/21549/files and it worked after upgrading terraform to v0.12.2

[dimisjim]

To omit a block one basically needs to add a conditional expression in the for_each expression, like:

Using a local var (useful when local var contains a conditional expression with interpolations and vars): for_each = var.cf_has_oai ? local.cf_origin_config : {}

Using a var (when using a static map of key value pairs):

for_each = var.cf_has_cer ? [for s in var.cf_cer: {
    error_caching_min_ttl = s.error_caching_min_ttl
    error_code            = s.error_code
    response_code         = s.response_code
    response_page_path    = s.response_page_path
}] : []

The above are not clearly mentioned in the docs. @apparentlymart

[anilnatha]

@prehor Just wanted to personally thank you for this post. I'm new to Terraform and have been attempting to create a module for S3. The only road block I have met was being able to optionally provide the logging block. The information you posted here helped me understand and implement how to do that successfully. So, thank you!

[dpkirchner]

A slightly more "compact" (heh) way to omit a block may be to use compact([...]), which strips out empty strings (and nulls). A practical example of this in use:

variable "start_time" {
  type = "string"
  default = null
  description = "Allowed start time for automatic cluster maintenance -- see https://cloud.google.com/kubernetes-engine/docs/how-to/maintenance-window"
}

resource "google_container_cluster" "default" {
  name       = "abc"
  location   = "us-central1-a"
  network    = "default"
  subnetwork = "default"

  dynamic "maintenance_policy" {
    for_each = compact([var.start_time])
    content {
      daily_maintenance_window {
        start_time = var.start_time
      }
    }
  }
}

In this example the maintenance_window block is not included at all. If you want that block, you'll need to find another method (I don't know of one, but I haven't needed that yet).

Terraform v0.12.7
+ provider.google v2.13.0
+ provider.google-beta v2.13.0

[jmgreg31]

Wanted to provide another use case in this thread as the ideas here helped me to come to the solution I needed. I have been working a lot with modularizing Cloudfront and needed a way to conditionally omit attributes AND blocks from a nested block. Here is a solution that worked for me where I could omit the custom_header variable and still build a custom_origin_config

dynamic "origin" {
    for_each = [for i in var.dynamic_custom_origin_config : {
      name                     = i.domain_name
      id                       = i.origin_id
      path                     = i.origin_path
      http_port                = i.http_port
      https_port               = i.https_port
      origin_keepalive_timeout = i.origin_keepalive_timeout
      origin_read_timeout      = i.origin_read_timeout
      origin_protocol_policy   = i.origin_protocol_policy
      origin_ssl_protocols     = i.origin_ssl_protocols
      custom_header            = lookup(i, "custom_header", null)
    }]
    content {
      domain_name   = origin.value.name
      origin_id     = origin.value.id
      origin_path   = origin.value.path
      dynamic "custom_header" {
        for_each = origin.value.custom_header == null ? [] : [ for i in origin.value.custom_header : {
          name = i.name
          value = i.value
        }]
        content {
          name  = custom_header.value.name
          value = custom_header.value.value
        } 
      }
      custom_origin_config {
        http_port                = origin.value.http_port
        https_port               = origin.value.https_port
        origin_keepalive_timeout = origin.value.origin_keepalive_timeout
        origin_read_timeout      = origin.value.origin_read_timeout
        origin_protocol_policy   = origin.value.origin_protocol_policy
        origin_ssl_protocols     = origin.value.origin_ssl_protocols
      }
    }
  }

[zx1986]

Could dynamic block just has a simple boolean trigger?

---

My variable like:

variable "machines" {
  type        = map
  default = {
    vm_name = {
      vm_size   = "Standard_B1S"
      subnet    = ""
      ip        = ""
      disk_size = 1 # if size == 0, then `dynamic block` does nothing.
    }
  }
}

Could a map has list inside an element?
For example, disk_size be a list, then dynamic block could read the list, if empty or not.

[apparentlymart]

I'm not sure exactly how to interpret the use-case in the previous comment, but here's an answer for two different interpretations in the hope that one of them is useful:

You can adapt a "count" value into a collection suitable for dynamic block for_each using the range function:

  for_each = range(var.machines.disk_size)

In the above, range returns a list of integers from zero up to (but not including) the value given for disk_size, so the iterator key and value in the content block will both be a numeric index. If the count given to range is zero then its result is an empty list, and so the dynamic block would "expand" to no blocks at all.

If the idea was instead to use a boolean test _derived from_ the number, then a conditional expression can achieve that. For example:

  for_each = var.machines.disk_size > 0 ? [var.machines.disk_size] : []

In the above case, each.value inside the content block will be the disk size, but the block will expand to no blocks at all in the special case where the disk size is zero. Because of how the above is written, the result will be a list with either zero or one elements.

[zx1986]

@apparentlymart great tip! thank you!

I apply your idea on this: https://github.com/LarvataTW/terraform-modules/commit/556ea60a52b8f1bf0c074f52411b6f0e9634bcdf

[mildwonkey]

The documentation has been updated, so I'm going to close this issue - thank you!

[dimisjim]

@mildwonkey Could you link to the doc you refer to?
Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.