Terraform: Cannot access values in a list of maps

[megakoresh]

There is no way to access map values inside a list of maps when it is passed to a module as a variable. I have tried the following to no avail:

${lookup(element(var.list_of_maps, count.index), "key_name")}
${element(var.list_of_maps.*.key_name, count.index)}
${lookup(var.list_of_maps[count.index], "key_name")}

Lookups and element functions don't work because for whatever reason they don't support any value complexity (that is if nested values are anything but strings, they will crash).

Seemed like the splat operator might do the trick but apparently it can't be used on variables. This is critical functionality for making reusable modules, and it is quite a basic one, so I suspect this is either a bug or an oversight - at any rate there is nothing in the documentation that says splat operators can't be used on variables.

Terraform Version

Terraform v0.11.7
+ provider.openstack v1.6.0

Terraform Configuration Files

Code in module:

resource "openstack_networking_subnetpool_v2" "subnetpool_ipv6" {
  count      = "${length(var.subnets)}"
  name       = "${element(var.subnets.*.name, count.index)}_ipv6"
  ip_version = 6
  prefixes   = "${element(var.subnets.*.pool_ipv6_prefixes, count.index)}"
}

Calling module:

module "customized_network" {
  source = "../../network/flat"
  router_name           = "custom-router"
  network_name          = "custom-net"
  external_network_name = "ext-net"

  subnets = [
    {
      name               = "subnet1"
      pool_ipv6_prefixes = ["2001:998:33:16ff::/120"]
      pool_ipv4_prefixes = ["192.170.0.0/24"]
      cidr_ipv6          = "2001:998:33:16ff::/116"
      cidr_ipv4          = "193.170.0.0/24"
      enable_dhcp        = true
    },
    {
      name               = "subnet2"
      pool_ipv6_prefixes = ["2001:998:33:16ff::/96"]
      pool_ipv4_prefixes = ["192.170.1.0/24"]
      cidr_ipv6          = "2001:998:33:16ff::fff/96"
      cidr_ipv4          = "192.170.1.0/24"
      enable_dhcp        = true
    },
  ]
}

Debug Output

Crash Output

Error downloading modules: Error loading modules: module customized_network: Error loading .terraform/modules/4ae54b31c728d74827e62ddca10b3686/main.tf: Error reading config for openstack_networking_subnetpool_v2[subnetpool_ipv6]: Invalid dot index found: 'var.subnets.*.name'. Values in maps and lists can be referenced using square bracket indexing, like: 'var.mymap["key"]' or 'var.mylist[1]'. in:

${element(var.subnets.*.name, count.index)}_ipv6

Expected Behavior

The value in list of maps should have been used.

Actual Behavior

No attempted methods have been found to utilize values in a statically defined list of maps in any way.

Steps to Reproduce

1. Create a module that uses a list of maps variable, which extracts values from the map
2. Use the module

Additional Context

N/A

References

N/A

[kopinions]

are there any updates about this issue? I have same problem too, with same version.

[apparentlymart]

Hi @megakoresh! Sorry this didn't work as expected.

The special .*. syntax in v0.11 and earlier applies only to resource blocks with count set; it is not a general operator that can be applied to all lists, but rather a special naming convention for resource variables.

This will change in the next major version of Terraform. The relevant item in this list is _Generalized "Splat" expressions_, which means that .* will be treated as a language operator that can apply to any list value.

Unfortunately the subnets value you're setting here is also invalid in the current version of Terraform: Terraform v0.11 doesn't support maps containing mixed element types, so the subnets maps are required to be either all strings or all lists, but not a mixture of the two. That will also be addressed in the next major release by allowing you to pass _objects_ (similar to maps, but with individual attribute value types) between modules.

This will be declarable using the new _type expression_ syntax in the next major release:

# Not yet implemented, and details may change before release

variable "subnets" {
  type = object({
    name = string
    pool_ipv6_prefixes = list(string)
    pool_ipv4_prefixes = list(string)
    cidr_ipv6 = string
    cidr_ipv4 = string
    enable_dhcp = bool
  })
}

---

With the current version of Terraform, users normally design configurations like this by separating the network and the subnets into separate modules, giving a configuration something like this:

module "customized_network" {
  source = "../../network/flat"

  router_name           = "custom-router"
  network_name          = "custom-net"
  external_network_name = "ext-net"
}

module "subnet1" {
  source = "../../network/subnet"

  network_id         = "${module.customized_network.id}"
  name               = "subnet1"
  pool_ipv6_prefixes = ["2001:998:33:16ff::/120"]
  pool_ipv4_prefixes = ["192.170.0.0/24"]
  cidr_ipv6          = "2001:998:33:16ff::/116"
  cidr_ipv4          = "193.170.0.0/24"
  enable_dhcp        = true
}

module "subnet2" {
  source = "../../network/subnet"

  network_id         = "${module.customized_network.id}"
  name               = "subnet2"
  pool_ipv6_prefixes = ["2001:998:33:16ff::/96"]
  pool_ipv4_prefixes = ["192.170.1.0/24"]
  cidr_ipv6          = "2001:998:33:16ff::fff/96"
  cidr_ipv4          = "192.170.1.0/24"
  enable_dhcp        = true
}

[apparentlymart]

We posted a more detailed article on "generalized splat expressions" today, which elaborates a little on what I said above.

With that said, on re-read here I see that the general improvements to the index syntax (not covered in their own article) will actually be more beneficial to what you described here, since they will allow you to skip using the splat syntax altogether:

# Not yet implemented, and some details may change before release

resource "openstack_networking_subnetpool_v2" "subnetpool_ipv6" {
  count      = length(var.subnets)
  name       = "${var.subnets[count.index].name}_ipv6"
  ip_version = 6
  prefixes   = var.subnets[count.index].pool_ipv6_prefixes
}

With the "resource for_each" feature planned for a future release (not in scope for v0.12.0 itself, but planned for afterwards, as described in the _for and for each_ article) this'll eventually be even nicer:

# Not yet implemented, and some details may change before release

resource "openstack_networking_subnetpool_v2" "subnetpool_ipv6" {
  for_each = {for s in var.subnets: s.name => s}

  name       = "${each.key}_ipv6"
  ip_version = 6
  prefixes   = each.value.pool_ipv6_prefixes
}

This will avoid the need to use count.index altogether, and will also allow Terraform to track the various subnet instances by their names rather than by their indexes in the list, which will avoid unnecessary changes if the ordering of items in the lists is changed later.

[apparentlymart]

Hi again @megakoresh!

The changes I mentioned earlier are now merged into master for inclusion in the forthcoming v0.12.0 release. I verified this in the v0.12.0-alpha2 prerelease build by modifying slightly your child module configuration to make it local-only for simplicity's sake:

variable "subnets" {
  type = list(object({
    name               = string
    pool_ipv6_prefixes = list(string)
    pool_ipv4_prefixes = list(string)
    cidr_ipv6          = string
    cidr_ipv4          = string
    enable_dhcp        = bool
  }))
}

resource "null_resource" "subnetpool_ipv6" {
  count = "${length(var.subnets)}"
  triggers = {
    name       = "${element(var.subnets.*.name, count.index)}_ipv6"
    ip_version = 6
    prefixes   = "${jsonencode(element(var.subnets.*.pool_ipv6_prefixes, count.index))}"
  }
}

You can see here that I used the same expressions you did, except that I had to jsonencode the prefixes just because null_resource expects all of the trigger values to be strings. I also defined a specific type for the variable as I showed in my earlier comment, though I realized in trying this that I should've defined a list of object instead of just an object, so I adjusted that.

In v0.12.0-alpha2 this works as-is:

$ terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.customized_network.null_resource.subnetpool_ipv6[0] will be created
  + resource "null_resource" "subnetpool_ipv6" {
      + id       = (known after apply)
      + triggers = {
          + "ip_version" = "6"
          + "name"       = "subnet1_ipv6"
          + "prefixes"   = jsonencode(
                [
                  + "2001:998:33:16ff::/120",
                ]
            )
        }
    }

  # module.customized_network.null_resource.subnetpool_ipv6[1] will be created
  + resource "null_resource" "subnetpool_ipv6" {
      + id       = (known after apply)
      + triggers = {
          + "ip_version" = "6"
          + "name"       = "subnet2_ipv6"
          + "prefixes"   = jsonencode(
                [
                  + "2001:998:33:16ff::/96",
                ]
            )
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

I also tried the "cleaner" version I suggested in my later comment:

variable "subnets" {
  type = list(object({
    name               = string
    pool_ipv6_prefixes = list(string)
    pool_ipv4_prefixes = list(string)
    cidr_ipv6          = string
    cidr_ipv4          = string
    enable_dhcp        = bool
  }))
}

resource "null_resource" "subnetpool_ipv6" {
  count = length(var.subnets)
  triggers = {
    name       = "${var.subnets[count.index].name}_ipv6"
    ip_version = 6
    prefixes   = jsonencode(var.subnets[count.index].pool_ipv6_prefixes)
  }
}

This gave exactly the same result, as expected.

The version with for_each I can't try yet because that feature is not in scope for the v0.12.0 final release; that'll come in a later release, tracked in #17179.

Since this is working in master, I'm going to close this out now. Thanks for reporting this, and thanks for your patience while we laid the groundwork to get this working better.

[zx1986]

Should I jump into 0.12 or keep writing 0.11 right now on 2019/04?
I am a newbie in Terraform.

I am using Terraform v0.11.13 in macOS now.

[zx1986]

Sorry, I confirm those code works in Terraform v0.11.13 in macOS:

variable "vms" {
  type        = "list"
  description = "虛擬主機的屬性列表，每個 {} 內定義各台主機的屬性，是一個 object 組成的 list"

  default = [
    {
      template_name = ""
      datastore     = ""
      disk_size     = ""
      num_cpus      = ""
      memory        = ""
      domain        = ""
      network       = ""
      gateway       = ""
      netmask       = ""
      ip            = ""
      name          = ""
      folder_path   = ""
      folder_type   = ""
    },
  ]
}

### Virtual Machines Folder

resource "vsphere_folder" "folder" {
  count         = "${length(var.vms)}"
  path          = "${lookup(var.vms[count.index], "folder_path")}"
  type          = "${lookup(var.vms[count.index], "folder_type")}"
  datacenter_id = "${data.vsphere_datacenter.datacenter.id}"
}

---

updated:

folder should be folders.
datastore may be datastores, in case you have multiple datastores.
And then do something like:

data "vsphere_datastore" "datastores" {
  count         = "${length(var.machines)}"
  name          = "${lookup(var.machines[count.index], "datastore")}"
  datacenter_id = "${data.vsphere_datacenter.datacenter.id}"
}

datastore_id     = "${element(data.vsphere_datastore.datastores.*.id, count.index)}"

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.