Terraform: for_each attribute for creating multiple resources based on a map

[mirogta]

Hi,

We are missing a better support for loops which would be based on keys, not on indexes.

Below is an example of the problem we currently have and would like Terraform to address:

• We have a list of Azure NSG (Network Security Group) rules defined in a hash. E.g.

locals {
  rules = {
    rdp_from_onprem = {
      priority         = 100
      protocol         = "TCP"
      destination_port = "3389"
      source_address   = "10.0.0.0/8"
    }

    winrm_from_onprem = {
      priority         = 110
      destination_port = "5985-5986"
      source_address   = "10.0.0.0/8"
    }

    dynatrace_security_gateway = {
      priority         = 120
      destination_port = "9999"
    }
  }
}

• This allows us to keep the Terraform resource definition DRY and use a loop to create all the rules:

resource "azurerm_network_security_rule" "allow-in" {
  count                       = "${length(keys(local.rules))}"
  name                        = "allow-${element(keys(local.rules), count.index)}-in"
  direction                   = "Inbound"
  access                      = "Allow"
  priority                    = "${lookup(local.rules[element(keys(local.rules), count.index)], "priority")}"
  protocol                    = "${lookup(local.rules[element(keys(local.rules), count.index)], "protocol", "*")}"
  source_port_range           = "*"
  destination_port_range      = "${lookup(local.rules[element(keys(local.rules), count.index)], "destination_port", "*")}"
  source_address_prefix       = "${lookup(local.rules[element(keys(local.rules), count.index)], "source_address", "*")}"
  destination_address_prefix  = "${lookup(local.rules[element(keys(local.rules), count.index)], "destination_address", "*")}"
  resource_group_name         = "${azurerm_resource_group.resource_group.name}"
  network_security_group_name = "${azurerm_network_security_group.nsg.name}"
}

• So far, so good. However since the resources and their state are uniquely identified by the index and not by their name, we can't simply change the rules later.
  
  
  
  • We can add new rules only at the end of the hash.
  
  
  • We can remove rules only at the end of the hash.
  
  
  • We can modify the rules, as long as their position in the hash doesn't change.
  
  
  • But we can never remove any other rule or change their position in the hash. This seems to be very restrictive and basically means we had to stop using this approach and define all individual rules as individual _azurerm_network_security_rule_ resources.
  
  

As you can guess, if we e.g. remove the first item from the hash, Terraform would not see that as a removal of the first resource (index 0), but rather removal of the last resource (index 2) and a related unexpected change of all the other resources (old index 1 becomes new index 0, old index 2 becomes new index 1).

Unfortunately this can also cause Azure provider to fail, because it may get into a conflict where an actual resource (old index 1) still exists in Azure, but Terraform now tries to modify the actual resource (old index 0) to have the same properties, but that is not possible (e.g. NSG _priority_ and _port_ have to be unique).

I've shown an example with 3 rules, but in reality we can have 50 rules and the tf file is 5x longer and more difficult to manage with individual resources compared to using a hash.

We would like to use hashes in Terraform in such a way that a position of an element inside a hash doesn't matter. That's why many other languages provide two ways of looping - by index (e.g. for i=0; i<list.length;i++) and by key (foreach key in list).

I'm sure that smart guys like you can figure out how to make this work in Terraform.

Thanks

[apparentlymart]

Hi @mirogta! Thanks for this feature request, and your detailed use-case.

This is definitely a request that has come up before, though it seems like it's only previously been discussed within the comments of other issues, so this issue seems like a good anchor for talking about our plans here, and updating as we make progress.

The current design sketch we have is a new for_each argument that can be used as an alternative to count, taking either a list or a map as its value:

# NOT YET IMPLEMENTED; some details may change before implementation

resource "azurerm_network_security_rule" "allow-in" {
  for_each                    = "${local.rules}"
  name                        = "allow-${each.key}-in"
  direction                   = "Inbound"
  access                      = "Allow"
  priority                    = "${each.value.priority}"
  protocol                    = "${lookup(each.value, "protocol", "*")}"
  source_port_range           = "*"
  destination_port_range      = "${lookup(each.value, "destination_port", "*")}"
  source_address_prefix       = "${lookup(each.value, "source_address", "*")}"
  destination_address_prefix  = "${lookup(each.value, "destination_address", "*")}"
  resource_group_name         = "${azurerm_resource_group.resource_group.name}"
  network_security_group_name = "${azurerm_network_security_group.nsg.name}"
}

The primary benefit of this, as you correctly suggested, is that if the for_each collection is a _map_ then we will use the map keys to correlate configuration instances with state instances when planning updates, and thus avoid the problem you've encountered with adding or removing items in the map.

If a user provides a _list_ to for_each then it'll behave in the same way as count -- correlating by index -- but will still provide the more convenient each.key and each.value accessors to interpolate from the collection elements, reducing the visual noise of all the element(..., count.index) expressions that result when multiplying a resource using count over the length of a list.

We are currently focused on some more general work to improve the configuration language's handling of collection types, which is a pre-requisite for this for_each feature. After that, we'll start designing and prototyping this feature in more detail.

I'm going to update the summary of this issue so that it's more specific about our currently-planned approach, since that should help us find it again to post updates when we have them.

Thanks again for this feature request!

[slawekm]

@apparentlymart: For the sake of readability - could this be implemented as a block directive instead?

Per following example:

target_groups = {
  "http" = {
      port = 80
      description = "HTTP port"
   }
  "https" = {
      port = 443
      description = "HTTPS port"
  }
}

resource "aws_lb_target_group" "example" {
  name = "example-${it.key}"
  port = "${it.values["port"]}"
  ...

  iterator {
    on "${var.target_groups}"
  }
}

or with list as input:

repositories = [ "repoA", "repoB"]

resource "aws_ecr_repository" "myrepos" {
  name = "${it.value}" 

  iterator {
    on "${var.repositories}""
  }
}

Where ${it.key} could be list index in this case. Ideally, current position in loop should be exposed to user via${it.index} too.

[apparentlymart]

Hi @slawekm,

Unfortunately HCL syntax doesn't work quite like that, so your nested on argument would need to include an equals sign:

  iterator {
    on = "${var.repositories}""
  }

Given that we expect for_each to become the main case, and count be more of an edge-case, we chose a count-like terse syntax here so that this usage would not create too much "visual noise" in configurations.

In practice today lots of users have the pattern of specifying the count attribute first and separating it from the others by a blank line so it stands out more from the other "normal" attributes, and so I'd expected that in practice people would use a similar pattern with for_each (even though I didn't illustrate that in my example above due to adapting the example in the original issue comment):

# NOT YET IMPLEMENTED; some details may change before implementation

resource "azurerm_network_security_rule" "allow-in" {
  for_each = local.rules

  name                        = "allow-${each.key}-in"
  direction                   = "Inbound"
  access                      = "Allow"
  priority                    = each.value.priority
  protocol                    = lookup(each.value, "protocol", "*")
  source_port_range           = "*"
  destination_port_range      = lookup(each.value, "destination_port", "*")
  source_address_prefix       = lookup(each.value, "source_address", "*")
  destination_address_prefix  = lookup(each.value, "destination_address", "*")
  resource_group_name         = azurerm_resource_group.resource_group.name
  network_security_group_name = azurerm_network_security_group.nsg.name
}

The above also illustrates a capability of the new configuration parser where it's no longer required to use "${ and }" to delimit standalone expressions, since expressions can now be specified directly.

Our current configuration work will also include an overhaul of the configuration-related documentation on the website that should include more "opinionated" best-practices than are currently given (since most of the documentation was written before best-practices emerged) and so we can explicitly recommend the above usage and make sure all of our examples follow it.

[slawekm]

Hi @apparentlymart

First of all, thanks for a detailed response.

on argument would need to include an equals sign

A product of c&p typo, my bad.

Given that we expect for_each to become the main case, and count be more of an edge-case, we chose a count-like terse syntax here so that this usage would not create too much "visual noise" in configurations.

Right.

Erm, I guess the main reason I've asked for this is that I'm subconsciously looking for a simpler way to define dynamic resources.

A configurable iterator which could also perform basic operations on input data, such like grouping or filtering on values and for_each looked like a good opportunity to create a foundation block for this.

This would make "feeding" dictionaries into resource blocks much easier.

And with that in mind, turning for_each into block probably would make more sense to you. Sorry for not being clear enough.

[apparentlymart]

Hi @slawekm,

Thanks for the additional information about your use-case.

The new configuration language interpreter has a feature called "for expressions" that I think will meet your use-case here. For example:

  # Not yet implemented and may change before release
  for_each = {
    for x in aws_subnet.main:
    x.id => x # Use subnet id as each.key
    if x.tags["Access"] == "public"
  }

This for construct can be used anywhere a map or list is expected, and can iterate over lists and maps.

[codyja]

Whoa, I'd love to see this. Very exciting

[joshuabaird]

Definitely exciting. Has any progress been made on the new config language interpreter that includes for_each?

[lorengordon]

Really hope this happens soon. To get around this type of problem, I've resorted to templating the .tf files using jinja to create individual resources rather than use count. Here's the general idea, https://github.com/Crapworks/terratools/tree/master/terratemplate

[fcoelho]

Will the for_each attribute be available for use in modules too?

[jay-stillman]

Is there any indication of when these will be rolled out?

[virtualbubble]

I had the same requirements for the NSG's and used this solution which lets me change ports as the rules are grouped and delete all previous rules with an update. This is just an example.

nsg_rules.tf.json (seperate global variables file for nsg rules.)

{
  "output": {
    "22-80-8080-443_all": {
      "description": "inbound_allow_SRC:*_Dest:*_Ports:22,80,8080,443",
    "value":"22_inbound_allow_tcp_*_*,80_inbound_allow_tcp_*_*,8080_inbound_allow_tcp_*_*,443_inbound_allow_tcp_*_*"
    },
    "443_all": {
      "description": "inbound_allow_SRC:*_Dest:*_Ports:443",
      "value":"443_inbound_allow_tcp_*_*"
    },
    "3389-22_wirelesetwork": {
      "description": "inbound_allow_SRC:63.200.10.5_Dest:*_Ports:3389,22",
      "value":"3389_inbound_allow_tcp_63.200.10.5_*,22_inbound_allow_tcp_63.200.10.5_*"
    },
    "3389-22_wirelesetwork_jumpbox1": {
      "description": "inbound_allow_SRC:63.200.10.5_Dest:10.1.0.1_Ports:3389,22",
      "value":"3389_inbound_allow_tcp_63.200.10.5_10.1.0.1,22_inbound_allow_tcp_63.200.10.5_10.1.0.1"
    }
  }
}

main.tf

module "nsg_rules" {
  source = "./core/nsg_rules"  //source to global nsg rules
}

module "DomainController1" {
  source               = "./core/.compute"
  location             = "${var.location}"
  resource_group_name  = "${var.resource_group_name}"
  vm_hostname          = "${var.vm_hostname}"
  custimagerecgroup    = "${var.custom_imagerecgroup}"
  customimage          = "${module.variables.2016-datacenter-latest}"
  vm_size              = "${var.vm_size["medium"]}"
  vnet_subnet_id       = "${data.azurerm_subnet.networks.id}"
  nsg_ports            = "${module.variables.3389-22_wirelessnetwork}" //REFERENCE THE RULE FROM THE JSON
}

core module code

resource "azurerm_network_security_group" "vm" {
  count               = "${var.nsg_required == "true" ? 1 : 0}"
  name                = "${var.vm_hostname}-${coalesce(var.remote_port,module.os.calculated_remote_port)}-nsg"
  location            = "${azurerm_resource_group.vm.location}"
  resource_group_name = "${azurerm_resource_group.vm.name}"
}

We then split the rules by manipulating the string values form the json.

Split by comma for count of rules

"22_inbound_allow_tcp__
80_inbound_allow_tcp__
8080_inbound_allow_tcp__
443_inbound_allow_tcp__

Then split using the underscores in order for port, direction, action, protocol, source and destination
note in the code below we have a * to accept from any.

resource "azurerm_network_security_rule" "vmnsg" {
  count                       = "${length(split(",", var.nsg_ports))}"
  name                        = "${element(split(",", var.nsg_ports), count.index)}"
  priority                    = "10${count.index}"
  direction                   = "${element(split(",", var.nsg_direction), count.index)}"
  access                      = "${element(split(",", var.nsg_access), count.index)}"
  protocol                    = "${element(split(",", var.nsg_protocol), count.index)}"
  destination_port_range      = "${element(split("_", element(split(",", var.nsg_ports), count.index)),0)}"
  source_port_range           = "*"
  source_address_prefix       = "${element(split("_", element(split(",", var.nsg_ports), 0)),4)}"
  destination_address_prefix  = "${element(split(",", var.nsg_dest_add), count.index)}"
  resource_group_name         = "${azurerm_resource_group.vm.name}"
  network_security_group_name = "${azurerm_network_security_group.vm.name}"
}

[stipx]

I also think the count feature is not really the best way to achieve creation of multiple resources.
The name will always be resource[0], resource[1] etc. But it would be much better to have resource[key1], resource[keySomething]. I just stumbled upon this when I was creating multiple VPN tunnels for AWS. I tried it with maps and of course the keys get sorted alphabetically.

Is there any progress with the for_each?

[jaloren]

how would this design handle sub resources? would there be a way to create a resource conditionally? For example, would i be able to say given condition X in one iteration, skip creating this resource.

[RishikeshDarandale]

@apparentlymart , will there be migration guide/document provided from count to for_each? This is basically to take advantage of for_each in existing logic where count is used for list and map.

[psalaberria002]

Is this currently being worked on? Is there any expected delivery date?

[lorengordon]

Looks like it's getting close, https://www.hashicorp.com/blog/terraform-0-1-2-preview

[apparentlymart]

As described in the blog article on for and for_each, some groundwork for this has be laid in the 0.12 development work so far but this particular feature won't land until a subsequent minor release, just because we don't want to hold up releasing the other improvements in order to include this one.

We are still planning to move ahead with this for_each design, but there is a little more internal reorganization to do within Terraform Core to get it ready to deal with all of the mechanisms we're familiar with for count: destroying instances when their corresponding key is no longer present in the for_each value, making the multiple instances available in expressions via their keys, etc.

[jakauppila]

@apparentlymart Is this the issue to watch for further for_each design/implementation or are there others? I'd like to keep up with and test the functionality as it becomes available.

I was trying it out with v0.12.0-alpha2 and was hitting a few issues, I'm guessing as it's not fully implemented yet?

variable "lb_subnet_numbers" {
  description = "Map from availability zone to the number that should be used for each availability zone's subnet for the LOAD BALANCER tier."
  type = "map"
  default = {
    "a" = 0
    "b" = 1
    "c" = 2
  }
}

resource "aws_subnet" "lb" {
  for_each = var.lb_subnet_numbers

  vpc_id            = aws_vpc.this.id
  availability_zone = "${var.aws_region}${lower(each.key)}"
  cidr_block        = cidrsubnet(aws_vpc.example.cidr_block, 8, each.value)
}

And the subsequent error:

λ terraform plan

Error: Incorrect value type

  on C:\GIT_Workspaces\foobar\main.tf line 33, in resource "aws_subnet" "lb":
  33:   for_each = var.lb_subnet_numbers

Invalid expression value: number required.

[apparentlymart]

Hi @jakauppila!

This feature is not part of the v0.12.0 scope but as you've seen we laid some groundwork for it in preparation for implementing it later. Funnily enough today I've been working on fixing up the error messages for these cases where we're reserving a name for later use, including that one.

This is the issue we'll post updates on as we have them. We're hoping to finish and release the for_each feature in a subsequent point release of 0.12, rather than make the 0.12.0 release later by holding for it.

[timotab]

I'm having the same issue - creating a bunch of subnets in AWS, using a list, currently, but adding to / removing from the list causes subnets to be destroyed and recreated, because they are associated with the list index rather than inherent properties

[colinrgarcia]

I'm also having this issue with persistent disk creation when scaling out. If I try to add to the list of disks when creating a new VM, sometimes the list becomes reordered causing my old disks to be destroyed.

[cypher7518]

@timotab Can you not use a lifecycle policy for that to make it never re-issue the create/modify?

[eredi93]

@reverendtimm not sure i follow, could you expand a bit more? not sure i understand how you would do that 😕

[cypher7518]

@eredi93 undo. I think the lifecycle on a state will actually cause it to try and create stuff that already exists. We tested it here with route53 entries, and it errors out due to it applying stuff that already exists, and not modifying the one you've added. 👍

[eredi93]

from my experience using count to create resources like VPC subnets, route tables associations etc. or Route53 records it will cause downtime.
when the index changes it will destroy and recreate the resource. i tried it in a test environment and it will not:

errors out due to it applying stuff that already exists

it will pull the trigger and destroy the resources. if you have a create_before_destroy on Route53 it might fail as the ID is probably the domain but for subnets and route tables it wont be the case.
actually if you add create_before_destroy on a route table association you will face this issue https://github.com/terraform-providers/terraform-provider-aws/issues/7245

I think the use of lists instead of a map to keep track of multiple resources is a major limitation in Terraform. Using count It is not safe and i will not recommend to use it in a production environment.
I hope @apparentlymart , @jbardin or @mildwonkey will get the time to work on this as soon as Terraform 0.12.0 stable is out. We currently stopped migrating resources to Terraform as we would have had to migrate them using resources and not modules. this would be extremely verbose considering the size of our infrastructure which would make it unacceptable for our engineering organisation.
@reverendtimm I would love to get hear more about your workaround but after spending a lot of time on this i don't think there is a safe way to sue count

[cypher7518]

@eredi93 you're right. We've ended up just creating lots of resources for the DNS, instead of counting. Would be good to get that resolved though.

[walterdolce]

Ditto to the previous two comments before this and especially so for:

I think the use of lists instead of a map to keep track of multiple resources is a major limitation in Terraform

The moment you introduce count in the codebase, you get in trouble.

I wonder whether instead of having count creating a list but creating a map of resource id + name (?) + number as key would solve the "sliding windows" problem, if you see what I mean.

[jakauppila]

@apparentlymart Just wanted to ask since I saw it out there; is this feature planned for the v0.12.1 milestone or a later one?

[apparentlymart]

The Terraform team only plans one release at a time, so anything that is not planned for v0.12.0 has no offical plan at all. (The v0.12.1 milestone is, in practice, representing just "after v0.12.0" for the moment, until the v0.12.1 release is actually planned.)

However, with that said we expect that this for_each feature can arrive in _some_ v0.12.x release because all of the foundational work to support it (which required breaking changes to state/plan file formats, etc) have been completed already for the v0.12.0 initial release, so no further breaking changes should be required to complete it.

[jleclanche]

Damn, so it's not possible yet! I downloaded the 0.12 beta all excited and got a nice "reserved for future use" message.

I was looking for it in order to push gsuite's mx records for every gsuite domain owned, in a very compact way.
Something like this should end up being possible, right?

variable "gsuite_mx_records" {
    default = [
        { address = "ASPMX.L.GOOGLE.COM.", priority = 1 },
        { address = "ALT1.ASPMX.L.GOOGLE.COM.", priority = 5 },
        { address = "ALT2.ASPMX.L.GOOGLE.COM.", priority = 5 },
        { address = "ASPMX2.GOOGLEMAIL.COM.", priority = 10 },
        { address = "ASPMX3.GOOGLEMAIL.COM.", priority = 10 },
    ]
}

resource "cloudflare_record" "mx-records" {
    for_each = var.gsuite_mx_records
    domain = "example.com"
    name = "example.com"
    type = "MX"
    value = each.address
    priority = each.priority
    proxied = false
}

[dunnill]

One thing I was wondering, for this feature (for each resources) ... would it be possible to have them dynamically named? E.g.

cnames = [
  { "www" = {"value": "www...", "name": "www-record" ...}, ...
]
resource "cloudflare_record" "cname-${each.key}" {
  for_each = var.cnames
  name = each.value.name
  value = each.value.value
  ...
}

This would be very helpful for state management. This is because I could then delete one of the items in the list but it would only delete one record. Currently the reliance of count means that deleting anything but the last item results in a lot of change (as the items move down). Apart from being confusing, this could also result in outages where resources are replaced for no reason.

[MostHated]

variable "gsuite_mx_records" {
  default = [
      { address = "ASPMX.L.GOOGLE.COM.", priority = 1 },
      { address = "ALT1.ASPMX.L.GOOGLE.COM.", priority = 5 },
      { address = "ALT2.ASPMX.L.GOOGLE.COM.", priority = 5 },
      { address = "ASPMX2.GOOGLEMAIL.COM.", priority = 10 },
      { address = "ASPMX3.GOOGLEMAIL.COM.", priority = 10 },
  ]
}

resource "cloudflare_record" "mx-records" {
  for_each = var.gsuite_mx_records
  domain = "example.com"
  name = "example.com"
  type = "MX"
  value = each.address
  priority = each.priority
  proxied = false
}

I only just today decided to finally convert my DNS to Terraform and this is exactly what I was hoping / looking to do as I have a few dozen domains. Is there any current way to go about doing something like this, even if not quite as simple as opposed to just having all 5 full independent record entries for every domain?

[non7top]

@MostHated
something with count=length(var.gsuite_mx_records) and element(var.gsuite_mx_records, count.index) should probably do it. in this case the entries can be destroyed and recreated without problem when you change your list.
And we need for_each for the case when the resources cant' be destroyed (e.g. ec2 instances).

[MostHated]

@non7top

I appreciate the reply. Glad to hear there is at least a way. I will do some more research on that and see what I can come up with.

Thanks!
-MH

[jleclanche]

@MostHated Yes, I ended up with this:

variable gsuite_mx_records {
… // as above
}

resource "cloudflare_record" "example_com_mx" {
    domain = "example.com"
    name = "example.com"
    type = "MX"
    value = "${lookup(var.gsuite_mx_records[count.index], "address")}"
    priority = "${lookup(var.gsuite_mx_records[count.index], "priority")}"
    proxied = false
    count = "${length(var.gsuite_mx_records)}"
}

It definitely feels less natural, hope we can get for_each soon :)

[brikis98]

@apparentlymart So will Terraform 0.12 still have no solution for the issue described in https://github.com/hashicorp/terraform/issues/14275 (which is now closed)? That is, if you use count to create multiple resources and delete one in the middle, all the others after it get deleted and recreated too?

[apparentlymart]

I closed that other issue because it is covering the same use-case as this one. This issue now represents that, so we'll close this one once the solution is available.

[eredi93]

@brikis98 from https://github.com/hashicorp/terraform/issues/17179#issuecomment-440358001 it seems they are hoping to finish and release the for_each feature in a subsequent point release of 0.12

I honestly hope this will happen in the near future as count is not safe for production use 😞

[brikis98]

Understood. Thx @apparentlymart and @eredi93.

[DenisBY]

According to Upgrading to Terraform v0.12 it's implemented. My colleague tried for_each with v0.12.0-rc1 and found it working. Or I missed something?

[apparentlymart]

I'm not sure what part of the upgrade guide you are referring to, but I suspect you're thinking of dynamic blocks, which are not the same thing as resource-level for_each even though they also use an argument named for_each.

dynamic blocks allow dynamic creation of nested blocks _within_ resources. This issue is about using for_each on the resource itself, as a replacement for count.

The reason these things are different is that blocks _within_ a resource block (aside from the meta-arguments) are just a normal part of the resource's object representation and so the dynamic block just behaves as a sort of macro, as if you had manually written out several blocks. Resource-level for_each must integrate with Terraform's built-in resource addressing, so it behaves slightly differently.

For example, if you were to write the following:

locals {
  things = {
    foo = bar
    baz = boop
  }
}

resource "null_resource" "example" {
  for_each = local.things

  triggers = {
    key   = each.key
    value = each.value
  }
}

...Terraform would see this as declaring two resource instances with the following addresses:

• null_resource.example["foo"]
• null_resource.example["baz"]

This is not the same as what would've happened if you just wrote two separate resource blocks, since in that case they would've been required to have distinct names. It's also different than using count, because the instance keys are the strings "foo" and "baz", rather than numeric indices 0, 1. This is the important detail that makes this better than count: Terraform will then correlate the indices by these string keys rather than by their positions in a sequence, so no particular ordering is implied or assumed.

What we did for Terraform 0.12 is prepared Terraform's internal models and file formats to support instance keys being strings. The remaining work is to change the current "expand" logic that handles count to also deal with for_each, and to track the expression result so that each.key and each.value will return suitable results inside those blocks.

[DenisBY]

I see now. Thanks!

[mariobede]

I understand there is not ETA or preview code i can compile to test/use in the meantime.

[bechampion]

I'd be really interested on a very rough estimate for for_each on maps , im making some place holders in my design and it would be handy to know when this amazing feature will come along.
in the meantime i guess terraform state mv will save the day in case of a deletion of a list element which isn't the last one.

[lyc0221]

version:
Terraform v0.12.1

main.tf

data "template_file" "userdata" {
  template = file("../../user_data.tpl")
}

locals {
  hosts = {
    test1 = "one",
    test2 = "two"
  }
}

resource "digitalocean_droplet" "this" {
    for_each = locals.hosts
    image = var.image
    name = each.key
    region = var.region
    size = var.size
    private_networking = var.private_networking
    user_data = data.template_file.userdata.rendered
    lifecycle {
      ignore_changes = ["user_data", "ssh_keys", "image"]
    }
    tags = concat(var.tags, list(digitalocean_tag.this.id))

    depends_on = ["digitalocean_tag.this"]
}

resource "digitalocean_tag" "this" {
    name = format("%s-%s", var.service, var.environment)
}

terraform init, and report the following error:

Error: Reserved argument name in resource block

  on ./main.tf line 13, in resource "digitalocean_droplet" "this":
  13:     for_each = locals.hosts

The name "for_each" is reserved for use in a future version of Terraform.

is not it support this now ?

[bostrowski13]

would love to see this implemented as well.

I like to use something like this for conditionally creating resources based on the value in a map thats part of a list (list of maps?).

variable "api_apps" {
  type = list(
    object({
      api_name                    = string
      api_swagger_definition_file = string
      api_endpoint_type           = string
      api_stage_variables         = map(string)
      lambda_handler              = string
      lambda_runtime              = string
      lambda_func_name            = string
      lambda_deploy_package       = string
      lambda_src_dir              = string
      deploy_package_cc_option    = string
      endpoint_version            = number
  }))
  description = "A list of objects that represent the properties of the configuration items and values for each Rest API"
}

api_apps = [
  {
    api_name                    = "TestAPI",
    api_swagger_definition_file = "./examples/files/swagger-definition.yaml",
    api_endpoint_type           = "PRIVATE", # EDGE, REGIONAL, PRIVATE
    api_stage_variables         = { answer = 42, things = "stuff" },
    lambda_handler              = "main.handler",
    lambda_runtime              = "nodejs8.10",
    lambda_func_name            = "ServerlessExample",
    lambda_deploy_package       = "examples/example.zip",
    lambda_src_dir              = "examples/src",
    deploy_package_cc_option    = "max-age=604800, public",
    endpoint_version            = 1,
  },
  {
    api_name                    = "SampleAPI",
    api_swagger_definition_file = "./examples/files/swagger-definition-sample.yaml",
    api_endpoint_type           = "REGIONAL", # EDGE, REGIONAL, PRIVATE
    api_stage_variables         = { answer = 25, rather = "six to four" },
    lambda_handler              = "main.handler",
    lambda_runtime              = "nodejs8.10",
    lambda_func_name            = "ServerlessSample",
    lambda_deploy_package       = "examples/sample.zip",
    lambda_src_dir              = "examples/src",
    deploy_package_cc_option    = "max-age=604800, public",
    endpoint_version            = 1,
  },
]

data "aws_iam_policy_document" "private_api_policy" {
  count = # check if any of the values of property of var.api_apps.api_endpoint_type == "PRIVATE" ? 1 : 0
  ... rest of resource properties
}

[PsyonixBarrett]

Forgive me if this has already been discussed previously, as I couldn't find a reference to it anywhere, but is there any plan to address the resource name within a for_each resource definition? With the current practice of using count the names are appended with their prefix within the loop.

In some cases this is fine behavior, but when referencing a resource that is generated from a loop it would be more accurate if we could also control the names of these resources.

[JadyDJ]

We look for exact same feature as @bostrowski13 posted above

[edwardaskew]

So, now that we're a few minor releases into 0.12 I don't suppose there's any indication as to how far away this feature is? It's currently a blocker on how I'd like to set Terraform up and I'd like to know whether it's worth continuing to wait on this or start looking into other options...

[wyardley]

I had assumed that a) this feature would have made it in to 0.12 release, and b) that the original pseudo-code (where the loop is outside the resource itself) was the way this was going to go, and that c) the templating features would allow using templating to create resources within the language (vs. using something like jinja to define tf resources).

Especially after such a long wait, the way some of these features are implemented in practice is definitely not ideal, IMHO, but just want to confirm... when this feature is eventually released, will there be a way to have the created resources named based on a list / map attribute?

In terms of actual pain this causes, we do a lot of things like, say, pass a list of roles to a module, and then create those roles. When the roles are indexed by [0], [1], etc., not only are the names of the resources meaningless (module.foo-gke-cluster.google_project_iam_member.project[3]), but the index slices get shuffled causing resources to get destroyed / re-created if the list order gets shuffled. Or, say we want a module to create resources based on a data structure, same sort of problem comes up.

[DavidGamba]

Mitchel had a twitter thread from someone that asked about it (https://twitter.com/_pczora/status/1145658448274759680).
He mentioned a WIP PR: https://github.com/hashicorp/terraform/pull/21922
No ETA though.
The design of the feature was described here: https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each
I am waiting for it myself. Hope this information helps.

[bentterp]

Recent activity in the PR, so I'm hopeful.
This is very much something that would make our work easier and our implementations much more robust.
Fingers crossed!

[ViggyNash]

I wish there was some documentation to indicate to users that count should not be used dynamically. I've had to refactor a a chunk of my code and throw out another project that would have relied on that after discovering the count issue.

[lorengordon]

@ViggyNash As with so many docs, the explanation is there, it just needs to be read carefully and probably doesn't make a lot of sense until it bites you. :/

The count meta-argument accepts expressions in its value, similar to the resource-type-specific arguments for a resource. However, Terraform must interpret the count argument before any actions are taken from remote resources, and so (unlike the resource-type-specifc arguments) the count expressions may not refer to any resource attributes that are not known until after a configuration is applied, such as a unique id generated by the remote API when an object is created.

Note that the separate resource instances created by count are still identified by their index, and not by the string values in the given list. This means that if an element is removed from the middle of the list, all of the indexed instances after it will see their subnet_id values change, which will cause more remote object changes than were probably intended. The practice of generating multiple instances from lists should be used sparingly, and with due care given to what will happen if the list is changed later.

https://www.terraform.io/docs/configuration/resources.html#count-multiple-resource-instances

[tmccombs]

Is there anything I could do to help get this feature added or the PR merged?

[pselle]

@tmccombs Thank you for the kind offer!

We are indeed prioritizing this for a near release of Terraform, at which point I'd say the best way to help is to use it and help us find bugs that will inevitably show up in the first pass of the feature (you could do this now, by building the active PR; it would help me best if anything you find you comment on the PR itself).

[bentterp]

for_each fails for me when using a local variable map
more details in the PR

[ghost]

I have a question regarding "count" vs "for". I am experiencing all the issues discussed here, too. But my design was planned to have something like

module "zookeepers" {
  source      = "../modules/virtual-machine"
  vm_count    = var.zk_count
  datadisks = [
    {
      id   = 0
      type = "Standard_LRS"
      size = "128"
    },
    {
      id   = 1
      type = "Standard_LRS"
      size = "128"
    },
  ]
 }

wich means I would need to maintain a two-dimensional array of count and length(var.datadisks). PLaying with DIV and MOD tricks doesn't work since disks would get reshuffled on count change. Any chance to get this done with the new approach?

[simonbirtles]

Using TF 0.12.4 this is how I am dealing with disks with a vsphere_virtual_machine with dynamic disks without relying on indexing with count.

The vsphere_virtual_machine machine resource has the following disk configuration:

  # Data disks #'s 1-...
  # disk.tag and disk.value available on each loop
  dynamic "disk" {
    for_each = [for data_disk in var.disks : {
      disk_unit_number = data_disk.unit_number
      disk_label       = format("disk%d", data_disk.unit_number)
      disk_size        = data_disk.size
      }
      if data_disk.unit_number != 0
    ]

    content {
      unit_number      = disk.value.disk_unit_number
      label            = disk.value.disk_label
      size             = disk.value.disk_size
      eagerly_scrub    = false
      thin_provisioned = true
      keep_on_remove   = false
    }
  }

The variables come from a file specific to the virtual machine similar to what you have above:

"module": [
    {
      "vm_id...": {
      "source": "......",
      "disks": [
            {
                "unit_number": 0,
                "size": "60"
            },
            {
                "unit_number": 1,
                "size": "120"
            },
            {
                "unit_number": 2,
                "size": "100"
            },
            {
                "unit_number": 3,
                "size": "100"
            },
            {
                "unit_number": 4,
                "size": "100"
            }
            ],

Note: I exclude disk 0 as its part of our image but we have the disk in the server spec file for completness, hence the skipping on disk 0 in the dynamic loop.

[ghost]

thanks @simonbirtles I get the idea.
Would you create the vm's based on that script, too? I just wonder how to convet the machine list entries intro resource commands...

[simonbirtles]

Hi @desixma ,

If I understand correctly I will briefly describe our workflow....

We have a workflow which starts with a file per resource (i.e. vm) which is our own DDL spec which contains all details required for a virtual machine (vm specific, configuration and software), each file is converted using a j2 template into a terraform config file per resource/vm which contains the module config with variables as I showed in the second snippet. These files (per vm) and a basic main/var/output.tf files for the full terraform configuration.

The files above import a generic module for a vmware virtual machine, the first snippet is part of a generic vmware vm resource which is imported by each terraform config file (file per vm).

From what I see, I think your approach is fine, you would need to have seperate modules for zookeeper1, zookeeper2, ... of course and add in the 2nd snippet I provided into your module ../modules/virtual-machine with some adjustment of var names etc.

TF does not yet support dynamic resources so I create a module per vm and import the generic module which is the usual approach I would say. Once the feature being discussed here is available we "hopefully" can move away from different modules.

Hope that helps.

[ghost]

ah I see. Thanks for the clarification

[cyrus-mc]

How do you access all resources when using for_each.

For example I have the following:

# create required policies
resource "aws_iam_policy" "main" {
  for_each = var.policies

  name = each.key
  path = lookup(each.value, "path", "/")

  policy = templatefile("${var.policies_dir}/${lookup(each.value, "template", each.key)}.tmpl",
                        lookup(each.value, "vars", {}))
}

Prior to this I just used count on a list and then could perform the following:

  /* create a map with key = policy name and value being the policy arn */
  policy_name_arn_mapping = zipmap(aws_iam_policy.main.*.name, aws_iam_policy.main.*.arn)

With for_each I get

Error: Unsupported attribute

  on ../variables.tf line 11, in locals:
  11:   policy_name_arn_mapping = zipmap(aws_iam_policy.main[*].name, aws_iam_policy.main[*].arn)

This object does not have an attribute named "name".


Error: Unsupported attribute

  on ../variables.tf line 11, in locals:
  11:   policy_name_arn_mapping = zipmap(aws_iam_policy.main[*].name, aws_iam_policy.main[*].arn)

This object does not have an attribute named "arn".

[tmccombs]

Unfortunately, HCL's splat syntax only supports lists (perhaprs that should be changed in the upstream hcl project?).

You can still access all of them by using the values function to convert the values to a list. for example:

/* create a map with key = policy name and value being the policy arn */
  policy_name_arn_mapping = zipmap(values(aws_iam_policy.main)[*].name, values(aws_iam_policy.main)[*].arn)

Although in this case it might be better just to use a for_each comrehension such as:

{for name, policy in aws_iam_policy.main: name => policy.arn}

[timota]

How dynamically we can create such map , so it can be passed to for_each within resource block

  disk_by_inst = {
    first-1 = {
      type = "gp2"
      size = 10
    },
    first-2 = {
      type = "io2"
      size = 20
    },
    second-1 = {
      type = "ght"
      size = 30
    },
    second-2 = {
      type = "gp2"
      size = 40
    }
  }

input params:

variable "instances_id" {
  default = ["first", "second", "third"]
}

variable "disks" {
  type        = list(map(string))
  default     = []
}

where

disks = [
  {
    size = 50
  },
  {
    iops = 100
    size = 10
  }
]

i tried different methods, but no luck. the closest is

  sum = {
    for s in var.instances_id :
    "${s}-vol-${local.counter + 1}" => { for k, v in var.disks : k => v }
  }

which gives this output

out = {
  "first-vol-1" = {
    "0" = {
      "size" = "50"
    }
    "1" = {
      "iops" = "100"
      "size" = "10"
    }
  }
  "second-vol-1" = {
    "0" = {
      "size" = "50"
    }
    "1" = {
      "iops" = "100"
      "size" = "10"
    }
  }
  "third-vol-1" = {
    "0" = {
      "size" = "50"
    }
    "1" = {
      "iops" = "100"
      "size" = "10"
    }
  }
}

[OJFord]

@timota It's not very clear to me what your desired output is, but if it's that:

"<nth>-vol-<i>" = {
  "0" = {...}
  "1" = {...}
}

should instead have only the 0 or 1 block corresponding to i - 1, then you need to change:

{ for k, v in var.disks : k => v }

to:

var.disks[local.counter]

(but that local.counter's not going to change in the loop, so you're only going to get *-vol-1.)

[timota]

ahg sorry.

my goal is to get this map

{
    <inst-01a-vol-01> = {...}
    <inst-01a-vol-02> = {...}
    <inst-02a-vol-01> = {...}
    <inst-02a-vol-02> = {...}
}

so i can use keys (inst-xxx-vol-xxx) in for_each for resource to create named resources

[OJFord]

@timota I think you want something like:

{
  for instdisk in setproduct(var.instance_ids, var.disks)
  : "inst-${index(var.instance_ids, instdisk[0])}-vol-${index(var.disks, instdisk[1])}"
  => instdisk[1]
}

(but note I haven't tested it!)

This is quite a good of where an 'enumerate' function would be helpful (#21940).

[timota]

cool, thanks. will test and let you know

[timota]

it works like a charm

Many thanks.

[Hatsou]

Hi guys im currently try to create dynamically data disk for some virtual machines. The creation of data disk. For exemple : I want to create 2 VM with 3 data disks
so the expected result for the name of data disks is :
DD01-VM-1
DD02-VM-1
DD03-VM-1
DD01-VM-2
DD02-VM-2
DD03-VM-2
I use the following code :

Creation of data disks

resource "azurerm_managed_disk" "MyDataDiskVm" {
  count                = "${var.nb_data_disk * var.nb_vms}"
  name = "${format("DD%02d", (count.index % var.nb_data_disk) +1)}-VM-${var.vm_name_suffix}${format("%d", floor((count.index / var.nb_data_disk) +1))}"
  location             = "${var.location}"
  resource_group_name  = "${var.resource_group_name}"
  storage_account_type = "Standard_LRS"
  disk_size_gb         = "${var.data_disk_size_gb}"
  create_option        = "Empty"
  depends_on = ["azurerm_virtual_machine.MyVms"]
}

Attach the created data disk to virtual machine

resource "azurerm_virtual_machine_data_disk_attachment" "MyDataDiskVmAttach" {
  count              = "${var.nb_data_disk * var.nb_vms}"
  managed_disk_id    = "${azurerm_managed_disk.MyDataDiskVm.*.id[count.index]}"
  virtual_machine_id = "${azurerm_virtual_machine.MyVms.*.id[ceil((count.index +1) * 1.0 / var.nb_data_disk) -1]}"
  lun                = "${count.index % var.nb_data_disk}"
  caching            = "ReadWrite"
  create_option      = "Attach"
  depends_on         = ["azurerm_managed_disk.MyDataDiskVm"]
}

Everything works fine, datadisks are created with the right name is correctly attached to vm but once i restart an "apply" Terraform wants to change the id of the datadisks and therefore destroy and recreate it..

-/+ azurerm_managed_disk.MyDataDiskVm[0] (new resource required)

Im using Terraform v0.11.11.

Do you know where the error may come from or is it possible to dynamically create data disks with an "for each" with the azurerm provider ?

Thx for your feedback

[neelam-007]

Hi Guys,

I need to dynamically generate an entire resource block based on a map. So, in one file, I need something like the following, repeated for each.value:

variable "my_map" {
  type = "map"
  default = {
    key1 = "key1val",
    key2 = "key2val"
  }
}
# do the following for each entry in my_map
resource "vault_policy" "${each.key}-admin" {
  name="${each.value}-admin"
  path "ab/cd/ef/${each.value}" {
    capabilities = ["list"]
  }
  path "gh/ij/kl/${each.value}" {
    capabilities = ["list", read"]
 }
} 

How can I achieve this for_each? So far what I've tried is not working. Note that I've successfully generated one _single_ resource block with a bunch of path definitions based on a map, but I don't exactly know what the syntax should look like for generating repeated resource blocks.

Thanks for feedback/help.

[pselle]

Hi friends! While I appreciate seeing folks help each other, please use the community forums to ask questions, and help future people who are asking similar questions. Thank you!

[sanzen193]

Hi All,

I am using a for_each to assign a new network to each VM that I create

data "vsphere_network" "network1" {
count = var.VMcount
name = "${var.network_name1}-${format("%02d", count.index + var.start_index)}"
datacenter_id = data.vsphere_datacenter.datacenter.id
}

data "vsphere_virtual_machine" "template" {
name = var.disk_template
datacenter_id = data.vsphere_datacenter.datacenter.id
}

=========================================================

FOLDER, TFVM RESOURCES ETC.

=========================================================

resource "vsphere_folder" "chefinfra" {
datacenter_id = data.vsphere_datacenter.datacenter.id
path = var.vmfolder
type = var.vsphere_folder_type
}

resource "vsphere_virtual_machine" "tfvm" {
for_each = {for net in data.vsphere_network.network1:net.id => net}

datastore_id = data.vsphere_datastore.datastore.id
resource_pool_id = data.vsphere_resource_pool.pool.id
#count = var.VMcount
name = "${var.vmname}${format("%02d", each.name + var.start_index)}"
annotation = var.tfvm_annotation
folder = vsphere_folder.chefinfra.path
hv_mode = var.hv_mode
nested_hv_enabled = var.nested_hv_enabled
num_cpus = var.cpu
num_cores_per_socket = var.cpu
cpu_hot_add_enabled = true
cpu_hot_remove_enabled = true
memory = var.memory
memory_hot_add_enabled = true
guest_id = var.guest_id
scsi_type = data.vsphere_virtual_machine.template.scsi_type
wait_for_guest_net_timeout = var.guest_net_timeout

unfortunately since I am using v0.12.6, I cannot use the count.index to dynamically name my VMs - since I have the for_each. What is the alternative to create VM names dynamically while in a for_each? The VM names need to increment by 1. Thanks in advance