Terraform: Define resources by iterating over a list

[ejoubaud]

Feature idea: a resources declaration to declare several resources, possibly by iterating over variables (list or map)

Say you want to manage IAM users with Terraform, but DRY-up their groups (or other attributes)

variable "developers" {
  default = ["username1", "username2"]
}

resources "map" "users" {
  list = "${var.developers}"
  resource "aws_iam_user" "user-${value}" {
    name = "${value}"
    path = "/"
  }
}

resource "iam_aws_group_membership" "developers" {
  name = "DevelopersMembership"
  users = "${map.users.aws_iam_user.*.name}"
  group = ...
}

* map refers the map function in functional programming, not the map data structure. It could be named otherwise, it's just a way to leave the door for several resources collection method.

[ejoubaud]

Note: The doc mentions the count method that can solve that problem (iterations), in a more concise but arguably less declarative way.

If length worked on maps, it would be pretty much functionally equivalent (#1389), except this proposal lets the user define a unique name for each defined resource.

[apparentlymart]

I was thinking about something like this over in #3310, originally with a more complex design but eventually concluding that iterating over a list was sufficient.

Just linking that here because there are some use-cases in it that might be helpful when thinking about the design for this.

[jen20]

I think there might be some use cases for this - but you can achieve effectively the same thing right now with an (untested) construct like this:

variable "list_of_users" {
    default = "username1,username2"
}

resource "aws_iam_user" "user-${value}" {
  count = "${len(split(",", var.list_of_users))}"
  name = "${element(split(",", var.list_of_users), count.index}"
  path = "/"
}

# repeat for the other resource

I'll leave this issue open tagged for future attention for now - as @apparentlymart says this is something that has been considered before and we will come back to soon.

[johnrengelman]

@jen20 the issue here is that the resource gets stored with the count.index value in the resource name in the state file. Thus any change to the source list (like adding a value in the middle of the list) causes all subsequent resources to be modified.

The construct your posted above also doesn't work because the ${value} in the resource name won't eval to anything.

I'm running into this issue when trying to manage a list of users for Github.
Someone adds a name to the list and then the next 40 resources get recreated b/c their indices changed.
Would be awesome, if you you could specify a custom value for the index instead of always using count.index.

[kpumuk]

👍

Consider this use case:

variable "repositories" {
  default = ["a", "b", "c"]
  type    = "list"
}

resource "github_team_repository" "repository" {
  team_id    = "${github_team.team.id}"
  repository = "${element(var.repositories, count.index)}"
  permission = "pull"
  count      = "${length(var.repositories)}"
}

If I delete repository "a" from the list, indexing will change, and so all the permissions will be deleted, and then created again. Normally it would not be a problem, but with Github it will send emails to ever team member. With the proposed change it wouldn't happen - only the resource in question would be deleted.

[keymon]

If I delete repository "a" from the list, indexing will change, and so all the permissions will be deleted, and then created again. Normally it would not be a problem,

Indeed that happens. And although "normally" it would not be a problem, it is a huge mess in other cases. Sometimes the resource is expensive to re-create, or even worse, it can trigger a recreation of dependent resources in cascade which would destroy your deployment.

For instance: a AWS subnet, if we create the subnets based on a list, deleting one subnet might trigger the recreation in cascade of security groups, DBs, instances, etc.

Even a simple case of a list of users to create: Deleting one user would recreate all the users in parallel, causing problems like errors due duplicated user names.

[apparentlymart]

Hi all!

I'm just looking through some older issues that weren't using our latest labeling scheme and so were missed on previous issue gardening expeditions.

This seems to be covering the same sort of use-case as #17179. Although this is the older of the two, there is some more discussion and updates over there so I'm going to close this one just to consolidate the discussion. Sorry for the long silence!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.