Terraform: Function call argument expansion fails when argument has unknown type

[unixninja92]

Terraform Version

Terraform v0.12.7

Terraform Configuration Files

vars.tf

variable "iam_groups" {
  type = map(map(list(string)))
  default = {
    "group1" = {},
    "group2" = {},
    "group3" = {},
    "group4" = {
      "pollicy-n" = ["resource-n1"],
      "pollicy-p" = ["resource-p1"]
      },
    "group5" = {},
    "group6" = {
      "policy-x" = ["resource-x1", "resource-x2"]
    }
  }
}

main.tf

locals {
  #map of all the policies
  policies = merge(toset(values(var.iam_groups))...)
  #list of all the resources
  iam_resources = concat(tolist(values(merge(toset(values(var.iam_groups))...)))...)
}

Crash Output

Error: Invalid expanding argument value

  on main.tf line 63, in locals:
  63:   iam_resources = concat(tolist(values(merge(toset(values(var.iam_groups))...)))...)

The expanding argument (indicated by ...) must be of a tuple, list, or set
type.

Expected Behavior

The list of lists of resources should have been combined into one list with the elements expanded as arguments for the function concat.

Actual Behavior

... fails to separate the elements of a list (this happens with sets as well) into separate function arguments when there is a nested .... This also occurs when referencing local.polices instead or recreating it in the iam_resources definition

Steps to Reproduce

terraform init

terraform plan

Additional Context

I'm trying to create a dynamic number of iam polices using for_each for a dynamic set of resources (with for) and attach them to a dynamic set of groups (with for_each) to get around aws iam group attach limits and create other aws resources based on the dynamic set of resources using for_each. I need iam_resources to be a set of strings, but because of this issue I was trying to test with a list of strings first.

[teamterraform]

Hi @unixninja92! Sorry for this strange behavior, and thanks for reporting it.

It looks like this is a bug in the underlying language engine: it's not correctly handling the situation where the type of the ... argument isn't known yet, which is likely the case here because during validation Terraform is just checking that this expression is correct for _any_ value of var.iam_groups and one of those intermediate functions is probably indicating that it doesn't yet have enough information to predict the result type.

The intended behavior for all situations is that an unknown type _always_ passes type checking, so that its validation can be deferred to a later step when more information is available. So the fix here will be to add an additional case to the language engine to deal with the unknown type situation and have the function call as a whole indicate that its result is unknown, allowing validation to complete successfully in this case.

[alxrem]

Bug is reproduced with such config:

variable "z" {
  type = "list"

  default = [1, 2]
}

locals {
    z = [1, 2]

    a1 = max(local.z...)
    a2 = max(var.z...)

    b1 = max([for x in local.z : x]...)
    b2 = max([for x in var.z : x]...) # it causes error
}

Seems impossible to expand result of "for" expression, if list passed as variable.

[cytopia]

Temporary solution

@unixninja92 I faced the same issue (also for an IAM role/policy module) for Terraform v0.12.12.

However, I managed to build a work-around that still makes it possible to do it.

Input var

terraform.tfvars

roles = [
  {
    name              = "ROLE-DEVELOPER"     # required: name of the role
    path              = "/"                   # defaults to 'path' variable if not set
    desc              = "TERRAFORM MANAGED"   # defaults to 'description' variable if not set
    trust_policy_file = "policies/trust.json" # required: defines trust/assume policy
    inline_policies = []
    policies = []
    policy_arns = []
  },
  {
    name              = "ROLE-ADMIN"     # required: name of the role
    path              = "/"                   # defaults to 'path' variable if not set
    desc              = "TERRAFORM MANAGED"   # defaults to 'description' variable if not set
    trust_policy_file = "policies/trust.json" # required: defines trust/assume policy
    inline_policies = [
      {
        name = "tmp-policy-1"
        file = "policies/policy-1.json"
      }
    ]
    policies = [
      {
        name = "tmp-policy-3"
        file = "policies/policy-3.json"
      },
      {
        name = "tmp-policy-4"
        path = "/"
        desc = "tmp-desc-2"
        file = "policies/policy-4.json"
      },
      {
        name = "tmp-policy-7"
        path = "/"
        desc = "tmp-desc-2"
        file = "policies/policy-7.json"
      },
    ]
    policy_arns = [
      "arn:aws:iam::aws:policy/PowerUserAccess",
      "arn:aws:iam::aws:policy/job-function/Billing",
    ]
  },
]

Desired loop-able locals

Does not work and produces the same error you get

locals.tf

locals {
  # Optional policies if specified.
  # Converts specified roles and policies into the following format:
  #
  #   policies = {
  #     "<role-name>:<policy-name>" = {
  #       "name" = ""
  #       "path" = ""
  #       "desc" = ""
  #       "file" = ""
  #     }
  #     "<role-name>:<policy-name>" = {
  #       "name" = ""
  #       "path" = ""
  #       "desc" = ""
  #       "file" = ""
  #     }
  #   }
  policies = merge([
    for i, role in local.roles : {
      for j, policy in lookup(local.roles[i], "policies", {}) :
        "${local.roles[i]["name"]}:${local.roles[i]["policies"][j]["name"]}" => policy
    }
  ]...)

  # Optional inline policies if specified.
  # Converts specified roles and policies into the following format:
  #
  #   inline_policies = {
  #     "<role-name>:<policy-name>" = {
  #       "name" = ""
  #       "file" = ""
  #     }
  #     "<role-name>:<policy-name>" = {
  #       "name" = ""
  #       "file" = ""
  #     }
  #   }
  inline_policies = merge([
    for i, role in local.roles : {
      for j, inline_policy in lookup(local.roles[i], "inline_policies", {}) :
        "${local.roles[i]["name"]}:${local.roles[i]["inline_policies"][j]["name"]}" => inline_policy
    }
  ]...)

  # Optional policy arns if specified.
  # Converts specified roles and policies into the following format:
  #
  #   policy_arns = {
  #     "<role-name>:<policy-arn>" = "<policy-arn">
  #     "<role-name>:<policy-arn>" = "<policy-arn">
  #   }
  policy_arns = merge([
    for i, role in local.roles : {
      for j, policy_arn in lookup(local.roles[i], "policy_arns", {}) :
        "${local.roles[i]["name"]}:${local.roles[i]["policy_arns"][j]}" => policy_arn
    }
  ]...)
}

Work around

This is the fix that I came up with, which works with variables as input. The commented code shows what is first produced by each _ underscored local. And in the second loop iteration (without underscore) the final local is produced as shown above in the desired local section

locals.tf

locals {
  # Optional policies if specified.
  # Converts specified roles and policies into the following format:
  #
  #   policies = [
  #     {
  #       "<role-name>:<policy-name>" = {
  #         "name" = ""
  #         "path" = ""
  #         "desc" = ""
  #         "file" = ""
  #       }
  #     },
  #     {
  #       "<role-name>:<policy-name>" = {
  #         "name" = ""
  #         "path" = ""
  #         "desc" = ""
  #         "file" = ""
  #       }
  #     },
  #   }
  _policies = flatten([
    for i, role in var.roles : [
      for j, policy in lookup(var.roles[i], "policies", {}) : {
        "${var.roles[i]["name"]}:${var.roles[i]["policies"][j]["name"]}" = policy
      }
    ]
  ])
  # The fix to bring it into the format stated at the top of this file
  policies = {
    for i, v in local._policies :
      keys(local._policies[i])[0] => local._policies[i][keys(local._policies[i])[0]]
  }

  # Optional inline policies if specified.
  # Converts specified roles and policies into the following format:
  #
  #   inline_policies = [
  #     {
  #       "<role-name>:<policy-name>" = {
  #         "name" = ""
  #         "file" = ""
  #       }
  #     },
  #     {
  #       "<role-name>:<policy-name>" = {
  #         "name" = ""
  #         "file" = ""
  #       }
  #     },
  #   ]
  _inline_policies = flatten([
    for i, role in var.roles : [
      for j, inline_policy in lookup(var.roles[i], "inline_policies", {}) : {
        "${var.roles[i]["name"]}:${var.roles[i]["inline_policies"][j]["name"]}" = inline_policy
      }
    ]
  ])
  # The fix to bring it into the format stated at the top of this file
  inline_policies = {
    for i, v in local._inline_policies :
      keys(local._inline_policies[i])[0] => local._inline_policies[i][keys(local._inline_policies[i])[0]]
  }

  # Optional policy arns if specified.
  # Converts specified roles and policies into the following format:
  #
  #   policy_arns = [
  #     {
  #       "<role-name>:<policy-arn>" = "<policy-arn">
  #     },
  #     {
  #       "<role-name>:<policy-arn>" = "<policy-arn">
  #     },
  #   ]
  _policy_arns = flatten([
    for i, role in var.roles : [
      for j, policy_arn in lookup(var.roles[i], "policy_arns", {}) : {
        "${var.roles[i]["name"]}:${var.roles[i]["policy_arns"][j]}" = policy_arn
      }
    ]
  ])
  # The fix to bring it into the format stated at the top of this file
  policy_arns = {
    for i, v in local._policy_arns :
      keys(local._policy_arns[i])[0] => local._policy_arns[i][keys(local._policy_arns[i])[0]]
  }
}

Usage

The usage will also stay the same, once the above Terraform issue has been resolved
main.tf

# ... this is only a small abstract of main.tf to show how that can be used


# Attach customer managed policies
resource "aws_iam_role_policy_attachment" "policy_attachments" {
  for_each = local.policies

  role       = replace(each.key, format(":%s", each.value.name), "")
  policy_arn = aws_iam_policy.policies[each.key].arn

  # Terraform has no info that aws_iam_roles must be run first in order to create the roles,
  # so we must explicitly tell it.
  depends_on = [aws_iam_role.roles]
}

# Attach policy ARNs
resource "aws_iam_role_policy_attachment" "policy_arn_attachments" {
  for_each = local.policy_arns

  role       = replace(each.key, format(":%s", each.value), "")
  policy_arn = each.value

  # Terraform has no info that aws_iam_roles must be run first in order to create the roles,
  # so we must explicitly tell it.
  depends_on = [aws_iam_role.roles]
}

[hashibot]

Hello! :robot:

This issue seems to be covering the same problem or request as #22404, so we're going to close it just to consolidate the discussion over there. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.