Terraform: Terraform Merge on Wildcard Tuple

[jcbagtas]

Terraform Version

Terraform v0.12.10
+ provider.azurerm v1.44.0

Terraform Configuration Files

locals {
  policy_definitions = [for item in data.azurerm_policy_definition.d_policy_definitions: {policyDefinitionId=item.id, parameters=jsondecode(item.parameters)}]
  test1 = merge(local.policy_definitions.*.parameters...) #returns error
  test2 = merge(local.policy_definitions.0.parameters,local.policy_definitions.1.parameters) #works
}

Debug Output

2020/04/13 13:54:44 [ERROR] module.policy_set_definitions: eval: *terraform.EvalLocal, err: Invalid expanding argument value: The expanding argument (indicated by ...) must be of a tuple, list, or set type.

Crash Output

test1 = merge(local.policy_definitions.*.parameters...)
The expanding argument (indicated by ...) must be of a tuple, list, or set
type.

Expected Behavior

test1 will have the merged parameter values

test1 = {test1={key=value},test2={key=value}}

Actual Behavior

Terraform Error

test1 = merge(local.policy_definitions.*.parameters...)
The expanding argument (indicated by ...) must be of a tuple, list, or set
type.

Steps to Reproduce

terraform plan -out=plans/test.tfplan

Additional Context

Manually adding indices on merge works but Expanding a wildcard variable breaks.

  test1 = merge(local.policy_definitions.*.parameters...) #returns error
  test2 = merge(local.policy_definitions.0.parameters,local.policy_definitions.1.parameters) #works

References

None

[danieldreier]

Hi! Thanks for reporting this. I think this is probably a valid issue, and I'd like to reproduce it locally.

To do that, I have to be able to run this and run it on my workstation without inventing any details in order to be confident we're seeing the same behavior. As-is, I don't know what's in your data.azurerm_policy_definition.d_policy_definitions or in item.parameters, and so I'm stuck.

Can you please restate your reproduction case such that I can copy-paste it and run it locally? Ideally, this would use the null resource provider rather than a real provider in order to minimize external dependencies.

[jcbagtas]

Hi! Thanks for reporting this. I think this is probably a valid issue, and I'd like to reproduce it locally.

To do that, I have to be able to run this and run it on my workstation without inventing any details in order to be confident we're seeing the same behavior. As-is, I don't know what's in your data.azurerm_policy_definition.d_policy_definitions or in item.parameters, and so I'm stuck.

Can you please restate your reproduction case such that I can copy-paste it and run it locally? Ideally, this would use the null resource provider rather than a real provider in order to minimize external dependencies.

Hello @danieldreier ! Thanks for looking in to this.

Actually, the values in azurerm_policy_definition are just basic Azure Policy configuration.

Here is an example:

{
  "properties": {
    "displayName": "require-terraTest2-tag",
    "policyType": "Custom",
    "mode": "Indexed",
    "description": "Policy to require the 'terraTest2' tag for all resources in a scope.",
    "metadata": {
      "createdBy": "xxx-xxx-xxx-xxx-xxx",
      "createdOn": "2020-04-03T11:26:08.7230604Z",
      "updatedBy": null,
      "updatedOn": null
    },
    "policyRule": {
     "if": {
       "allOf": [
         {
           "anyOf": [
             {
               "field": "tags",
               "notContainsKey": "terraTest2"
             },
             {
               "field": "tags['terraTest2']",
               "notIn": "[parameters('allowedTerraTest2')]"
             }
           ]
         },
         {
           "field": "type",
           "notIn": "[parameters('listOfResourceTypesAllowed')]"
         },
         {
           "notIn": "[parameters('listOfAllowedResourceGroups')]",
           "value": "[resourcegroup().name]"
         }
       ]
     },
     "then": {
       "effect": "deny"
     }
   },
   "parameters": {
     "allowedTerraTest2": {
       "type": "Array",
       "metadata": {
         "displayName": "Allowed Terratest Values",
         "description": "The list of allowed Terratest Values."
       },
       "allowedValues": [
         "True",
         "False"
       ]
     },
     "listOfAllowedResourceGroups": {
       "type": "Array",
       "metadata": {
         "displayName": "Allowed resource groups",
         "description": "The list of resource groups that can be deployed."
       },
       "defaultValue": [
         "terraTest2-terratest2-rg"
       ]
     },
     "listOfResourceTypesAllowed": {
       "type": "Array",
       "metadata": {
         "displayName": "Allowed resource types",
         "description": "The list of resource types that can be deployed.",
         "strongType": "resourceTypes"
       },
       "defaultValue": [
         "Microsoft.Compute/virtualMachines/extensions"
       ]
     }
   },
  "id": "/subscriptions/xxx-xxx-xxx-xxx-xxx/providers/Microsoft.Authorization/policyDefinitions/require-terraTest2-tag",
  "type": "Microsoft.Authorization/policyDefinitions",
  "name": "require-terraTest2-tag"
}

In my test I just duplicate this Policy as require-terraTest-tag, require-terraTest2-tag and require-terraTest3-tag

Then to reference the resource I use data "azurerm_policy_definition"

data "azurerm_policy_definition" "d_policy_definitions" {
    count = 3
    display_name = "require-terraTest${count.index}-tag"
}

I found a workaround

Ideally the solution should be as simple as:

merge(local.policy_definitions.*.parameters...)

But I manage to find a workaround. (I do not claim this to be an optimized code)

policy_parameters = [
    for key,value in data.azurerm_policy_definition.d_policy_definitions:
      {
        parameters = jsondecode(value.parameters)
      }
  ]
  ph_parameters = local.policy_parameters[*].parameters
  input_parameter = [for item in local.ph_parameters: merge(item,local.ph_parameters...)][0]

Break down:

1. Extracts the parameter values into a list of JSON values

policy_parameters = [
    for key,value in data.azurerm_policy_definition.d_policy_definitions:
      {
        parameters = jsondecode(value.parameters)
      }
  ]

1. Reference the parameters as a variable

ph_parameters = local.policy_parameters[*].parameters

1. Merge all item content into each item.

input_parameter = [for item in local.ph_parameters: merge(item,local.ph_parameters...)]

The 3rd step gives all items in the list the same value, so we can use any index.

Usage:

    parameters = "${jsonencode(local.input_parameter[n])}"

I think this workaround is wasteful. Imagine having 200 indices with the same value.

[danieldreier]

Thanks for explaining more. After thinking about it, and reading your explanation, this looks to me like a proposal for new functionality, rather than than a bug report. I'm going to categorize this as an enhancement request. We heavily weight community input in the form of 👍 votes, so if other people read this and would find this useful, please upvote it.