Terraform-provider-azurerm: azurerm_policy_set_definition always updates metadata in-place

[richeney]

Any terraform plan or terraform apply for a config including an existing azurerm_policy_set_definition resource will generate changes.

It appears that the REST API calls maintains a set of metadata properties:

• createdBy = oid
• createdOn = ISO8601 timestamp
• updatedBy = oid
• updatedOn - ISO8601 timestamp

Terraform is always settingll
these values back to null (or the whole object if no other metadata properties are specified), triggering an in-place update despite the config files remaining unchanged.

I assume that these four values should be ignored by a terraform plan or apply, and they should not be permitted in the azurerm_policy_set_definition block.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.0-rc1
+ provider.azuread v0.3.1
+ provider.azurerm v1.28.0

Affected Resource(s)

• azurerm_policy_set_definition

Terraform Configuration Files

variables.tf:

variable "tenantId" {
  type        = string
  description = "The tenantId, i.e. the tenant GUID (`az account show`)"
}

custom_policies.tf:

resource "azurerm_policy_definition" "auditemptytagvalue" {
  name         = "auditEmptyTagValue"
  display_name = "Audit tag exists and has a value"
  description  = "This policy audits that a tag exists and has a non-empty value."
  policy_type  = "Custom"
  mode         = "Indexed"

  management_group_id = var.tenantId

  parameters = <<PARAMETERS
    {
        "tagName": {
            "type": "String",
            "metadata": {
                "description": "Name of the tag, e.g. 'Environment'",
                "displayName": "Tag Name"
            }
        }
    }
PARAMETERS

  policy_rule = <<POLICY_RULE
    {
        "if": {
            "anyOf": [
                {
                    "exists": "false",
                    "field": "[concat('tags[', parameters('tagName'), ']')]"
                },
                {
                    "field": "[concat('tags[', parameters('tagName'), ']')]",
                    "match": ""
                }
            ]
        },
        "then": {
            "effect": "audit"
        }
    }
POLICY_RULE
}

resource "azurerm_policy_definition" "audittagvaluefromlist" {
name         = "auditTagValueFromList"
display_name = "Audit tag exists and has a value from the allowedList"
description  = "This policy audits that a tag exists and has a value from the specified list."
policy_type  = "Custom"
mode         = "Indexed"

management_group_id = var.tenantId

parameters = <<PARAMETERS
    {
        "tagName": {
            "metadata": {
                "description": "Name of the tag, such as 'costcode'",
                "displayName": "Tag Name"
            },
            "type": "String"
        },
        "tagValues": {
            "metadata": {
                "description": "The list of permitted tag values",
                "displayName": "Permitted Tag Values"
            },
            "type": "Array"
        }
    }
PARAMETERS

policy_rule = <<POLICY_RULE
    {
        "if": {
            "anyOf": [
                {
                    "field": "[concat('tags[', parameters('tagName'), ']')]",
                    "exists": "false"
                },
                {
                    "field": "[concat('tags[', parameters('tagName'), ']')]",
                    "notIn": "[parameters('tagValues')]"
                }
            ]
        },
        "then": {
            "effect": "audit"
        }
    }
POLICY_RULE
}

resource "azurerm_policy_definition" "audittagvaluepattern" {
  name         = "auditTagValuePattern"
  display_name = "Audit tag exists and that the value matches the pattern"
  description  = "This policy audits that a tag exists and has a value that matches the specified pattern."
  policy_type  = "Custom"
  mode         = "Indexed"

  management_group_id = var.tenantId

  parameters = <<PARAMETERS
    {
        "tagName": {
            "type": "String",
            "metadata": {
                "description": "Name of the tag, e.g. 'Costcode'",
                "displayName": "Tag Name"
            }
        },
        "tagValuePattern": {
            "type": "String",
            "metadata": {
                "description": "Pattern to use for names. Use ? for characters and # for numbers.",
                "displayName": "Tag Value Pattern"
            }
        }
    }
PARAMETERS

  policy_rule = <<POLICY_RULE
    {
        "if": {
            "not": {
                "field": "[concat('tags.', parameters('tagName'))]",
                "match": "[parameters('tagValuePattern')]"
            }
        },
        "then": {
            "effect": "audit"
        }
    }
POLICY_RULE
}

custom_initiatives.tf:

```hcl
locals {
allowedLocations = "/providers/Microsoft.Authorization/policyDefinitions/e56962a6-4747-49cd-b67b-bf8b01975c4c"
allowedVirtualMachineSkus = "/providers/Microsoft.Authorization/policyDefinitions/cccc23c7-8427-4f53-ad12-b6a63eb452b3"
appendTagAndItsDefaultValue = "/providers/Microsoft.Authorization/policyDefinitions/2a0e14a6-b0a6-4fab-991a-187a4f81c498"
requireTagAndItsValue = "/providers/Microsoft.Authorization/policyDefinitions/1e30110a-5ceb-460c-a204-c1c3969c6d62"
// tenantRootGroupCustomPolicy = "/providers/Microsoft.Management/managementgroups/${var.tenantId}/providers/Microsoft.Authorization/policyDefinitions"
}

resource "azurerm_policy_set_definition" "deny" {
name = "Deny"
policy_type = "Custom"
display_name = "Standard Deny Policy Initiative"
description = "Limit the permitted regions and virtual machine SKUs"

management_group_id = var.tenantId // Tenant Root Group

parameters = < {
"regions": {
"type": "Array",
"metadata": {
"displayName": "List of regions",
"description": "List of permitted region. Only permitted pairs as West / North Europe or UK South / West."
},
"defaultValue": [
"West Europe",
"North Europe"
],
"allowedValues": [
[
"West Europe",
"North Europe"
],
[
"UK South",
"UK West"
]
]
}
}
PARAMETERS

policy_definitions = < [
{
"comment": "Permitted regions",
"parameters": {
"listOfAllowedLocations": {
"value": "[parameters('regions')]"
}
},
"policyDefinitionId": "${local.allowedLocations}"
},
{
"comment": "Permitted VM SKUs. Non-compliant SKUs will be denied.",
"parameters": {
"listOfAllowedSKUs": {
"value": [
"Standard_B1ms",
"Standard_B1s",
"Standard_B2ms",
"Standard_B2s",
"Standard_B4ms",
"Standard_D2_v3",
"Standard_D2s_v3"
]
}
},
"policyDefinitionId": "${local.allowedVirtualMachineSkus}"
}
]
POLICY_DEFINITIONS

}

resource "azurerm_policy_set_definition" "tags" {
name = "Tags"
policy_type = "Custom"
display_name = "Standard Tagging Policy Initiative"

management_group_id = var.tenantId

parameters = < {
"Environment": {
"type": "String",
"metadata": {
"description": "Environment, from permitted list",
"displayName": "Environment"
},
"defaultValue": "Dev",
"allowedValues": [
"Prod",
"UAT",
"Test",
"Dev"
]
}
}
PARAMETERS

policy_definitions = < [
{
"comment": "Create Owner tag if it does not exist",
"parameters": {
"tagName": {
"value": "Owner"
},
"tagValue": {
"value": ""
}
},
"policyDefinitionId": "${local.appendTagAndItsDefaultValue}"
},
{
"comment": "Audit Owner tag if it is empty",
"parameters": {
"tagName": {
"value": "Owner"
}
},
"policyDefinitionId": "${azurerm_policy_definition.auditemptytagvalue.id}"
},
{
"comment": "Create Department tag if it does not exist",
"parameters": {
"tagName": {
"value": "Department"
},
"tagValue": {
"value": ""
}
},
"policyDefinitionId": "${local.appendTagAndItsDefaultValue}"
},
{
"comment": "Check if Department is in the defined list",
"parameters": {
"tagName": {
"value": "Department"
},
"tagValues": {
"value": [
"Finance",
"Human Resources",
"Logistics",
"Sales",
"IT"
]
}
},
"policyDefinitionId": "${azurerm_policy_definition.audittagvaluefromlist.id}"
},
{
"comment": "Create Application tag if it does not exist",
"parameters": {
"tagName": {
"value": "Application"
},
"tagValue": {
"value": ""
}
},
"policyDefinitionId": "${local.appendTagAndItsDefaultValue}"
},
{
"comment": "Audit Application tag if it is empty",
"parameters": {
"tagName": {
"value": "Application"
}
},
"policyDefinitionId": "${azurerm_policy_definition.auditemptytagvalue.id}"
},
{
"comment": "Create Environment tag with parameters value if it does not exist",
"parameters": {
"tagName": {
"value": "Environment"
},
"tagValue": {
"value": "[parameters('Environment')]"
}
},
"policyDefinitionId": "${local.appendTagAndItsDefaultValue}"
},
{
"comment": "Deny Environment tag if it isn't set to the parameter",
"parameters": {
"tagName": {
"value": "Environment"
},
"tagValue": {
"value": "[parameters('Environment')]"
}
},
"policyDefinitionId": "${local.requireTagAndItsValue}"
},
{
"comment": "Create Downtime tag if it does not exist, with default value",
"parameters": {
"tagName": {
"value": "Downtime"
},
"tagValue": {
"value": "Tuesday, 04:00-04:30"
}
},
"policyDefinitionId": "${local.appendTagAndItsDefaultValue}"
},
{
"comment": "Audit Downtime tag if it is empty",
"parameters": {
"tagName": {
"value": "Downtime"
}
},
"policyDefinitionId": "${azurerm_policy_definition.auditemptytagvalue.id}"
},
{
"comment": "Create Costcode tag if it does not exist",
"parameters": {
"tagName": {
"value": "Costcode"
},
"tagValue": {
"value": ""
}
},
"policyDefinitionId": "${local.appendTagAndItsDefaultValue}"
},
{
"comment": "Check that Costcode tag value is a six digit number",
"parameters": {
"tagName": {
"value": "Costcode"
},
"tagValuePattern": {
"value": "######"
}
},
"policyDefinitionId": "${azurerm_policy_definition.audittagvaluepattern.id}"
}
]
POLICY_DEFINITIONS

}
```

Other Terraform files exist in the config, but these are the key ones.

Debug Output

https://gist.github.com/richeney/ebdfe02626ec2ca4b940f3b9a8c5558e

Panic Output

Expected Behavior

Running terraform plan after a successful terraform apply should not plan changes if the terraform files are unmodified.

Actual Behavior

The metadata is nulled each time, but is actually updated by the Azure REST API provider.

Steps to Reproduce

1. terraform apply -auto-approve
2. terraform plan

Important Factoids

References

• #0000

[dneelyep]

This problem seems to add a lot of time to deployments and noise in apply logs. In my environment with ~59 custom policy definitions, terraform applys are taking ~10 minutes. Granted, the relevant repository contains other things than just policy definitions, but the policy definition appear to be a large contributing factor.

[mlushpenko]

Happy someone reported it already, but already 3 months without any reaction except giving it a label month ago :) but can imagine it's not a critical issue.

[markmqhr]

Having the exact same issue, where the policy metadata is causing a change. This is only affecting azurerm_policy_set_definition but works as expected for azurerm_policy_definition

[mwywong]

Good workaround in #5014

[mlushpenko]

Thanks @mwywong, my colleague found the same, I forgot to share it here :)

[richeney]

I'll pop it in here as well - credit to @yangdeal:

lifecycle {
    ignore_changes = [
        metadata
    ]
}

[richeney]

As this is low priority, has a workaround, and is duplicated with #5014 (which has a better title) then I suggest closing this one.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!