Terraform-provider-azurerm: azurerm_policy_assignment policy_definition_id fails validation in 2.5, expects scope to be prepended

[anlutro]

This Terraform code works fine in 2.5 and older:

data "azurerm_policy_definition" "allowed_locations" {
  display_name = "Allowed locations"
}
resource "azurerm_policy_assignment" "allowed_locations" {
  name                 = "allowed_locations-policy-assignment"
  scope                = "/subscriptions/${var.subscription_id}"
  policy_definition_id = data.azurerm_policy_definition.allowed_locations.id
  description          = "Allowed locations"
  display_name         = "Allowed locations"
  parameters = <<PARAMETERS
{
  "listOfAllowedLocations": {
    "value": [ "West Europe", "North Europe", "UK South", "UK West"]
  }
}
PARAMETERS
}

However, in 2.6 it fails with validation errors:

Error: cannot parse "policy_definition_id" as a Policy Definition ID: unable to parse Policy Definition ID "/providers/Microsoft.Authorization/policyDefinitions/e56962a6-4747-49cd-b67b-bf8b01975c4c": unable to parse Remediation Scope ID: ID is empty

  on az_policy_location.tf line 5, in resource "azurerm_policy_assignment" "allowed_locations":
   5: resource "azurerm_policy_assignment" "allowed_locations" {

Error: cannot parse "/providers/Microsoft.Authorization/policyDefinitions/e56962a6-4747-49cd-b67b-bf8b01975c4c" as a Policy Set Definition ID: unable to parse Policy Set Definition ID "/providers/Microsoft.Authorization/policyDefinitions/e56962a6-4747-49cd-b67b-bf8b01975c4c"

  on az_policy_location.tf line 5, in resource "azurerm_policy_assignment" "allowed_locations":
   5: resource "azurerm_policy_assignment" "allowed_locations" {

"unable to parse Policy Definition ID "/providers/Microsoft.Authorization/policyDefinitions/e56962a6-4747-49cd-b67b-bf8b01975c4c": unable to parse Remediation Scope ID: ID is empty" is the relevant error message here.

I tracked down a change in #6305 which adds validation to the field:

https://github.com/terraform-providers/terraform-provider-azurerm/blob/0488b7a3a097f4ea6dbd884e1395d88ede26e0c9/azurerm/internal/services/policy/resource_arm_policy_assignment.go#L62-L65

Reading up on the code that actually does this validation, I think this is what's causing the issue:

https://github.com/terraform-providers/terraform-provider-azurerm/blob/b7ae3e23f5f8c64a6c6f4da52421dbc34f46cdd9/azurerm/internal/services/policy/parse/definition.go#L15-L17

This expects there to be a {scope} part that occurs before "/providers/Microsoft.Authorization...", and the name of the policy to occur after. The name is present ("e56962a6-4747-49cd-b67b-bf8b01975c4c") but the scope is not.

It seems like we're expected to duplicate what is already passed in as scope and prepend it to policy_definition_id. We can do this as a workaround, but is this intented behavior?

[kellmv1]

Experiencing a similar issue after upgrading from azurem provider version 2.5 to 2.6
When setting the policy_definition_id, it appears it needs to be prepended with "/subscriptions/<id>"

The below code works with azurerm 2.5

provider "azurerm" {
  version = "=2.5.0"
  features {}
}
data "azurerm_subscription" "current" {}

resource "azurerm_policy_assignment" "example" {
  name                 = var.name
  scope                = data.azurerm_subscription.current.id
  policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/aa633080-8b72-40c4-a2d7-d00c03e80bed"
}

Workaround for 2.6 below.

provider "azurerm" {
  version = "=2.6.0"
  features {}
}
data "azurerm_subscription" "current" {}

resource "azurerm_policy_assignment" "example" {
  name                 = var.name
  scope                = data.azurerm_subscription.current.id
  policy_definition_id = "/subscriptions/<id>/providers/Microsoft.Authorization/policyDefinitions/aa633080-8b72-40c4-a2d7-d00c03e80bed"
}

[enorlando]

Also experiencing similar issue after upgrading azurerm provider from v2.5 to v2.6 against all azurerm_policy_assignment resources. Error below

Error: cannot parse "/providers/Microsoft.Authorization/policyDefinitions/ea3f2387-9b95-492a-a190-fcdc54f7b070" as a Policy Set Definition ID: unable to parse Policy Set Definition ID "/providers/Microsoft.Authorization/policyDefinitions/ea3f2387-9b95-492a-a190-fcdc54f7b070"

[jarmbruster74]

I'm seeing the same issue.

Error: cannot parse "policy_definition_id" as a Policy Definition ID: unable to parse Policy Definition ID "/providers/Microsoft.Authorization/policySetDefinitions/1a5bb27d-173f-493e-9568-eb56638dde4d"

[ArcturusZhang]

Sorry about this.... This bug should be introduced by me, will have a look at this issue and possibly provide a fix for this issue.

A possible workaround is that you can just remove the validation for policy_definition_id in azurerm_policy_assignment resource.

Sorry again for the inconvenience!

[ArcturusZhang]

And some explanation for this behavior:
This is not asking you to provide the scope of the policy definition again, the error actually comes from the parsing function of policy definition. A custom policy definition ID will be in form of {scope}/providers/Microsoft.Authorization/policySetDefinitions/{name} where scope can be a subscription or mgmt group. In other words, a custom policy definition ID should be like
/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/policySetDefinitions/policy1
or
/providers/Microsoft.Management/managementgroups/{group_name}/providers/Microsoft.Authorization/policyDefinitions/policy1

But there is a flaw that I overlooked when writing the parsing function -- the built-in policy definitions actually do not have a scope part. Therefore when you are using a built-in policies the validation failure will pop out...

[hashibot[bot]]

This has been released in version 2.7.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.7.0"
}
# ... other configuration ...

[jarmbruster74]

It looks to me like the same logic needs to be applied to the policySet parsing - https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/azurerm/internal/services/policy/parse/set_definition.go

On the Policy Set I'm seeing - I'm getting cannot parse "/providers/Microsoft.Authorization/policySetDefinitions/1a5bb27d-173f-493e-9568-eb56638dde4d" as a Policy Set Definition ID: unable to parse Policy Set Definition ID "/providers/Microsoft.Authorization/policySetDefinitions/1a5bb27d-173f-493e-9568-eb56638dde4d": unable to parse Remediation Scope ID: ID is empty

[kellmv1]

@jarmbruster74 - aye, I'm still getting the same error.

[jarmbruster74]

@ArcturusZhang - would it be possible to add the same logic for the policy set defintion? Should we open a new issue?

[ArcturusZhang]

@ArcturusZhang - would it be possible to add the same logic for the policy set defintion? Should we open a new issue?

Oh I think a new issue works for me, since this has been closed. I will get it fixed as soon as possible...

[ArcturusZhang]

@ArcturusZhang - would it be possible to add the same logic for the policy set defintion? Should we open a new issue?

In the meantime, could you please provide some more context about the policy set that has issues here? Is it a built-in policy set? If so what is the display name of it? I am thinking about adding a new test case to cover this scenario

[jarmbruster74]

The policyset that's failing validation is a built in policyset. I'll create a new issue. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!