Terraform-provider-azurerm: Azure Management Group creation with Service Principal returns 403

[julioas09]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform version: 0.12.20
Azurerm version: 2.0.0

Affected Resource(s)

• azurerm_management_group

We use a Service Principal to connect to out Azure environment. This SP has Owner role at Root Management Group. If we login to Azure CLI with this SP, we can manage Management Groups without a problem.
When we try to run from terraform, we get a 403 error:

Error: Error checking for presence of existing Management Group "d45bbbf8-72bd-486b-ac78-73e07c2e4043": managementgroups.Client#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'f0e7a3ad-21cb-4968-aea2-b3ef5ec930d4' with object id 'f0e7a3ad-21cb-4968-aea2-b3ef5ec930d4' does not have authorization to perform action 'Microsoft.Management/managementGroups/read' over scope '/providers/Microsoft.Management/managementGroups/d45bbbf8-72bd-486b-ac78-73e07c2e4043' or the scope is invalid. If access was recently granted, please refresh your credentials." 

Terraform Configuration Files

provider "azurerm" {
  version = "=2.0.0"
  features {}
}
resource "azurerm_management_group" "ccoe" {
  display_name = "azure-ccoe"
}

Debug Output

Provided in Gist

Expected Behavior

Management Group is created

Actual Behavior

Terraform apply fails with error 403 forbidden

Steps to Reproduce

1. Assign service principal as owner of Root Management Group
2. Set proper local env variables to connect with SP:
   export ARM_SUBSCRIPTION_ID="xxxxxx"
   export ARM_CLIENT_ID="xxxxxx"
   export ARM_CLIENT_SECRET="xxxxx"
   export ARM_TENANT_ID="xxxxx"
3. terraform init
4. terraform apply

[dhoeld]

The problem also appears if you use a user principal, not only with a service principal.
The same code runs with provider version 1.44.0. It seems like a bug introduced with the new terraform provider in version 2.

[pborgo]

I'm experiencing the same issue with v2.3.0. Is there any update on this? thx

[dkooll]

same experience here with v2.4.0.

[ChrisF987]

same here on v2.6.0

[markrevill]

Hi guys,

Problem is still occuring in the version 2.7.0 of the AzureRM provider. Pinning to version 1.44 resolves the issue. Using Service Principal secret authentication.

2.7.0:

Error: unable to check for presence of existing Management Group "3be387eb-cf0c-4a6f-af01-28f7bea69495": managementgroups.Client#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '9b00f19a-9d9b-4756-88ee-cee0bd7de522' with object id '9b00f19a-9d9b-4756-88ee-cee0bd7de522' does not have authorization to perform action 'Microsoft.Management/managementGroups/read' over scope '/providers/Microsoft.Management/managementGroups/3be387eb-cf0c-4a6f-af01-28f7bea69495' or the scope is invalid. If access was recently granted, please refresh your credentials."

1.44:

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

azurerm_management_group.module: Creating...
azurerm_management_group.module: Still creating... [10s elapsed]
azurerm_management_group.module: Still creating... [20s elapsed]
azurerm_management_group.module: Still creating... [30s elapsed]
azurerm_management_group.module: Creation complete after 31s [id=/providers/Microsoft.Management/managementGroups/df3a10a8-4ff4-4c42-89da-2f5302c12603]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Hoping to get some traction on this issue.

Cheers,
Mark.

[wsf11]

Hi all,

I am currently working on a fix for this issue. The AzureRM provider first runs a GET on the management group you requested to create, to ensure it doesn't exist. The problem occurs when you run a GET on a management group that either doesn't exist, or you don't have access to. It returns with the same 403 Authorization error.

If you are trying to just run a GET on a management group resource, make sure that the User you're authenticating with has proper access. Proper access would be the Management Group Reader role on the Management Group scope, or the Tenant Root Group scope.

[boillodmanuel]

I was debugging the error, when I find this issue.

Before I get this error, I was using version 2.1.0. Now, I'm using the version 2.6.0

I suppose that the regression is due to this pull-request: https://github.com/terraform-providers/terraform-provider-azurerm/pull/6276, released in 2.4.0

@wsf11 , I confirm your analyze. In my case, I have proper access but the management group is new and it fails with Error: unable to check for presence of existing Management Group

Looking forward for the fix,

Thanks

[wsf11]

I was debugging the error, when I find this issue.

Before I get this error, I was using version 2.1.0. Now, I'm using the version 2.6.0

I suppose that the regression is due to this pull-request: #6276, released in 2.4.0

@wsf11 , I confirm your analyze. In my case, I have proper access but the management group is new and it fails with Error: unable to check for presence of existing Management Group

Looking forward for the fix,

Thanks

@boillodmanuel Did you get a 403 or 404 error?

[boillodmanuel]

@wsf11 , It's a 403 error as you can see:

    error: unable to check for presence of existing Management Group "xxxx": managementgroups.Client#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'xxxx' with object id 'xxxx' does not have authorization to perform action 'Microsoft.Management/managementGroups/read' over scope '/providers/Microsoft.Management/managementGroups/xxxx' or the scope is invalid. If access was recently granted, please refresh your credentials."

But, I did a mistake. I tested again and the bug was already there in version 2.1.0. But wasn't here in version 1.3.1 (to the regression is not due to #6276). Sorry

[ArcturusZhang]

Actually in my PR #6276 , I introduced a new bug here.
This bug actually blocks you from assigning name (you will always get a mgmt group with UUID), but I suppose this should be independent from the 403 issue here.

I am planning fix the bug I introduced along with #6757 , but as for this 403 issue, I actually have no clue....

[wsf11]

Actually in my PR #6276 , I introduced a new bug here.
This bug actually blocks you from assigning name (you will always get a mgmt group with UUID), but I suppose this should be independent from the 403 issue here.

I am planning fix the bug I introduced along with #6757 , but as for this 403 issue, I actually have no clue....

I have fixed the bug introduced in PR #6276 in my PR mentioned above. As well as the 403 issue

[mdob88]

Hello @wsf11
When are you able to finalize this #6668 PR and release new version?

[wsf11]

@mdob88 Waiting review...

[petr-stupka]

well done @wsf11! Works for me

[tombuildsstuff]

Fixed via #6668

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!