Terraform-provider-azurerm: 403 on AKS creation

[florianrusch]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.11.14
+ provider.azuread v0.3.1
+ provider.azurerm v1.26.0 - v1.28.0

Affected Resource(s)

• azurerm_kubernetes_cluster

Terraform Configuration Files

provider "azurerm" {
  version = "1.26.0"
}

provider "azuread" {
  version = "0.3.1"
}

resource "azuread_application" "sp" {
  name                       = "${var.cluster_name}"
  homepage                   = "https://${var.cluster_name}"
  available_to_other_tenants = false
  oauth2_allow_implicit_flow = false
}
resource "azuread_service_principal" "sp" {
  application_id = "${azuread_application.sp.application_id}"
}
resource "azuread_service_principal_password" "sp" {
  service_principal_id = "${azuread_service_principal.sp.id}"
  value                = "${var.client_secret}"
  end_date             = "${var.end_date}"
}

resource "azurerm_kubernetes_cluster" "k8s" {
  name                = "${var.cluster_name}"
  location            = "${var.location}"
  resource_group_name = "${var.resource_group_name}"
  dns_prefix          = "${var.cluster_name}-dns"
  agent_pool_profile {
    name           = "default"
    count          = "${var.agent_pool_count}"
    vm_size        = "${var.agent_size}"
  }
  service_principal {
    client_id     = "${azuread_application.sp.application_id}"
    client_secret = "${var.client_secret}"
  }
}

Debug Output

https://gist.github.com/florianrusch/d82608d497bca2a2c05928d42f8810b0

Actual Behavior

Since the azurerm version v1.26.0 I'm getting problems on provisioning a kubernetes cluster at azure. I'm getting two different error messages if I try it a view times to plan:

First error message:

* provider.azurerm: Error ensuring Resource Providers are registered: Cannot register provider Microsoft.HDInsight with Azure Resource Manager: resources.ProvidersClient#Register: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '<USER-EMAIL>' with object id '<USER-ID>' does not have authorization to perform action 'Microsoft.HDInsight/register/action' over scope '/subscriptions/<SUBSCRIPTION-ID>'.".

Second error message:

* provider.azurerm: Error ensuring Resource Providers are registered: Cannot register provider Microsoft.StreamAnalytics with Azure Resource Manager: resources.ProvidersClient#Register: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '<USER-EMAIL>' with object id '<USER-ID>' does not have authorization to perform action 'Microsoft.StreamAnalytics/register/action' over scope '/subscriptions/<SUBSCRIPTION-ID>'.".

If I'm going back to version v1.25.0 everything is working fine. The problem also appears in newer version of azurerm.

Questions

1. Are there breaking changes which are not documented in the Changelog.md?
2. Why does I need to create StreamAnalytics and HDInsights? Even if I'm not enabling the oms_agent in the addon_profile block.

Steps to Reproduce

1. terraform init
2. terraform plan

[tombuildsstuff]

hi @florianrusch

Thanks for opening this issue :)

Azure's concept of a Resource Provider requires that the Resource Provider is registered before any API's within that Resource Provider can be called. As such, rather than failing during an API call / to improve the user experience Terraform automatically attempts to register any Resource Providers that it supports by default - to ensure that the API's are available when they're needed.

If you'd prefer to manage Resource Provider Registration outside of Terraform (for example you're running in a restricted environment) - it's possible to opt-out of this behaviour by setting this field in the Provider block.

To answer your questions specifically:

Are there breaking changes which are not documented in the Changelog.md?

Nope - the behaviour here is to register any Resource Providers which it supports (and require registration) - as such adding another item to this list isn't a breaking change per-se. As mentioned above if you're running in a more restrictive environment, it's possible to opt-out of automatic Resource Provider registration.

Why does I need to create StreamAnalytics and HDInsights? Even if I'm not enabling the oms_agent in the addon_profile block.

Unfortunately it's not possible for us to determine which Resource Providers would be required based on the users configuration ahead of time; whilst one option would be to register these on demand as they're required - Azure sometimes returns bad (400, 500) error codes for operations on these endpoints whilst the Resource Providers are being registered; as such whilst this'd work if Resource Provider Registration was synchronous, should one be being registered whilst another is trying to use it, this'd cause an error (and as such is an approach we can't use).

Since this is working as intended I'm going to close this issue for the moment, but please let us know if that doesn't work for you and we'll take another look.

Thanks!

[florianrusch]

@tombuildsstuff thank you for your answer! That explains a lot.

So, if I would like to use terraform to create anything on azure, I have to be sure, that I have the right in our subscription for every resource?

[tombuildsstuff]

@florianrusch you need permission to register Resource Providers (which comes with Contributor, for example, but there may be a specific/more limited permission you can use) which is per subscription

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!