Terraform-provider-azurerm: AzureRm resource providers verification requires it to be enabled even when specific resource is not used anywhere.

[centur]

Last week we updated our azurerm provider from 1.19 to 1.31 (latest version at that moment) and noticed a strange behavior: this provider is now requires to have all corresponding resource providers enabled in Azure AD, otherwise it won't work correctly. We traced back the issue to this file required_resource_providers.go where all providers are listed in requiredResourceProviders.
We haven't had all resourceProviders enabled in our Azure AD tenant on production as, from security perspective, we want to reduce a potential attack surface.

I might be wrong, but I think this is an incorrect behavior when azurerm terraform provider requires all azure resourceProviders to be enabled, even those which are not used in any user scripts. Also it introduces a breaking behavior for any CI which uses terraform in one of the steps - any _addition_ to the requiredResourceProviders list in a newer minor version of azurerm provider would break the builds that were successfully ran before if corresponding resourceProvider is not enabled\registered in a specific subscription. As well as requiring that all providers unnecessary increases attack surface for Azure tenants.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform: v0.12.5
azurerm :

provider "azurerm" {
  version         = "~> 1.31"
  subscription_id = "GUID-HERE"
}

Affected Resource(s)

• azurerm provider itself

Expected Behavior

azurerm provider must validate only providers that are actually used in the .tf files, not all of the providers it support. terraform plan must successfully run if all resources that are actually used in the script are allowed in corresponding subscription.

Actual Behavior

terraform plan produces an error:

Error: Error ensuring Resource Providers are registered: Cannot register provider Microsoft.Maps with Azure Resource Manager: resources.ProvidersClient#Register: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'GUID-HERE' with object id 'GUID-HERE' does not have authorization to perform action 'Microsoft.Maps/register/action' over scope '/subscriptions/GUID-HERE' or the scope is invalid. If access was recently granted, please refresh your credentials.".

The error itself is a bit confusing and doesn't talk about Azure providers registration, but this is how Azure API communicate back the lack of registered provider - everything is permission error.

Steps to Reproduce

Use with any terraform script that is not using azurerm_maps_account resource

1. invoke az provider unregister --namespace Microsoft.Maps to remove a specific resource provider in your Azure subscription.

2. Run terraform plan - see an error from above

3. To restore terraform working again - call az provider register --namespace Microsoft.Maps to register back the resource provider

[katbyte]

Hi @centur,

We have always tried to register all resource providers required, however until a couple version ago would silently ignore any errors. Whilst it would be ideal to only register the ones required (or allow users to specify a list of providers to register) there are currently no plans to do so.

However you can disable this behaviour by setting the skip_provider_registration flag to true either in the provider block or as an environment variable.

Hope this helps

[centur]

Thanks for the explanation @katbyte,
It feels a bit strange that this was selected as an intended behaviour. Enabling additional providers may be quite complicated in enterprise environments, where security practices are "defensive". It'd be hard to reason about terraform adoption in any security-conscious organisation, as the dialogue would look like this:
"Hello, security team, we want to use azurerm with terraform, but Azure subscription admins need to enable these irrelevant providers, so we can use it. Also, every time when we update to the latest azurerm - we need to ensure things are not broken when Azure adds a support for another provider, so it's developers (to ensure that a minor version upgrade doesn't break builds) and security's (to enable all new providers) ongoing task".

I'd say it's a common sense to have only providers that are actually used in subscription to be enabled, not the other way around. Same rule as anyone would apply to e.g. opening ports - "close all, open only the ports that are actually needed". So a similar rule - "disable all providers that are not used or not planned to be used" - makes more sense to me, than the existing expectation that all providers are enabled.

I have no problem with providers registration validation, I think it's a right way to do it.
But I have a feeling that validation must be done not over a hard-coded list but using an actual resources from user's *.tf scripts.
E.g. when I will add a resource that is using azurerm_maps_account - it'd be great if azurerm would hint me that this provider is missing from Azure AD registration, current behavior with hardcoded list still looks like a bug to me.

[katbyte]

@centur,

Ideally that would be the best solution, however right now i don't believe we have and easy way in the provider to enumerate over all resources used in a terraform configuration. Alternatively we could potentially allow a list to be passed in, but that is not on our current roadmap. So we default to what will enable most users to start using the provider, and advanced users who don't wish to enable any more resource providers then required the skip_provider_registration will disable this behaviour.

[simonclausen]

Does it make sense to include an example in the documentation of the provider, mentioning the "defensive security" case @centur outlines, and the skip_provider_registration workaround?

[tombuildsstuff]

@simonclausen @centur after chatting about this internally the other day we're going to add a section to the website explaining why this exists and how you can opt out of it if you need to.

To give a little more context for the _why_ this exists; if you attempt to call an API method on a Resource Provider which isn't registered Azure returns an error message along the lines of:

API version `2019-01-01` was not found for Resource Provider `Microsoft.Foo`

.. which isn't overly helpful to folks unfamiliar with Azure - however once the Resource Provider is registered the error messages should be clearer. As such this is primarily a user experience feature to help ensure useful errors are returned - but I can see the desire to disable this for Resource Providers which aren't registered.

Unfortunately Terraform Core doesn't expose a list of Resources which are being used, as such it's not possible for us to only register the Resource Providers which are needed - which is why we attempt to register everything we support.

Instead in order to better explain this we're going to add a section to this page documenting this behaviour and how you can opt out of it if you're in a more locked down environment.

Thanks!

[terminatingcode]

@tombuildsstuff despite using the workaround to skip_provider_registration we still require permissions for unused features. Is there a possibility of opting out of checking for the existence of new features? We use default Azure roles and permissions and for security reasons do not want to open them up any more than for what we use.

[tombuildsstuff]

@terminatingcode unfortunately not due to the way this is implemented in Azure; that said, this should work with the Contributor/ReadWrite permissions on the Resource Group where the Storage Account is being provisioned - it could be the role needs re-applying to apply the new child-role available here?

[bmpotter13]

I think the error generated for failing to register a provider is more misleading than reporting that a provider wasn't found. I ran into this situation today, confused why creating Azure Key Vaults were failing because Microsoft.CognitiveServices wasn't enabled in my production subscription.

I understand that the provider can't know which resources it'll be creating. But, could the provider instead interrogate a returned error to see if it matches the "provider not found" error. Then, print a hint to check if that provider is registered? Or similarly, print a hint that the check can be disabled?

[hashibot[bot]]

This has been released in version 1.33.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 1.33.0"
}
# ... other configuration ...

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!