Terraform-provider-azurerm: Features being required in V2 breaks V1 usage with unpinned providers

[alastairtree]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Features has become a required field in version 2 of the azurerm provider:
https://www.terraform.io/docs/providers/azurerm/index.html#features

If you are not pinning your provider version (and yes we should have been!) then making this field required breaks all existing v1.x terraform deployments. However, it seems that all the "features" properties are actually optional (key_vault, virtual_machine, virtual_machine_scale_set ) and there must be many people who would not use any of them beyond the sensible defaults.
See https://www.terraform.io/docs/providers/azurerm/index.html#features-2

If they are all optional why not make the features property optional as well, and then this change would not break all users with v1 code?

[drdamour]

i just came here to report this same breakage. But i think the ticket doesn't complain enough. In the docs

https://www.terraform.io/docs/configuration/terraform.html#specifying-required-provider-versions says

Re-usable modules should constrain only the minimum allowed version, such as >= 1.0.0. This specifies the earliest version that the module is compatible with while leaving the user of the module flexibility to upgrade to newer versions of the provider without altering the module.

so in some cases you are supposed to NOT lock it down.

regardless, we tf init and tf validate our reusable modules during their CD pipeline, and they all just broke today because features is now required. We pessimistic locked them to work around this, but even when we go to adopt 2.0 this is gonna be an issue.

How can a reusable module (that should NOT have a provider block) now run a tf init and tf validate with this new required block? Do we have to have another test only tf file that specifies we want no features and bring that in only when tf init and tf validate our module?

[DSakura207]

I think the document should be revised.
According to https://www.terraform.io/docs/extend/best-practices/versioning.html, providers are recommended to adopt semantic versioning. Therefore, a re-usable module should lock major versions of its dependencies, to protect user from upstream breaking changes.

[djsly]

we have been hit by the same issue, ALL of our terraform modules runs

terraform init
terraform validate

Right now we are getting CI pipeline failures

Error: "features": required field is not set

I updated all of our modules by creating an empty provide block with with only the features{} and this gets me through the terraform validate command

The problem is after this when I run the terraform code which instantiate this module, I now get

Error: Error building AzureRM Client: Azure CLI Authorization Profile was not found. Please ensure the Azure CLI is installed and then log-in with `az login`.


  on .terraform/modules/sly-aks-dev.aks/providers.tf line 1, in provider "azurerm":
   1: provider "azurerm" {

How can we go about fixing this...

More importantly, why would the terraform validate command expects the features arguments... this is where the flaw is on my opinion.

Seems like a HUGE regression on the Terraform core logic with modules.

For now we are forced to disable all CI validation of the terraform code.

[djsly]

this is related to : https://github.com/hashicorp/terraform/issues/21408

With no real fix from terraform, they ask that users set the AWS_PROVIDER_REGION env to have terraform validate pick it up while keeping the module empty form any provider block definition.

Maybe we should set a default features {} or allow setting it with an ENV

[djsly]

By looking at the code I was able find a loophole...

❯ terraform validate

Error: "features": required field is not set

❯ TF_ACC=true terraform validate
Success! The configuration is valid.

https://github.com/terraform-providers/terraform-provider-azurerm/blob/b6c410e6dbe56c1401296480ee69763a5331554f/azurerm/internal/provider/features.go#L62-L71

not super elegant BUT at least is fixes terraform validate

[drdamour]

@djsly nice find and i agree the proposed fix for https://github.com/hashicorp/terraform/issues/21408 would fix this too

[djsly]

@drdamour so @tombuildsstuff busted my bubble this morning :( he mentioned that the TF_ACC is a hack that will be removed shortly...

I'm voting for an official ARM_FEATURES ENV variable to be created which will allow users to set the feature settings programatically before doing validates. Or even better, that the features setting becomes optional

A workaround needs to exists until https://github.com/hashicorp/terraform/issues/21408 is fully resolved.

[drdamour]

Hack should remain until the actual solution is implemented, whatever that may be

[DSakura207]

I vote for marking features optional.

Besides, I've got another question:
Did the behaviors controlled by features change from 1.x to 2.0?

If yes, I would say features can be mandatory, even if all blocks inside it are optional. It does not seem to be a good way to encourage people visiting document, but it reminds people for the breaking changes - in some way.

If no, I would say features should be optional. Nothing has changed ever, then why we have to change a module?

[tombuildsstuff]

👋 hi folks

As mentioned in the changelog, version 2.0 of the Azure Provider now requires that a provider block be defined and a features block be defined within it. The features block is new in version 2.0 of the Azure Provider and allows controlling of the behaviour of certain resources (including whether data disks/key vaults should be purged).

Since this controls the behaviour of the Provider this block is now required as we believe it's important that folks are aware which Provider block is being used, since this was ambiguous as it was optional in 1.x (and thus we often saw confusion where users were using one provider block when they thought they were using another with modules).

As mentioned above this should be a case of updating your Terraform Configurations to include a Provider block as shown below:

provider "azurerm" {
  version = "=2.0.0"
  features {}
}

When working with Modules you can define the Provider blocks at the top level and then pass them into the module like so:

provider "azurerm" {
  # ...
  features {}
}

module "test" {
  source = "../"
  providers = {
    "azurerm.default" = "azurerm"
  }
}

5880 is tracking adding some additional documentation for this however since this should be fixed by updating the Terraform Configuration to include the provider block I'm going to close this issue for the moment, since this is by design in version 2.0 (and upwards) of the Azure Provider.

---

@djsly in the scenario you've mentioned above it's possible to achieve this by using a test folder to init the root module to confirm it works as users will use it, for example:

# file: ./main.tf
resource "azurerm_resource_group" "test" {
  # ...
}

# file: ./test/main.tf
provider "azurerm" {
  version = "=2.0.0"
  # ...
  features {}
}

module "test" {
  source = "../"
  providers = {
    "azurerm.default" = "azurerm"
  }
}

Whilst I appreciate this is a breaking change to your workflow, defining provider blocks in your code is terraform best practice and version 2.0 of the Azure Provider now requires this. When using modules these are required to be defined at the top level and then passed in (as shown above).

As you've mentioned this unintentionally worked before when using a single provider block however this wasn't intentional and won't be supported going forward - as such you'll need to define the provider block and pass this in here (as shown above).

Since the features block defines the behaviour of the provider being used (and we're intentionally requiring a provider block to be specified) unfortunately we have no plans to add an environment variable for ARM_FEATURES since this'd hide this information. Instead it should be possible to work around this by defining a provider block in a test folder and passing it as shown above, which whilst I appreciate involves updating your codebase to account for this as this is now required as of version 2.0 of the Azure Provider.

It's worth noting the workaround mentioned above will be removed in the near future since this isn't intended to be used outside of the test framework. Instead we'll be updating the test suite (and examples/docs) to include provider blocks in the near future, this was a stop-gap so that we could decouple shipping version 2.0 of the Azure Provider and making those changes.

In the interim you should be able to work around this by pinning the version of the Azure Provider being used to a 1.x release to continue using this as before.

Thanks!

[drdamour]

I cant make heads or tails of what your test example is trying to say...can you link to an actual example? Are you saying i have to create a tf file for testing my module?

[drdamour]

Also the doc unlink says modules shouldnt have a providers block

Provider version constraints can also be specified using a version argument within a provider block, but that simultaneously declares a new provider configuration that may cause problems particularly when writing shared modules. For that reason, we recommend using the required_providers block as described above, and not using the version argument within provider blocks. version is still supported for compatibility with older Terraform versions.

And

Modules have some special requirements when passing in providers; see Providers within Modules for more details. In most cases, only root modules should define provider configurations, with all child modules obtaining their provider configurations from their parents.

[djsly]

I'm with you @drdamour , something isn't clean with the proposed solution. all this because we are using a provider that decided to not make one field optional ;)

[tombuildsstuff]

@drdamour

I cant make heads or tails of what your test example is trying to say...can you link to an actual example? Are you saying i have to create a tf file for testing my module?

You can see a structured example of how this works in this example module which contains a test folder calling into the root module, which is where the commands you're using above can be used to confirm this works as expected - which essentially mirrors how end-users use the module.

Since a "module" refers to any Terraform code, unfortunately this isn't overly clear - to clarify:

In most cases, only root modules should define provider configurations, with all child modules obtaining their provider configurations from their parents.

In this instance this is referring to the "root module" which is the "root" directory where Terraform is run which contains Terraform Configurations (which may provision resources directly or call into other modules). In this example linked above, the root directory would be the "test" directory since this is where Terraform's being run.

Hope that helps :)

[alastairtree]

@tombuildsstuff - first up thanks for all the hard work, the azurerm provider is great and we really appreciate it.

I understand your motivations, and your example of how to do testing for modules with v2 is helpful. However I think it is worth restating the original problem - by making the features block manual the 2.0 release has broken all azurerm 1.x code in the wild that was not pinning the provider (but I accept we should have been pinning). If you have a lot of terraformed apps thats a non-trivial amount of work.

I have had to have teams of people go and add an empty features block to all their root modules to fix otherwsise working code, and very few other changes were actually required by the accidental upgrade to 2.0 in our experience with an unpinned provider. It also grates a little bit because the block is empty and seems to serve no actual purpose other than forcing you to think about the upgrade.

Even if you did pin providers, just to do the upgrade you have to go round all your code and add the new empty features block - in most of our cases that was only change required. From our perspective this just feels like unneccesary work. This is not a great user experience is probably the main gripe.

[djsly]

FYI. The backdoor is officially gone with v2.1.0. All Hell just broke this PM again ;)

[djsly]

@tombuildsstuff FYI, I tried your proposed solution and I cannot get it to work

I have a module: tf-azure-network where I'm defining an empty provider block which makes the terraform validate call works without problem.

provider "azurerm" {
  features {}
}

when insensate the module in the root module.

provider "azurerm" {
  alias           = "prodk8sa-eastus"
  subscription_id = local.prod_eastus_subid
  client_id       = local.prod_client_id
  client_secret   = var.prod_client_secret
  tenant_id       = local.tenant_id
  environment     = local.environment
  features {}
}

module "prodk8sa-network" {
  providers = {
    azurerm.default = azurerm.prodk8sa-eastus
  }
  ...
}

Generates this error.

Error: Error building AzureRM Client: Azure CLI Authorization Profile was not found. Please ensure the Azure CLI is installed and then log-in with `az login`.

  on .terraform/modules/prodk8sa-network/provider.tf line 1, in provider "azurerm":
   1: provider "azurerm" {

Would yo have any insight how this should work ?

note that I also tried without default

module "prodk8sa-network" {
  providers = {
    azurerm = azurerm.prodk8sa-eastus
  }
  ...
}

[djsly]

if we set an alias in the provider block in the module

provider "azurerm" {
  alias = "default"
  features {}
}

we still get Error: "features": required field is not set during the validate command...

so it seems that your proposal to create a /test/provider.tf and /test/module.tf is the only way that I can run a terraform validate over a module that wants to use the _default_ implicit provider block for a module.

While you. might have your reasoning to enforce features {}, I not understanding why it wasn't possible to keep TF_ACC or create ARM_FEATURES. until the core issue gets fixed around the validate command. This would have prevented this /test/ folder to be forced in place over 80 different modules...

[drdamour]

@tombuildsstuff thanks for the example it makes a lot of sense now. I like that i now have a tf file in my test tree of my project now too..it was always kinda weird running tests from /src/tf/

[davidtom]

if we set an alias in the provider block in the module

provider "azurerm" {
  alias = "default"
  features {}
}

we still get Error: "features": required field is not set during the validate command...

so it seems that your proposal to create a /test/provider.tf and /test/module.tf is the only way that I can run a terraform validate over a module that wants to use the _default_ implicit provider block for a module.

While you. might have your reasoning to enforce features {}, I not understanding why it wasn't possible to keep TF_ACC or create ARM_FEATURES. until the core issue gets fixed around the validate command. This would have prevented this /test/ folder to be forced in place over 80 different modules...

This is breaking our module validation workflow as well and I agree with your points here and above. I fail to understand why features was made to be required if it is also able to be empty. It just seems like bad design which we'll now have to work around to get back to a workflow that was functioning just fine for us.

Combine that with the removal/omission of environment variables to ease the pain and I'm left scratching my head about this whole thing.

@tombuildsstuff we have a repository of ~20 modules whose CI pipeline runs a simple terraform init and terraform validate to catch any validation errors. We don't currently have the need or resources to implement complete use testing of these modules, as it seems your posted example is demonstrating. Given this, we're left with either going against best practice and specifying provider "azurerm" {...} blocks in each module just so we can give them an empty features block to pass this new requirement or overhauling our CI steps to start addressing the requirement another way.

I assume that making features optional is off the table at this point, but allowing it to be set with an environment variable like ARM_FEATURES as @djsly mentions above would save my team hours of work just to get back to our current state.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!