Terraform: Module provider constraints after v0.11.0

[dghubble]

Terraform Version

v0.11.0
v0.11.1

In v0.11.0 there was some provider/module/inheritance changes and now providers within modules says provider blocks should only appear in the root module. Providers should be passed to modules. This seems to break the idea that a module is a reusable chunk that can declare its own provider constraints which (the client's) top-level providers must satisfy.

Is this an intended side effect of the rearchitecture? Any advice?

In a module:

• Removing the provider block could allow users of a module to use old (broken) versions of providers
• Keeping the provider block seems to cause issues cleanly removing / deleting modules in Terraform v0.11.x (nice the problem is noted in upgrade notes, but still a problem)

Apply module, delete module (.tf). Plan or apply fails on one a (sub)provider (random):

$ terraform plan
configuration for module.somecluster.provider.tls is not present; a provider configuration block is required for all operations
$ terraform plan
configuration for module.somecluster.provider.template is not present; a provider configuration block is required for all operations

These errors don't make sense. Which provider block? Those providers don't need configuring.

Terraform Configuration Files

• https://github.com/poseidon/typhoon/blob/master/aws/container-linux/kubernetes/require.tf
• https://github.com/poseidon/typhoon/blob/master/bare-metal/container-linux/kubernetes/require.tf
• https://github.com/poseidon/typhoon/blob/master/digital-ocean/container-linux/kubernetes/require.tf
• https://github.com/poseidon/typhoon/blob/master/google-cloud/container-linux/kubernetes/require.tf

References

• https://github.com/hashicorp/terraform/issues/15762
• https://github.com/hashicorp/terraform/issues/16777

[dghubble]

cc @jbardin

[jbardin]

Hi @dghubble,

The errors you have there don't necessarily mean that the providers _need_ configuration other than the defaults (and empty config block), but terraform has no way of knowing this because the configuration no longer exists. The major change here is that terraform will no longer destroy resources if it can't confirm it's using correctly configured providers. In previous versions it might work, but it also was quite common for it to fail because it would using and incorrect provider config from a higher level.

By moving the actual provider config out of the module, you can declare exactly what configuration is used by the module, as well as retain the config when you remove the module. There is also the option of using -target to destroy a specific module before removing the configuration. This might be an area where can add improved tooling for handling modules.

[dghubble]

Removing the constraints as in https://github.com/poseidon/typhoon/pull/68 does solve my issue and allows deletions again. Its just a steep price to pay.

moving the actual provider config out of the module, you can...

What's the story for module authors? I understand you're saying "you" as in the end-user can specify/configure the provider, but how can modules continue to place constraints on the plugins in use? Modules can't be expected to work with old, broken, or known incompatible plugins, which is why we added constraints in the first place.

Is the answer now just to document what plugin versions end-users should use against my modules?

terraform will no longer destroy resources if it can't confirm it's using correctly configured providers

I understand what you're saying. In practice this forces me to remove the provider blocks in modules since its not viable to create modules that can't be deleted.

There is also the option of using -target to destroy a specific module before removing the configuration

That's good to know. In practice, I can't ask end-users to jump through these hoops. Its unreasonable. A lot of these deployment systems are automated and "declarative" (often a sync loop that runs apply on diffs). Those systems don't "know" the right imperative commands to run.

[jbardin]

The constraints can stay, as the version field is a special case that isn't part of the configuration. The only caveat to this is that you then need to pass in a provider via the providers map to declare where the actual configuration resides, even if that configuration is empty. This sort of makes providers in modules similar to empty variables in modules.

One enhancement I'm thinking about, is internally moving providers that have no configuration to the top level so that when removing the module terraform knows it can just use the defaults. This would ease the use of providers like "null" and "local" where you don't normally use a config, or even declare them except for version constraints.

[dghubble]

Ok. It seems I can leave the provider block in modules (since they're only used for version constraints), and deletes will work iff I instruct end users to:

Copy providers into a top level providers.tf:

provider "local" {
  version = "~> 1.0"
  alias = "default"
}

provider "null" {
  version = "~> 1.0"
  alias = "default"
}

provider "template" {
  version = "~> 1.0"
  alias = "default"
}

provider "tls" {
  version = "~> 1.0"
  alias = "default"
}

And also edit every module usage to have a providers block:

module "aws-cluster" {
  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes?ref=5ea7ce0af559857591f20fe19b03aab177fd7032"

  providers = {
    aws = "aws.default"
    local = "local.default"
    null = "null.default"
    template = "template.default"
    tls = "tls.default"
  }

  cluster_name = "blah"
   ...

That's what you're describing for v0.11.x right?

[jbardin]

@dghubble,

Yes, that's basically the pattern now. It's a little more verbose, but it allows you to know _exactly_ how each provider being used by a module is configured.

I agree that it makes things a little more cumbersome if your only goal is to provide version constraints, because the version field is in the provider block which is tied its configuration. We can see what could be done there to try and smooth out the workflow, like possibly having another method for declaring provider constraints.

[texascloud]

Does this mean resources created with an older version of terraform with a config that both no longer exists and doesn't explicitly pass providers to a module is unable to be deleted using this new terraform version?

Also I don't quite understand the inheritance of providers. Assume a root module with the aws provider is using another module in which the aws provider is also declared, but this time aliased as in the case of route53 in the below example. Does this mean the root-level aws provider config overrides the module's aws provider config, which is why I'm getting this error?

Ex: terraform apply in a directory with foo.tf which creates foo.tfstate
foo.tf is using module bar in which this provider block and resource is defined within:

provider "aws" {
    alias = "route53"
    ...
}

resource "aws_route53_record" "route53_record" {
    provider = "aws.route53"
    ...
}

I am in a situation where foo.tf no longer exists but the state file does. Attempting to run terraform destroy -state=foo.tfstate gives the following error:

Error: module.bar.aws_route53_record.route53_record: configuration for aws.route53
is not present; a provider configuration block is required for all operation

How can I go about resolving this?

[jbardin]

@CamelCaseNotation,

This would happen with the current version as well. Presumably the configuration for aws.route53 differed from the defaults in some way, seeing how it has an alias, so in this instance the feature is doing exactly what it was designed to do and not destroying infrastructure with an incorrectly configured provider.

If the config no longer exists, make a dummy config containing only the providers with the desired configuration and aliases. If this doesn't work for you, feel free to file a new issue with the details of your previous and current configurations.

[apparentlymart]

I created #16835 with an idea for how we might resolve the fact that it's now hard to specify provider version constraints for child modules. I'd love to hear any feedback you might have on that idea @dghubble, and whether it would address your needs here.

[texascloud]

@jbardin My new config is logically the same but restructured, so I assume that's why it appears to work when I do a terraform destroy against foo.tfstate.

I solved my issue by moving the aliased provider to the root config and passing it to the module via the providers block, as the documentation says.

[dghubble]

I'll take a look at the proposal soon.

I need to figure out how to guide end-users of the module to be able to use v0.11.x. The way existing module instances need to be updated to have a providers block, users run plan and see no changes, run terraform apply anyway, and only then you can delete the resource if odd. However I try to document the workaround, it feels like I'm making folks jump through hoops.

[dghubble]

Just to followup, for my module users, I've written a migration guide from Terraform v0.10.x to v0.11.x to handle the changes in https://github.com/poseidon/typhoon/pull/95. I'm sharing this in case it helps others and to provide a complete feedback loop showing the impact - end-users must make specific edits and take action to continue managing resources with Terraform v0.11.x.

Feel free to comment if the guide can be made any clearer or simpler, if you like.

[dghubble]

Terraform v0.12 addresses this issue with providers needing to be passed explicitly into a module. For module end users, usage returns to how it appeared in v0.10.

Within the re-usable module, provider plugin requirements can be expressed like:

terraform {
  required_version = "~> 0.12.0"
  required_providers {
    google       = "~> 2.5"
    ct           = "~> 0.3.2"
    template     = "~> 2.1"
    null         = "~> 2.1"
  }
}

Module usage can drop the provider passing.

module "google-cloud-yavin" {
  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.14.4"

-  providers = {
-    google   = "google.default"
-    local    = "local.default"
-    null     = "null.default"
-    template = "template.default"
-    tls      = "tls.default"
-  }
...
}

I wrote a migration guide for my users https://github.com/poseidon/typhoon/pull/486. Hopefully that closes the full loop on this issue.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.