Terraform: Provider Aliases Interact Poorly with Reusable Modules

[apparentlymart]

Recently support was added for having multiple instances of a given provider with different settings, using the alias attribute:

provider "aws" {
    region = "us-east-2"
    alias = "use2"
}

provider "aws" {
    region = "us-west-1"
    alias = "usw1"
}

resource "aws_something" "foo" {
    provider = "aws.use2"
}

resource "aws_something" "bar" {
    provider = "aws.usw1"
}

This is very useful for defining multi-region infrastructures, but several unfortunate interactions make this difficult to use in conjunction with reusable modules.

Child modules inherit the providers defined in the root module, but when aliases are in play it's necessary to explicitly specify the provider on each resource, even when those resources are defined in a module. This is problematic when trying to write a generic module that can be used in multiple contexts, since it needs to "know" the appropriate provider string.

My first thought was to pass the alias into the child module as a variable, but it looks like interpolations are not supported on the provider attribute.

It feels like fixing this could go in two directions:

• Require each module to define its own providers, rather than inheriting from the root, and have a separate provider alias namespace per module. This allows the modules to be fully encapsulated, but would require many modules to take provider parameters as input variables.
• Allow the provider resource attribute to be interpolated, and consider the empty string to be the same as not providing it at all so that there is a way to select the "default" provider of that type. This would still require extra parameters as module input variables, but the parameters would be provider alias names rather than provider parameters directly.

For the sorts of use-cases I've encountered, I prefer the former option. It more cleanly encapsulates module behavior, reduces the need to duplicate the provider attribute across many resources, and forces a reusable module to be clearer in its interface (via its input variables) which providers it will interact with. However, I could also make the latter work. I'm sure other compromises are possible too.

With the current setup I've been unable to make use of provider aliases at all, and I've been forced to use a separate config per region with a helper module that consumes their states via the terraform_remote_state resource and exposes the variables in a single spot. (My use case: a reusable module which creates an AWS route table that needs to be duplicated in each region.)

[nickryand]

I'm pretty sure the first option used to work on older versions of terraform. I built a set of terraform modules to build out the infrastructure for cross region VPN tunnels using this document: http://aws.amazon.com/articles/5472675506466066.

Having interpolation for the provider is one way to do it but if I am already passing variables into the module, I might as well pass API keys and region into the module and handle the construction of the provider there. This prevents me from having multiple providers in the root module and it removes the need for setting the provider parameter on every resource.

[ibains007]

+1 for this. My preference would be to override attributes of the default provider. so for example..
i would have my main provider in the root of the project. i would then have a provider declaration in the module, but only override the region using interpolation. That way i could re-use my module without having to add a region to every resource. I could also set a default region in the module..

[dannietjoh]

+1

[tj]

Big blocker for me as well, unless I want to copy/paste everything a dozen times. I'd also +1 being able to pass the provider in, which is confusingly what #451 mentions to do.

[jseaidou]

Hey guys, has there been any progress on this? This is a blocker for us at Box

[opokhvalit]

yep, this is really necessary, because it does not allow write one module instead of tons of code....

[apparentlymart]

There has been some recent work in #6186 that should clean up some weird behavior when mixing combinations of inherited and overridden providers, so this will hopefully make my first suggestion (pass the provider settings into a child module via variables and have it instantiate its own provider) work in more cases.

There are still some issues where the value passed into the child module is computed and then the child module tries to use it in a provider configuration block. Computed values on providers are a bit broken in general, which is one of the motivations of my proposal in #4149. #4149 can also potentially open the door for my second suggestion (allowing the "provider" meta-argument to be interpolated) by giving Terraform a way to deal with the case where the provider is interpolated.

So there hasn't been direct progress on this issue as stated, but there are foundational things going on that should make this issue less severe in the short term and make it a well-supported path in the longer term.

[donaldguy]

I suspect this would require more rearchitecting than will make it happen soon (or ever), but in terms of clarifying reasoning about overrides...

another option that I wish were the case right now would be to let providers be explicitly passed into and out of modules as a ~variable and ~output, with a mostly normal dependency graph over constrained providers, likely with weights on the constraint solving that would have them fully configured as early as possible to allow their use in a full refresh

In particular, I am currently in want of a world where my submodule responsible for the aws_db_instance could export a postgresql provider that I could use to make dbs and roles in another module. But one could also imagine IAM resources that emitted an aws provider, or any variety of cloud instance resource being presupposed to generate a docker provider

But obviously this would need at least #1178 or its implicit version, and some syntax to export a provider alias (that could be looked up in a global namespace) if not the provider itself.

A middle ground might involve being more strictly iterative on partial runs, saving more info about a previously configured provider in tfstate and then allowing providers to be fetched out of a remote_state but that kind of complicates the execution model, and maybe frustrates the already iffy state of secrets-in-state.

Anyway, something to think about

(Edit: probably related one way or another to #4169 as well)

[nickithewatt]

IMHO this really is required to help with writing re-usable terraform code. Just checking if there has been any progress on this, maybe directly/indirectly via 0.7?

[g-marius]

+1 ^

[dmikalova]

This issue really gets in the way of standing up multiple regions. It means I have to instantiate a provider in many different places. Our setup encapsulates resources into modules, those modules are called by component specific modules that define them, those modules are called by a regional module that defines the region, and then that module is called by a region coordinating module.

This means we have to define providers at each of those levels, rather than just being able to do a `count = "${length(var.regions)}" for the AWS provider at the very top level and passing that down. Every resource now needs to declare it's provider in the same module - if you forget then dependent resources won't be able to locate another resource just by passing the resource's ID because they will default to the default, unaliased provider.

[nickmaccarthy]

+1 on this. It would be great to be able to control the provider by a variable. It would encourage code reuse and more portability between deployments.

[spencerthang]

+1 on this, we really need multi-region deployments.

[jakubigla]

I'm massively surprised there's no "provider" variable. Because of that I need to pass region to my module through variables, which is far away from ideal.
Any chance someone is on that subject in a short term future?

[StoyanIvanovI]

This just disables us from using modules altogether. Please fix it sooner rather than later as it will make our infrastructure a lot more organised if it is within modules.

[csabatini]

Needing this as well

[apparentlymart]

The current situation is that it's possible to use provider blocks in child modules with and without aliases, and they will be correctly scoped to the child module. However, the remaining poor interaction is as follows:

If you later remove the module block from the parent module, that causes Terraform to want to destroy all of the resources from that module as expected, but it finds itself unable to do so because the provider configuration for those resources has also been removed. Since the provider configuration and the resources appear as one single unit from the perspective of the parent module, it's not possible to remove the resources while the provider block is still present.

There is a workaround that is slightly fiddly:

• Use terraform plan -destroy -target=module.foo to create a plan to destroy the resources in that module while they still exist in config.
• Apply that plan
• Now remove the module's configuration block from the configuration module and plan/apply again, which should be a no-op except to silently drop the remaining empty remnant of the module from the Terraform state.

[analytically]

+1 need this as well

[jamescarr]

This is blocking me as well :-(

[roccato]

+1

[taiidani]

Having the same issue here.

[apparentlymart]

Given the broad scope of this issue (which I filed long before I was a HashiCorp employee working on Terraform full time) and that the problem has drifted somewhat since I originally opened it, I feel inclined to replace it with one or more "tighter" issues that focus on specific problems, since such things are more actionable.

As far as I know, the main remaining issue here is the one I mentioned in my last comment (also before I was an employee) with removing a module that has its own provider configuration. I'm considering opening a fresh issue for that with more details on repro steps, etc. I see some others were leaving "me too" comments here so I just wanted to clarify whether those were in response to my latest comment or whether there are other issues people are seeing that we should also continue to track.

I'm going to leave this as-is for the moment but I'd welcome any details from other participants in this issue about specific problems they've run into, if they are different than the problem I described last November.

Thanks for the great discussion here, and sorry that this rather-generic issue has languished here so long.

[davedash]

@apparentlymart I think what would help is for the documentation to be a bit more instructive. I've had to search from issue to issue to realize that I might be able to reuse a module between regions in AWS. I think if there was a portion of the documentation, or even a cookbook of sorts that said at a bare minimum how to create a region-agnostic module, then I would not be following this and many other similar issues.

[joshuaspence]

@apparentlymart Correct me if I am wrong, but another issue is that there is no way to pass a provider to a module. This means that I can't use a module to create resources in different AWS regions, for example. I guess this might be what @davedash is referring to.

[PeterWeiler]

Agreed.

Today I'm able to pass a region variable into a module, resulting in each module defining it's own provider. Or I can define a provider in a parent of the module(s), which they'll use...but without aliases leading to namespace collisions in the parent.

What I'd ideally like to do is declare all providers at the root level, with appropriate aliases / names, and have the module use variable interpolation to select the right provider. In doing so, removing a module does not result in the loss of a provider. It also allows a module to leverage more than one provider.

[karnauskas]

Perhaps we need special variable type "provider" next to string, map and list? Modules could group multiple resources, potentially working with multiple providers. Even at this stage it would be easier to pass into module something like aws.alias1, aws.alias2, foo.bar etc. which could be reference to top level provider block.

[apparentlymart]

Thanks for this additional feedback, all. Agreed that the current way providers are "inherited" into child modules is rather strange, forcing some inconvenient usage and making multi-region setups harder than they ought to be. I'll try to mold this into a more detailed use-case description and open a fresh issue for it so we'll have somewhere to discuss different implementation approaches.

[apparentlymart]

I merged what I wrote above with the other use-cases discussed yesterday into #15762, which hopefully presents a more actionable task which we can do design work against.

This is not something we'll be able to start attacking immediately but at least a more direct description of a concrete problem is easier to plan around.

Thanks again for the great (multi-year!) discussion here.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.