Terraform: Terraform multiple providers

[melbit-anthonyw]

Terraform Version

Terraform v0.11.3

Terraform Configuration Files

ancestor module

provider "aws" {
  region = "us-east-1"
  alias  = "ado"

  assume_role {
    role_arn     = "role"
    session_name = "session"
  }
}

provider "aws" {
  region = "ap-southeast-2"
  alias  = "237"

  assume_role {
    role_arn     = "different role"
    session_name = "different session"
  }
}

module "firstmodule" {
  source = "git::[email protected]:blah/module"

  account = "${lookup(var.role_map["key"], "account")}"
  role    = "${lookup(var.role_map["key"], "role")}"
  region  = "${var.region}"

  providers = {
    "aws.ado"      = "aws.ado"
    "aws.customer" = "aws.237"
  }
}

descendant module

provider "aws" {
  alias  = "ado"
  region = "us-east-1"
}

provider "aws" {
  alias  = "customer"
  region = "us-east-1"
}

module "deploy_user_ses" {
  source = "./modules/ses"

  account = "${var.account}"

  providers = {
    "aws" = "aws.ado"
  }
}

child module

provider "aws" {}

resource "aws_ses_receipt_rule" "deploy_account_ses" {
  name          = "${var.account}-deploy"
  rule_set_name = "default-rule-set"
  recipients    = ["${var.account}[email protected]"]
  enabled       = true
  scan_enabled  = true

  s3_action {
    bucket_name       = "-ses-mail"
    object_key_prefix = "${var.account}-deploy"
    position          = 1
  }
}

Debug Output

module.ses.module.ses.aws_ses_receipt_rule.deploy_account_ses: configuration for module.ses.module.ses.provider.aws is not present; a provider configuration block is required for all operations

Steps to Reproduce

1. terraform init
2. terraform apply

Additional Context

Basically my intention is to have an "ancestor" module that calls a "descendant" module and then that descendant has a series of child modules it would run.
The scenario is I have multiple things in different AWS accounts that need configuring. So an SES module would configure something in us-west-1 in a certain account then the dynamoDB module would then configure something in another account in another region.
Looking over the doco that should technically be feasible with aliased providers but I'm struggling to get the logic right for sub modules

Is anyone able to shine some light on how I can get the above working?

[jbardin]

Hi @melbit-anthonyw,

Sorry you're having trouble with this, it can be quite confusing at times.

This is covered under the Providers Within Modules documentation. In order for module to receive a provider, that provider must not have any configuration in the module. It referred to in on that documentation page as a "proxy configuration block", which means it only sets the provider name.

Adding the region attribute to the providers in your module means that they appear to be configured, and won't be overridden by the modules being passed in. There is no way to partially configure or override part of a provider's config, and merge it with a parent instance of the provider. Earlier versions of terraform attempted to merge providers via an inheritance algorithm, but that turned out to be too complex to reason about and quite error prone. We've found that requiring the provider be explicitly passed around to be much easier to deal with.

[melbit-anthonyw]

Hi @jbardin,

Thanks for the reply. You are certainly right about the confusion part! I did read the doco and I thought I understood it and I've tried multiple approaches before posting. The approach in my intial post was just where I ended

Doing it the way I think it should be done as per the doco

ancestor module - main.tf

module "aws-237" {
  source = "git::[email protected]:blah/blah"

  account = "${lookup(var.role_map["key"], "account")}"
  role    = "${lookup(var.role_map["key"], "role")}"
  region  = "${var.region}"

  providers = {
    "aws.ado"      = "aws.ado-us-east"
    "aws.customer" = "aws.237"
  }
}

ancestor module - provider.tf

terraform {
  backend "s3" {
    key      = "terraform/state.tfstate"
    role_arn = "role"
    region   = "ap-southeast-2"
  }
}

provider "aws" {
  region = "us-east-1"
  alias  = "ado-us-east"


  assume_role {
    role_arn     = "first role"
    session_name = "first role session"
  }
}

provider "aws" {
  region = "${var.region}"
  alias  = "237"

  assume_role {
    role_arn     = "second role"
    session_name = "second role session"
  }
}

provider "aws" {
  assume_role {
    role_arn     = "role"
    session_name = "session"
  }

  alias  = "ado-sydney"
  region = "${var.region}"
}

decendant module - main.tf

provider "aws" {
  alias = "ado-us-east"
}

provider "aws" {
  alias = "237"
}

module "deploy_user_ses" {
  source = "./modules/ses"

  account = "${var.account}"

  providers = {
    "aws.ado" = "aws.ado-us-east"
  }
}

ses module resource - main.tf

resource "aws_ses_receipt_rule" "deploy_account_ses" {
  name          = "${var.account}-deploy"
  rule_set_name = "default-rule-set"
  recipients    = ["${var.account}[email protected]"]
  enabled       = true
  scan_enabled  = true

  s3_action {
    bucket_name       = "ses-mail"
    object_key_prefix = "${var.account}-deploy"
    position          = 1
  }
}

Results in the following

provider.aws: "region": required field is not set

Which is weird because region is definitely set in all of the providers in the root ancestor module so it leads me to believe it's not making it to the descendant and child modules.

[jbardin]

@melbit-anthonyw,

The error indicates that provider.aws is not configured, which is correct.
The only resource in the your example here is aws_ses_receipt_rule, and it is not configured to use a specific provider, so it ends up using the aws provider with only the defaults.

[melbit-anthonyw]

@jbardin,

It technically makes sense but still isn't acting like it should. Chances are I'm missing something very simple...

If I set the provider for the resource to be aws.ado-us-east I get the below

resource "aws_ses_receipt_rule" "deploy_account_ses" {
  name          = "${var.account}-deploy"
  rule_set_name = "default-rule-set"
  recipients    = ["${var.account}[email protected]"]
  enabled       = true
  scan_enabled  = true
  provider      = "aws.ado-us-east"

  s3_action {
    bucket_name       = "ses-mail"
    object_key_prefix = "${var.account}-deploy"
    position          = 1
  }
}

module aws-237.deploy_user_ses: provider alias must be defined by the module or a parent: aws.ado-us-east

If I set it to aws.ado I get the following

resource "aws_ses_receipt_rule" "deploy_account_ses" {
  name          = "${var.account}-deploy"
  rule_set_name = "default-rule-set"
  recipients    = ["${var.account}[email protected]"]
  enabled       = true
  scan_enabled  = true
  provider      = "aws.ado"

  s3_action {
    bucket_name       = "ses-mail"
    object_key_prefix = "${var.account}-deploy"
    position          = 1
  }
}

module aws-237.deploy_user_ses: provider alias must be defined by the module or a parent: aws.ado

I am most definitely defining the provider in the parent.

[jbardin]

The error here is for deploy_user_ses, but the config blocks are for deploy_account_ses.

Can you create a minimal example of the layout you're trying to achieve? You can use something that doesn't take extra arguments like null to make the config easier to view as a whole.

[melbit-anthonyw]

Figured it out. It was referencing an old SES module in my state... Nuked the state and it started working!

Thanks for your help!

[engharb]

@jbardin @melbit-anthonyw it will be kind of you to get help here: https://github.com/hashicorp/terraform/issues/18714.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.