Terragrunt: passing in bucket name for multiple remote states

[halfpastjohn]

We have our terragrunt config set up identical to your documentation.

This is more of a best practices, or strategical, type of question. Since there is no forum I'm unsure where else to ask this..

What I want to know is how to dynamically reference common remote state values from multiple AWS account state files, which are in different buckets.

We need to be able to pass in "bucket" to the remote state resource so that the module can grab values from whichever AWS account its being launched into.

data "terraform_remote_state" "common" {
backend = "s3"
config {
bucket = "${var.tfstate_bucket_common}"
key = "${var.region}/prod/terraform.tfstate"
region = "us-east-1"
}
}

Now - this works as long as I create a "tfstate_bucket_common" var in every single module and then pass in the variable in every single tfvars file, but there has to be a better way.

I tried to include the var in the account level .tfvars file, like so:

terragrunt = {
remote_state {
backend = "s3"
config {
bucket = "ops-terraform-state"
key = "${path_relative_to_include()}/terraform.tfstate"
region = "us-east-1"
encrypt = true
}
}
tfstate_bucket_common = "ops-tfstate"
}

.. but that didn't work.

Is there a better way to do this that I just can't find?

Thank you!

• John

[brikis98]

I haven't had a chance to add an example of this to the sample repo, but one way to solve this is with extra_arguments and find_in_parent_folders().

For example, in your root terraform.tfvars, you could have:

terragrunt = {
  terraform {
    extra_arguments "bucket" {
      commands = "${get_terraform_commands_that_need_vars()}"
      optional_var_files = [
        "${find_in_parent_folders("account.tfvars", "ignore")}"
      ]
    }
  }
}

You can then define all sorts of common variables in account.tfvars (e.g., such as tfstate_bucket_common), and all of them would automatically get passed to every module.

I'd welcome a PR for this in the sample repo if anyone has the time :)

[halfpastjohn]

When you say root terraform.tfvars file are you talking about the account level one? If that's the case then find_in_parent_folders would just be the same exact folder, wouldn't it?

[halfpastjohn]

terragrunt = {

  remote_state {
    backend = "s3"
    config {
      bucket     = "tfstate-live"
      key        = "${path_relative_to_include()}/terraform.tfstate"
      region     = "us-gov-west-1"
      encrypt    = true
      profile    = "gov"
    }
  }

  terraform {
    extra_arguments "bucket" {
      commands           = ["${get_terraform_commands_that_need_vars()}"]
      optional_var_files = [ "${find_in_parent_folders("account.tfvars", "ignore")}"]
    }
  }

}

and then just add account.tfvars in the same dir as this?

[brikis98]

When you say root terraform.tfvars file are you talking about the account level one?

Yes

If that's the case then find_in_parent_folders would just be the same exact folder, wouldn't it?

The code in the root file is _included_ in the child files, as if you had copy/pasted it, so find_in_parent_folders will work as expected when you run terragrunt apply on those child modules.

[halfpastjohn]

That's what I thought it would do but I'm getting this when running plan-all from the root dir:

invalid value "../../../global.tfvars" for flag -var-file: Error reading ../../../global.tfvars: open ../../../global.tfvars: no such file or directory
Usage: terraform plan [options] [DIR-OR-PLAN]

And I can cat that location from the module itself:

[John@PC1 /Users/John/Git/terraform-live/gov/us-gov-west-1/production/nginx$]$  (master) cat ../../../global.tfvars
tfstate_bucket_common = "g-ops-tfstate"
tfstate_bucket_region = "us-gov-west-1"
aws_profile           = "gov"

It's trying to find global.tfvars three directories above the root dir.

[brikis98]

Ah, sorry, I missed the ${get_tfvars_dir()} piece, which will turn that relative path into an absolute path:

terragrunt = {
  terraform {
    extra_arguments "bucket" {
      commands = "${get_terraform_commands_that_need_vars()}"
      optional_var_files = [
        "${get_tfvars_dir()}/${find_in_parent_folders("account.tfvars", "ignore")}"
      ]
    }
  }
}

[halfpastjohn]

Awesome! Works perfectly and thank you for such quick responses.

[brikis98]

Glad to help. If you have a min, a PR in the sample repos with this example would be great :)

[kelsmj]

@brikis98 How would one reference the global variables from the account.tfvars file in each subsequent terraform.tfvars? What is the syntax for that?

[brikis98]

What do you mean by "subsequent" terraform.tfvars?

[kelsmj]

Sorry, wasn't too clear with that question. If I have the following in my account.tfvars

tfstate_bucket_common = "g-ops-tfstate"

How and where can I actually use 'tfstate_bucket_common'? Can I use it in other terraform.tfvars files? Can it only be used in terragrunt blocks? I couldn't find an example where any of those global variables defined in account.tfvars are actually being used.

[brikis98]

If you have a foo.tfvars and you add it to optional_var_files, then it's passed to your Terraform commands with the -var-file argument. For more info, see: https://www.terraform.io/docs/configuration/variables.html#variable-files

[kelsmj]

So, does that mean I should be able to use "${var.tfstate_bucket_common}" in any main.tf file?

[brikis98]

Assuming you have a variable "tfstate_bucket_common" { }, yes.

[brikis98]

I believe the original question has been answered, so closing.