Terragrunt: Restricting terragrunt *-all to specific environment?

[dudicoco]

Hey guys,
I have the following configuration in the parent terragrunt.hcl file:

inputs = merge(
  yamldecode(
    file("${get_parent_terragrunt_dir()}/../global/global-values_${get_env("ENVIRONMENT", "")}.yaml"),
  ),
  yamldecode(
    file("${get_terragrunt_dir()}/values_${get_env("ENVIRONMENT", "")}.yaml"),
  )
)

Running terragrunt plan-all/apply-all fails due to some directories which don't contain a yaml file for the specific environment.
Is there a way to restrict terragrunt *-all to run only on modules which contain the environment's yaml file, without resorting to --terragrunt-include-dir or --terragrunt-exclude-dir?

Thanks!

[yorinasub17]

This isn't an answer to your exact question, but should workaround the specific issue you are hitting with the usage of yamldecode and file. The canonical way to address this is to use find_in_parent_folders and default to an empty yaml file at the root of your project. See this repo for an example setup. Specifically: https://github.com/gruntwork-io/terragrunt-infrastructure-live-example/blob/master/non-prod/terragrunt.hcl#L26

[dudicoco]

Thanks @yorinasub17.
I did try this example, however the module is asking for the missing variables when an empty yaml file is used as input.
Any other suggestions?

[dudicoco]

I have found a workaround to the missing variables issue, I added the following to the child terragrunt.hcl:

skip = fileexists("../values_${get_env("ENVIRONMENT", "")}.yaml")  ? false : true

However, I'm still not loving the idea of having to create a subdirectory for the terragrunt,hcl file just for the sake of using the find_in_parent_folders function. Is there an alternative?
Thanks

[yorinasub17]

however the module is asking for the missing variables when an empty yaml file is used as input.

What is the normal way you are providing these variables? As in, are these modules designed such that you need the variables to be provided manually and thus want to exclude from the apply-all?

I'm still not loving the idea of having to create a subdirectory for the terragrunt,hcl file

I am confused by this. Are you referring to the terragrunt.hcl inheritance model with include? Or are you referring to the yaml files? What is the ideal way you want to organize your folders?

[dudicoco]

@yorinasub17 I'm providing the variables with a yaml values file specific for the component within a specific environment:

yamldecode(
    file("${get_terragrunt_dir()}/${find_in_parent_folders("values_${get_env("ENVIRONMENT", "")}.yaml", local.default_yaml_path)}"),
  )

Currently this is my directory structure:

├── devops
│   ├── amazing-app
│   │   ├── ecr
│   │   │   ├── terragrunt
│   │   │   │   └── terragrunt.hcl
│   │   │   └── values_us-east-1_global.yaml
│   │   ├── k8s
│   │   │   ├── terragrunt
│   │   │   │   └── terragrunt.hcl
│   │   │   └── values_us-east-1_dev.yaml
│   │   └── s3
│   │       ├── terragrunt
│   │       │   └── terragrunt.hcl
│   │       └── values_us-east-1_dev.yaml
├── empty.yaml
├── global
│   ├── global-values_us-east-1_dev.yaml
│   ├── global-values_us-east-1_global.yaml
│   ├── global-values_us-east-1_prod.yaml
│   ├── global-values_us-east-1_staging.yaml
├── modules
│   ├── ecr
│   │   ├── main.tf
│   │   ├── outouts.tf
│   │   └── variables.tf
│   ├── k8s
│   │   ├── main.tf
│   │   ├── outouts.tf
│   │   └── variables.tf
│   └── s3
│       ├── main.tf
│       ├── outouts.tf
│       └── variables.tf
└── terragrunt.hcl

As you can see I had to move the child terragrunt.hcl files to a terragrunt subdirectory due to the use of the find_in_parent_folders function.

[yorinasub17]

Oh in that case, can you use path_relative_from_include for the default path? E.g:

file("${get_terragrunt_dir()}/${find_in_parent_folders("values_${get_env("ENVIRONMENT", "")}.yaml", "${path_relative_from_include()}/global/values_${get_env("ENVIRONMENT", "")}.yaml")}")

[dudicoco]

Thank @yorinasub17.
I ended up with this workaround:

locals {
  default_yaml_path = find_in_parent_folders("empty.yaml", "empty.yaml")
  file_path = "${get_terragrunt_dir()}/values_${get_env("TF_VAR_environment", "")}.yaml"
  environment_yaml_file = fileexists(local.file_path) ? local.file_path : local.default_yaml_path
}

inputs = merge(
  yamldecode(
    file("${get_parent_terragrunt_dir()}/global/global-values_${get_env("TF_VAR_environment", "")}.yaml")
  ),
  yamldecode(
    file(local.environment_yaml_file),
  ),
  {
    team = "${split("/", path_relative_to_include()).0}"
  },
  {
    account = "${split("_", get_env("TF_VAR_environment", "")).0}"
  }
)

[yorinasub17]

Thanks for closing the loop. Going to close this issue now as we don't intend on implementing complex build logic for the xxx-all commands in terragrunt at the moment.