terragrunt plan-all - Add option to apply dependencies

[markhopwoodnvm]

We're implementing a pipeline around our terraform builds where each module has a CI build which runs a plan and apply. We'd like to be able to use this model on modules that have dependencies on other modules. This would require all dependencies to be applied prior to running a plan on the module we are testing. Using the example from your documentation :

root
├── backend-app
│ ├── main.tf
│ └── terraform.tfvars
├── frontend-app
│ ├── main.tf
│ └── terraform.tfvars
├── mysql
│ ├── main.tf
│ └── terraform.tfvars
├── redis
│ ├── main.tf
│ └── terraform.tfvars
└── vpc
├── main.tf
└── terraform.tfvars

with the following dependencies between Terraform modules:

backend-app depends on mysql, redis, and vpc
frontend-app depends on backend-app and vpc
mysql depends on vpc
redis depends on vpc
vpc has no dependencies

terragrunt = {
dependencies {
paths = ["../vpc", "../mysql", "../redis"]
}
}

A terragrunt plan-all (with apply dependencies opiton) for backend-app would run the apply on vpc, mysql and redis, then a plan on backend-app

[brikis98]

I think the expectation for the plan command is that it's read-only and therefore always safe to run. It may be a bit surprising for it to be able to apply changes, even if you ask for them explicitly via a flag.

Perhaps the proper abstraction for this is an apply-all-dependencies command that runs apply-all on the dependencies of the current module, and their dependencies recursively, but not the current module?

[markhopwoodnvm]

Yep, that sounds good

[vas1468]

Can we give some love to this enhancement please? No one is assigned yet :( Unfortunately I cannot help.

[w32-blaster]

+1

[Hmerac]

I also encountered with this, some of our modules have dependencies over other modules.

The problem is, at first-run of terragrunt plan-all phase, dependent modules aren't able to retrieve output variables from other module's remote state file because terragrunt plan-all doesn't initialize state files as you know.

We needed to specify terraform apply for each module with regard to dependencies. Even this doesn't solve the fail problem when you run terragrunt plan-all at first-run.

So -> +1 🙌

[brikis98]

It says it right in the README:

Note: It is important to realize that you could get errors running plan-all if you have dependencies between your projects and some of those dependencies haven't been applied yet.

Ex: If module A depends on module B and module B hasn't been applied yet, then plan-all will show the plan for B, but exit with an error when trying to show the plan for A.

[brikis98]

Hm, that's weird. I was responding to a comment that's now gone... Uh... I guess the commenter deleted it?

[Xtigyro]

Hey folks - has there been any progress since the last comment?

[brikis98]

Not that I'm aware of. PRs still welcome.

[derekrprice]

Not sure how complicated this would be, but it seems like the right way to approach this would be not to actually apply the dependencies, but to fake it. As each level is planned, the desired state is already output. That desired state could be used to create a mock remote state used by each dependent plan. Terragrunt is already manipulating remote state in order to share a single definition block down the tree. Is there a way that it could substitute a "terragrunt-mock" state backend provider in the specific case of plan-all?

In my defense, if this is a crazy idea, I don't have any direct experience with the Terragrunt code base or Go yet. Maybe soon, if I keep finding issues like this. :)

[brikis98]

@derekrprice Don't think that would work. Figuring out the right fake outputs to return for every modules is hard enough, but if we're just returning mock data, then when those outputs are fed as inputs to other modules so those modules can run plan, they'll hit errors because those outputs won't correspond to real resources, so all the API calls will fail.

[derekrprice]

@brikis98 Vanilla Terraform can do it when all the resources are in the same state file. It seems like it should be possible. Not easy, perhaps, but possible.