Terragrunt: Core changes on terraform remote subcommand

[mgagliardo]

As you can see on Terraform's CHANGELOG, next release (0.9) will remove the terraform remote subcommand (see PR here).

This of course will break terragrunt's usability as soon as you upgrade Terraform to 0.9

Not sure how to fix this in the future, but maybe putting this issue on the table and some brainstorming will give us some ideas?

Regards!

[josh-padnick]

Thanks for opening this issue. This is a great community discussion to have. I'll start by sharing my thoughts, and of course @brikis98 should offer his as well.

First, from its inception, we've been in favor of all Terragrunt functionality eventually living in Terraform if possible. One tool that does everything you need will always be better than two tools that together do everything you need. So, overall, we're delighted that perhaps Terragrunt played a role in inspiring these changes and now they'll be even easier to use directly in Terraform.

Second, we, Gruntwork, have multiple clients using Terragrunt so we'll think about what we can do to support an easier migration and document/announce something accordingly.

Third, it's possible Terragrunt may wind up functioning as a sort of sandbox for future Terraform development, which is why it's well-suited to a separate project. Once Terraform 0.9.x is released, our remote state config functionality will be deprecated, but other features (automatic locking, spin-up, tear-down) will still be useful. Since locking is also now on the Terraform roadmap (per https://github.com/hashicorp/terraform/pull/11482), we'll deprecate that at some point, too.

Going forward, we're currently working on using Terragrunt to more efficiently spin up multiple environments with a minimum of duplication. Our goal is to maintain our need for many, small Terraform states (versus one giant Terraform state that does everything) while creating smoother workflows for standing up a new environment by minimizing copy & paste, launching a single environment with one command, etc.

Feedback is always welcome, so please share your thoughts!

[brikis98]

+1 to @josh-padnick's response. As Terraform matures, the need for Terragrunt will diminish, and that's a good thing for everyone.

Only thing to add is that we may be able to create some sort of migrate command that moves your remote state configuration from .terragrunt into the backend stuff being added in Terraform 0.9.

[mbainter]

One key thing that at least appears to still be missing from the Terraform implementation are the helper functions for managing state/lock paths.

[grimm26]

Any estimate on making terragrunt compatible with terraform 0.9.x? The longer we're pinned at terraform 0.8.8 the less compelling terragrunt becomes. I know PRs are welcome, but I have no Go-fu (yet) :(

[brikis98]

@grimm26 No estimate at the moment. We'll get to it as soon as we can.

[emoshaya]

Hi all,

We've been using Terragrunt for a while now to manage state locking. Unfortunately, Terragrunt is not compatible with 0.9.0. Therefore, we now want to deprecate use of Terragrunt and migrate to 0.9.0 which fully supports state locking. However, we've noticed a few discrepancies and missing features available in the Terragrunt tool and missing in 0.9.0 Terraform. Please could you advise if these features will be available in future releases and if so, are their any timelines?

The configuration I had in place was for the following:

S3 Backend for States and DynamoDB for State Locking

• In Terragrunt, if the S3 bucket or DynamoDB table is missing, then Terragrunt would create them for you based on the configuration specified. This is missing in Terraform 0.9.0

• In Terragrunt, if you ran a Terragrunt apply, it will poll the dynamoDB table for the lock and will retry based on the value assigned in "max_lock_retries: (Optional) The maximum number of times to retry acquiring a lock. Terragrunt waits 10 seconds between retries. Default: 360 retries (one hour)."

• However, in Terraform 0.9.0 it doesn't retry or poll for lock to be released. Instead you get the following error:

Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed status code: 400, request id: LTCHNG5G8U10REB2A6DV39LO6BVV4KQNSO5AEMVJF66Q9ASUAAJG Lock Info: ID: a4728cc3-303d-4987-af4a-25a54e954907 Path: ce-cog-test-tfstates/devops-terraform/cog.test.tfstate Operation: OperationTypePlan Who: josh@magpie Version: 0.9.1 Created: 2017-03-22 13:19:29.60094345 +0000 UTC Info:

This is potentially a blocker for our CI builds as we have multiple CI builds running simultaneously at any given day and having the build just retrying to acquire the lock is a must to avoid failing builds.

[grimm26]

@emoshaya-cognito Are you asking for terraform features? This is the Gruntwork terragrunt repository. They don't develop terraform.

[emoshaya]

@grimm26 I'm just saying maybe Terragrunt shouldn't deprecate the state locking feature due to the issues I mentioned above.

[josh-padnick]

Closing this thread so that we can direct all comments around Terraform 0.9.0 compatibility to https://github.com/gruntwork-io/terragrunt/issues/158.