Terragrunt: Do you want to copy existing state to the new backend?

[davidvasandani]

Terragrunt wants to copy state to a new backend each run.

2018/07/02 16:57:54 [DEBUG] command: asking for input: "Do you want to copy existing state to the new backend?"
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "s3" backend to the
  newly configured "local" backend. No existing state was found in the newly
  configured "local" backend. Do you want to copy this state to the new "local"
  backend? Enter "yes" to copy and "no" to start with an empty state.

No changes to the backend have been made.

Debug Output
Test Repo

Steps to reproduce:

1. Update backend bucket ./terraform/live/terraform.tfvars
2. Run terragrunt plan in ./terraform/live/dev/echo/

[brikis98]

What version of Terragrunt? Could you paste the full log output?

[davidvasandani]

@brikis98 the the log output is here Debug Output.

State is never pushed to S3 and S3 remains empty. This seems like a config issue on my end but 2 days of learning Terragrunt and I don't see where I've gone wrong.

[davidvasandani]

terragrunt version v0.15.2
Terraform v0.11.7

During the init the backend is specified by the command-line flags:

[terragrunt] Running command: terraform init \
-backend-config=region=us-west-2 \
-backend-config=dynamodb_table=terraform-locks \
-backend-config=encrypt=false \
-backend-config=bucket=lz-terragrunt-terraform-state \
-backend-config=key=dev/vpc-eip/terraform.tfstate \
-from-module=file:///Users/davidneudorfer/dev/lrnz-ops/terraform/modules/aws/eip \
/Users/davidneudorfer/.terragrunt/229HQvoqVyk4F6er0XeSM8ci_wk/QG6HYkqiY7ug8JgGU0H6AdPSmRw

The backend config is written to the state file but when plan runs Terragrunt doesn't fill in the backend details and Terraform is left thinking I am changing the backend.

[terragrunt] Running command: terraform plan \
-var-file=/Users/davidneudorfer/dev/lrnz-ops/terraform/live/dev/vpc-eip/../../account.tfvars

account.tfvars

[davidvasandani]

@brikis98 I've gotten my env working and I'll post a follow up asap. It took reworking how I was dealing with modules but I'm glad I stuck with it. Hopefully my learnings can prevent others from going down a similar rabbit hole.

[brikis98]

Did you have a backend "s3" { ... } block in your Terraform module?

Looking forward to hearing what you found!

[TimJones]

@davidneudorfer I'm running in to a similar problem. Can you update us?

[augustgerro]

I have simular issue when changing Environment, eg:

1. I am working with prod env, here is all ok:

TF_VAR_env=prod terragrunt apply

1. I'm swithcing to another environment and receive the confirmation dialog:

TF_VAR_env=dev terragrunt apply

Do you want to overwrite the state in the new backend with the previous state?
Enter "yes" to copy and "no" to start with the existing state in the newly
configured "s3" backend.

> Enter a value: no

The problem here is that a remote Bucket under the dev name will be rewritten to the state prod. Then the code with dev variables will change the prod resources. This is very dangerous.

All environments exist and were created before.

@brikis98 Is this normal behavior, or am I doing something wrong?

[yorinasub17]

@augustgerro This is not typical terragrunt usage, as the canonical usage is separating your environments into separate terragrunt.hcl files and folders (see our example in Keep your terraform code DRY).

What's happening here is the same thing that would happen if you modify the terraform remote state in the same terraform folder:

1. When you first run prod, terragrunt creates the source dir in .terragrunt-cache and calls terraform init. This sets up the state tracker in .terraform of the working dir. Then terragrunt proceeds to call terraform apply.
2. When you then switch to dev, terragrunt will reuse the source dir in the .terragrunt-cache and call terraform init again with the new setting. However, since the new remote state conflicts with the state that terraform expects, you get the warning. This happens because the old state metadata is tracked in the same .terraform folder that is being used.

This is one of the reasons why we use different folders for each environment, as it creates completely different working directories per environment.

If you want to keep your current organization, the best way to handle this is to use the --terragrunt-source-update flag so that terragrunt will always wipe the working dir and start clean each time.