Terraform: taint command doesn't recognize tfstate in workspace

[takayukioda]

Terraform Version

$ terraform -v
Terraform v0.12.5
+ provider.aws v2.20.0

Terraform Configuration Files

...

It's about terraform taint command does not check tfstate file generated in workspace, so I think there is no relevant configuration.

Debug Output

$ TF_LOG=trace terraform taint aws_instance.main
2019/07/21 20:33:19 [INFO] Terraform version: 0.12.5  
2019/07/21 20:33:19 [INFO] Go runtime version: go1.12.4
2019/07/21 20:33:19 [INFO] CLI args: []string{"/usr/local/Cellar/tfenv/1.0.1/versions/0.12.5/terraform", "taint", "aws_instance.main"}
2019/07/21 20:33:19 [DEBUG] Attempting to open CLI config file: /Users/username/.terraformrc
2019/07/21 20:33:19 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/07/21 20:33:19 [INFO] CLI command args: []string{"taint", "aws_instance.main"}
2019/07/21 20:33:19 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2019/07/21 20:33:19 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2019/07/21 20:33:19 [DEBUG] New state was assigned lineage "3c378636-89d2-e3e0-2fe0-96d081ce0630"
2019/07/21 20:33:19 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2019/07/21 20:33:19 [TRACE] Meta.Backend: instantiated backend of type <nil>
2019/07/21 20:33:19 [DEBUG] checking for provider in "."
2019/07/21 20:33:19 [DEBUG] checking for provider in "/usr/local/Cellar/tfenv/1.0.1/versions/0.12.5"
2019/07/21 20:33:19 [DEBUG] checking for provider in ".terraform/plugins/darwin_amd64"
2019/07/21 20:33:19 [DEBUG] found provider "terraform-provider-aws_v2.20.0_x4"
2019/07/21 20:33:19 [DEBUG] found valid plugin: "aws", "2.20.0", "/Users/username/.../.terraform/plugins/darwin_amd64/terraform-provider-aws_v2.20.0_x4"
2019/07/21 20:33:19 [DEBUG] checking for provisioner in "."
2019/07/21 20:33:19 [DEBUG] checking for provisioner in "/usr/local/Cellar/tfenv/1.0.1/versions/0.12.5"
2019/07/21 20:33:19 [DEBUG] checking for provisioner in ".terraform/plugins/darwin_amd64"
2019/07/21 20:33:19 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2019/07/21 20:33:19 [TRACE] backend/local: CLI option -state is overriding state path to terraform.tfstate
2019/07/21 20:33:19 [TRACE] backend/local: state manager for workspace "develop" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: no previously-stored snapshot exists
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: locking terraform.tfstate using fcntl flock
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: read nil snapshot
2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info

Error: No such resource instance

2019/07/21 20:33:19 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
The state currently contains no resource instances whatsoever. This may occur
if the configuration has never been applied or if it has recently been
destroyed.

Crash Output

Expected Behavior

terraform taint command should read state from workspace and mark resource as taint without specifying state file

$ terraform workspace select develop
$ terraform taint aws_instance.main
Resource instance aws_instance.main has been marked as tainted.

Actual Behavior

terraform taint need to run with -state option to recognize resources as expected

$ terraform workspace select develop
$ terraform taint aws_instance.main

Error: No such resource instance

The state currently contains no resource instances whatsoever. This may occur
if the configuration has never been applied or if it has recently been
destroyed.

$ terraform taint -state=./terraform.tfstate.d/develop/terraform.tfstate aws_instance.main
Resource instance aws_instance.main has been marked as tainted.

Steps to Reproduce

1. terraform init
2. terraform workspace new develop
3. terraform apply
4. terraform taint aws_instance.main
   
   
   
   • gets an error that resource doesn't exists
   
   

here's reproduce repo
https://github.com/takayukioda/reproduce-lab/tree/master/taint-in-workspace

Additional Context

References

[justhamade]

I think this issue also applies to terraform refresh

[stefano-m]

I can confirm the unexpected behavior with terraform refesh with

$ terraform version
Terraform v0.12.5

despite having a state file in a workspace (e.g. ./terraform.tfstate.d/my-workspace/terraform.tfstate) and the workspace is selected, when running terraform refresh I get

Empty or non-existent state file.

and then terraform creates a new state file in the current directory ./terraform.tfstate, as already mentioned, the workaround is to run

$ terraform refresh -state=./terraform.tfstate.d/my-workspace/terraform.tfstate

[derekbelrose]

This issue seems to go back as far as 12.0. I just reproduced it in a test environment.

How do you specify a state when it is remotely stored in s3?

[cevich]

Still happening in 12.6 also apply shows the incorrect path when a workspace is used:

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

That should read State path: terraform.tfstate.d/$WORKSPACE/terraform.tfstate

[kulinacs]

Running with trace debug shows:

2019/08/13 17:12:49 [TRACE] backend/local: CLI option -state is overriding state path to terraform.tfstate
2019/08/13 17:12:49 [TRACE] backend/local: state manager for workspace "develop" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup

Not sure if it's the correct fix, but replacing

https://github.com/hashicorp/terraform/blob/fcd4e9822fadadbea8fbd928f75032f7102184fe/command/taint.go#L34

with

cmdFlags.StringVar(&c.Meta.statePath, "state", "", "path")

stops the overwrite from happening and fixes the issue.

[sgirones]

I've seen this behaviour when running plan/apply with a local state.

$ terraform state pull > local.tfstate
$ terraform plan -state=local.tfstate
$ terraform apply -state=local.tfstate

These commands modify both the local and the remote state in S3

[sgirones]

Even when I specify the -state-out explicitly, it still persists it on the remote state:

$ terraform state pull | grep serial
  "serial": 63,
$ terraform apply -state=local.tfstate -state-out=local.tfstate
$ terraform state pull | grep serial
  "serial": 64,

[atecce]

i've just upgraded to v0.12 and noticed this as well. this is a pretty big deal for our use case as we often taint vms so that we don't have to rebuild entire networks when remote-exec goes wrong or something else. is this slated for a release?

[cadavre]

So few is harmed by that? Looks like just few people are using workspaces and I can't be surprised by that as it is close to useless.

Any news on this issue? I know we can eventually use --state=terraform.tfstate.d/{workspace}/terraform.tfstate but all commands must respect selected workspace!

[KptnKMan]

This definitely affects me and is really annoying. I want this fixed for sure.

[jphalip]

I confirm I'm seeing this issue in v0.12.24 as well. Explicitly passing the --state parameter fixes it, although I agree that seems like a non-ideal work-around.

[TheMrZZ]

Still a problem today. Even with the --state, I didn't manage to get it working. It's very annoying.

[curtbusch]

Any update on this? I'm not having any success with the --state flag either using azurerm as a backend.

[adambutler]

Aggree this is still an issue even with --state on v0.12.24

[varnav]

Still problem with 0.12.26

[ayoubeddafali]

Problem persist in v0.12.24

[Hakujou]

Here's an easy workaround until the issue is fixed:

ln -s terraform.tfstate.d/workspace/terraform.tfstate .
terraform taint xxx
rm -f terraform.tfstate

[bryanasdev000]

Still a problem in Terraform v0.12.28.

[phil-bee]

This is still an issue in v0.13.2. I really feel that the --state argument is a workaround. I wonder how many other commands do not respect the workspace?

[mildwonkey]

A PR fixing this issue has been merged. The fix will be available in Terraform v0.14.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.