Terraform: Terraform 0.12.9 Init Fails to Read Local State File

[marcboudreau]

Terraform Version

Terraform v0.12.9

Terraform Configuration Files

terraform {
    required_version = "~> 0.12"
}

provider "google" {
    project = "npav-172917"

    version = "~> 2.13"
}

resource "google_compute_instance" "test" {
    boot_disk {
        initialize_params {
            size = 10
            image = "family/adh-debian"
        }
    }

    machine_type = "n1-standard-2"
    name         = "marc-test"
    zone         = "us-central1-c"

    network_interface {
        network = "default"

        access_config {}
    }
}

Environment Setup

$ export TF_DATA_DIR=$HOME/.terraform/issue
$ ls -al $HOME/.terraform/issue
total 0
drwxr-xr-x  2 mboudreau  staff   64  3 Oct 16:36 .
drwxr-xr-x  4 mboudreau  staff  128  3 Oct 16:04 ..

Steps to Reproduce

I execute terraform init successfully.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "google" (hashicorp/google) 2.16.0...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

I execute terraform destroy -state=$HOME/.terraform/issue/terraform.tfstate successfully

terraform destroy -state=$HOME/.terraform/issue/terraform.tfstate
Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes


Destroy complete! Resources: 0 destroyed.

Examining the state file

$ cat $HOME/.terraform/issue/terraform.tfstate
{
  "version": 4,
  "terraform_version": "0.12.9",
  "serial": 1,
  "lineage": "fbc81ebb-0d22-2369-48f9-e3ac9afcfdcc",
  "outputs": {},
  "resources": []
}

Then when I re-run terraform init, the error occurs

$ terraform init

Initializing the backend...

Error: Failed to load state: Terraform 0.12.9 does not support state version 4, please update.

Debug Output

The debug output of all of the above commands in included in this Gist https://gist.github.com/marcboudreau/98d840a3d6067d4fb792fa18ab4c9f80

Crash Output

n/a

Expected Behavior

Re-running the terraform init command multiple times should be ok. As per the statement on https://www.terraform.io/docs/commands/init.html#usage

Actual Behavior

An error indicating that the version 0.12.9 of Terraform doesn't support state file version 4.

Additional Context

I used the terraform destroy to simplify the step cases. I found that running any Terraform command that writes out a state file, leads to the same results.

[marcboudreau]

Interestingly, if the -state=... argument is not used on the terraform destroy command, the subsequent terraform init succeeds.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "google" (hashicorp/google) 2.16.0...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform destroy
Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes


Destroy complete! Resources: 0 destroyed.
$ cat terraform.tfstate
{
  "version": 4,
  "terraform_version": "0.12.9",
  "serial": 1,
  "lineage": "5fc2ffa8-c2af-5846-0f5e-f9cbb866bc89",
  "outputs": {},
  "resources": []
}
$ terraform init

Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

[marcboudreau]

⛑ Workaround ⛑

For other users hitting this issue, I've found that if I explicitly specify the local backend as follows:

terraform {
    backend "local" {}
}

And then specify the location that I want to use with the -backend-config argument on the terraform init command like this:

$ terraform init -backend-config="path=$HOME/.terraform/issue/terraform.tfstate"

The state file gets written in the correct location as a version 3 state file and there's no issue.

[marcboudreau]

Actually, I don't think that the above workaround actually works.

[divmgl]

I'm running into this now. As soon as I started running commands Terraform now says Failed to load state: Terraform 0.12.10 does not support state version 4, please update.

[zioalex]

I have the same problem and the proposed workaround doesn't work for me.
I had a previous backed up state file and I have tried to restore it but I get the same error. I am pretty sure that my Terraform version didn't change from the time when i created the backup.

[marcboudreau]

@zioalex Yeah, when I posted that comment, I thought I had thoroughly established that it worked, but the next day, I could get anything that involved a local backend configuration, or using the TF_DATA_DIR variable to change where Terraform stores files, to work.

[arcnmx]

This seems to only occur if both -state= and $TF_DATA_DIR/terraform.tfstate point to the same file. If you're able to rename/move the state file outside of $TF_DATA_DIR, everything seems to start working properly again (or alternatively, never use an explicit state path?)

[marcboudreau]

Thanks @arcnmx. I came to the same conclusion. For now, I've decided to abandon trying to fix the location of the state file, so I am up and running, but for the interest of the wider community, I'll describe what I was originally trying to do and why I wanted to do and why I was trying to fix the location of the state file.

All of the resources that make up our environment are modelled in a Terraform module. We then have a set of Terraform projects that represent our various production/persistent deployments. Each of these projects include the same Terraform module. For those projects, a backend is defined to store the state file in a Google Cloud Storage (gcs). The problem I was trying to solve was creating a _generic_ project that could be used by individual developers to manage their own individual environment. Because these individual environments are short-lived, I wanted to avoid storing the state file in gcs. Instead, I wanted to store the file on the developer's workstation file system. Ideally, I wanted the state file stored outside of the directory where the terraform code lives, since it's a git repository. Since, I couldn't get that to work, the state file is being written into the local git repository for now, but I have added a pattern to the .gitignore file to make sure that it doesn't get tracked. However, there remains the risk of the state file getting deleted if the entire local repository is deleted.

[anyeone]

I hit a similar issue just now. I created a workspace against a remote backend and some terraform files using 0.12.19.

Colleague made a change to a file and executed using 0.12.20 against same remote backend

When I went to terraform init again, I was told I had to upgrade to 0.12.20 because that's what created the state, so I did, with brew upgrade terraform.

Then, I initialized again, made a change to one of the scripts and ran terraform plan
It hung after saying" Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage."

and refreshing the state of about 12 resources in the script.

I had to eventually abort and unlock the state through the web console.
I tried removing my local terraform.tfstate that was created during initialization and reinitializing -- same thing, hung during terraform plan.

So I tried downloading my colleagues (most recent) state from the web console --> then when I tried to initialize I got "Error: Failed to load state: Terraform 0.12.20 does not support state version 4, please update."

I notice when my terraform inits 'from scratch' it is saying it is version 3 state. However, even the states I created with v.0.12.19 earlier today -> when viewed through web console -> show version 4.

So none of this makes sense to me, and as of yet I have not found a workaround.

[anyeone]

I determined in my case that terraform plan was hanging because my azure token had expired, though it was difficult to tell this was the case since there was no message and I could still view my azure account info. This issue is similar: https://github.com/hashicorp/terraform/issues/16271

It doesn't explain the whole state mismatch thing but it got me going again anyway

[daxmc99]

I hit this issue when I ran terraform destory while inside .terraform. It created an empty state file that needed to be removed manually

[thusoy]

I got this while trying to set -state to $TF_DATA_DIR/terraform.tfstate, if I put it in any other directory it works. Same results if setting the path for the local backend instead of using the -state argument.

[universalvishwa]

This issue is till there in 0.13.0 when trying with local state.

[mkyc]

Also present in 0.12.25 version.

[moritlha]

I deleted the .terraform folder and it solved the issue when I ran "terraform init" again.

[gblikas]

⚠️ It's 2020 and still present in Terraform version 0.14.2 (and 0.14.3) ⚠️

% terraform --version
Terraform v0.14.2
+ provider registry.terraform.io/hashicorp/aws v3.20.0
+ provider registry.terraform.io/hashicorp/template v2.2.0
% cat terraform.tfstate
{
  "version": 4,
  "terraform_version": "0.14.2",
  // ...
}

I've tried and received this error when doing terraform init, terraform destroy and terraform plan. _Oddly enough_, this doesn't seem to matter when doing a terraform apply, _even if_ there is a _prexisting terraform.tfstate_ file; this seems consistent with the comments on this thread: https://github.com/hashicorp/terraform/issues/22999#issue-502351952,

I used the terraform destroy to simplify the step cases. I found that running any Terraform command that writes out a state file, leads to the same results.

---

Side Notes

For people using github actions, this is also quite annoying, as the error message:

Error: Failed to load state: Terraform 0.14.2 does not support state version 4, please update.

seems somewhat incorrect; if I pull down my terraform.tfstate file (uploaded as an artifact), it appears to work with my local cli installation of terraform (info above), which matches that of my github action:

- name: Setup Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          terraform_version: 0.14.2

If anyone has any related issues, I would appreciate a link to them. So far, this thread seems to be the only one which really is trying to get at the crux of the problem... 😢