Terraform: Terraform init fails on Linux (works on Mac)

[dak1n1]

Terraform Version

I'm using the master branch at commit 31033001a8e8af6a81ac1e41c3049f57665605d9

go clean -cache
go install

$ terraform version
Terraform v0.14.0-dev

Terraform Configuration Files

Here is my main.tf used as a reproducer:

resource "null_resource" "test" {
}

Debug Output

Short output:

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/null...
- Installing hashicorp/null v3.0.0...
- Installed hashicorp/null v3.0.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Error: Failed to update dependency lock file

Error while writing new dependency lock information to .terraform.lock.hcl:
failed to replace .terraform.lock.hcl with temporary file
/tmp/.terraform.lock.hcl459580865: rename /tmp/.terraform.lock.hcl459580865
.terraform.lock.hcl: invalid cross-device link.

Full debug output here:
https://gist.githubusercontent.com/dak1n1/f6b3d695b7b792d9c9a1d863dd5c0e04/raw/4bff7cc510f6e772f7c6955bbd3a5ad270c0f46c/gistfile1.txt

Crash Output

N/A

Expected Behavior

terraform init should initialize my providers.

Actual Behavior

No providers are initialized. When I go to apply, no schemas can be found.

$ terraform apply --auto-approve

Error: Could not load plugin

Plugin reinitialization required. Please run "terraform init".
...

Steps to Reproduce

1. Compile the provider using the master branch. go install
2. Create a new directory. Copy and paste the above terraform config into main.tf in that directory.
3. terraform init

Additional Context

This is failing consistently on my Linux machine (Fedora 32) and my teammate's Linux machine. However, he said it works using MacOS.

I tested this on many commits, and found I could consistently reproduce the error starting in commit 0009768c7f12e82de9ccefddd8e07cd48058b52c.

References

[thiagoarrais]

I have also been hit by this. Tested with both 0.14.0-beta1 and 31033001.

Some (maybe relevant, maybe not) additional info: my terraform directory is below /home and isn't located in the same device as /tmp. Here's the relevant output from df:

$ df -h /tmp /home
Sist. Arq.      Tam. Usado Disp. Uso% Montado em
/dev/sda1        37G   24G   11G  69% /
/dev/sda2       880G  597G  238G  72% /home

[apparentlymart]

Hi all! Thanks for reporting this.

This does indeed look like a bug. The _intended_ behavior here is for Terraform to create that temporary file in the current working directory, alongside the existing .terraform.lock.hcl if any, and then rename it into place so that the file swap is atomic. It seems like Terraform is incorrectly placing the temporary file in TMPDIR, which then fails because (as @thiagoarrais noted) there's no guarantee that /tmp will be in the same filesystem as the current working directory.

We'll investigate why exactly Terraform is putting the temporary file in the wrong place and fix it for the next beta release. In the meantime, you may be able to work around this by temporarily setting the TMPDIR environment variable to some path that is on the same filesystem as the current working directory when you run Terraform.

[pisto]

cc
This breaks terraform on all distros where /tmp is on tmpfs (see on Fedora)

[alexsomesan]

I can confirm that the workaround of setting TMPDIR explicitly to be on the same FS works well.

 terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/kubernetes-alpha...
- Installing hashicorp/kubernetes-alpha v9.9.9...
- Installed hashicorp/kubernetes-alpha v9.9.9 (unauthenticated)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Error: Failed to update dependency lock file

Error while writing new dependency lock information to .terraform.lock.hcl:
failed to replace .terraform.lock.hcl with temporary file
/tmp/.terraform.lock.hcl908089146: rename /tmp/.terraform.lock.hcl908089146
.terraform.lock.hcl: invalid cross-device link.

 TMPDIR=$PWD terraform init 

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/kubernetes-alpha...
- Installing hashicorp/kubernetes-alpha v9.9.9...
- Installed hashicorp/kubernetes-alpha v9.9.9 (unauthenticated)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

[DrStrangepork]

This fails when /tmp is not on the same filesystem as $PWD.

[apparentlymart]

Thanks for reporting this, everyone! I merged a fix and backported it to the v0.14 branch, so the fix should be included in the forthcoming v0.14-beta2 release.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.