Terraform: Override path for .terraform.lock.hcl

[ademariag]

Current Terraform Version

terraform --version
Terraform v0.14.2

Use-cases

I am one of the founders of the https://kapitan.dev project and we use Kapitan to generate (json) terraform config files using jsonnet.

The recent introduction of the .terraform.lock.hcl breaks our workflows (and any other terraform user of Kapitan) because the file is rewritten every time we compile a target.

Proposal

If you could provide a way to store the file under another path (still in git as you suggest, but a path where it is not deleted) that would be awesome.

For example, currently terraform files could be under:

compiled/
  production/
    terraform/
      .terraform.lock.hcl
      ...other hcl files...

Ideally we would like to pass the correct path using an env variable, i.e. TERRAFORM_LOCK_FILE_PATH

export TERRAFORM_LOCK_FILE_PATH=$(git rev-parse --show-toplevel)/terraform/$TARGET_NAME/

We could then store the files somewhere else:

terraform/   <---- not destroyed on compile
  production/
      .terraform.lock.hcl
compiled/
  production/
    terraform/
      ...other hcl files...

[stazz]

I have similar need for this, but for different reason:

Within the infra pipelines we are running Terraform from, we are exclusively doing it using Docker (among many things, using Docker allows for easy tool dependency management and security model in our pipelines). It is extremely important for us to verify that pipelines do not introduce any extra changes to the repository extracted at certain commit by the pipeline. This is why we pass our TF code as readonly Docker volume mount to the TF container - certainly running TF shouldn't do any changes to our TF code, right?

Starting with 0.14 version and introduction of this dependency lock file, the terraform init command fails to write the dependency lock file:

Error while writing new dependency lock information to .terraform.lock.hcl:
cannot create temporary file to update .terraform.lock.hcl: open .terraform.lock.hcl217706229: read-only file system.

While dependency lock file as an idea is awesome, I agree with @ademariag that it should be a way to customize the actual location of the file within the file system.

[bigpapabear805]

Can someone please do something it's starting to hurt my lower back

[starkers]

Yes please, the clash with the lock file has been extremely painful for my team and we've had to look to revert terraform

[apparentlymart]

It seems like there are at least two different situations represented here:

• When using Terraform with code generation tools rather than hand-written configuration, it's better to keep the lock file with the input files rather than the generated files.
• When the working directory is on a read-only filesystem, it's inconvenient that Terraform wants to rewrite the lock file even if the new content would be identical anyway, because there are no new provider dependencies.

I'm considering these separately because it seems like the solutions for them are likely to be separate too. For the second, it would be best if no special Terraform settings were required to run in a read-only directory, so I'd prefer not to address that by forcing to relocate the file from the default location.

[thusoy]

I'm having the same issue with terraform running on a read-only filesystem.

Couple options that would work for me at least:
1) Envvar or cli flag to disable updating the lockfile
2) Gracefully ignore updating the lockfile on permission denied (or write errors in general)
3) Specify path to lockfile. I'm actually being hit by both this issue and https://github.com/hashicorp/terraform/issues/27158 (we have the same setup with a directory of symlinks for each environment), if I could specify the path I wouldn't need the lockfile symlink).

[toadjaune]

I have a third use-case :

I have a project with 3 different environments (test, preprod, prod), and each of them is in a separate directory/tfstate. Most of the comfig is shared via a module.

I'd like to have single lockfile for all three environments, in order to guarantee that the config validated on test environment is applied similarly on production.

(I think workspaces could allow us to work around the issue in this specific usecase, but due to various constraints, we chose not to go down this path when implementing this project)

This usecase is actually pretty close from the one when terrafom is used with generation tools.
Any of the mentioned solutions (cli argument, envvar, configuration directive in manifests) would work here (with a preference for the latter, as we wouldn't want a user to forget specifying the lockfile path)

[lorengordon]

1. Envvar or cli flag to disable updating the lockfile

I'm using terraform-bundle, and would actually just like to disable the lock-file generation entirely (not just updating it).