Terragrunt: Terragrunt does not create local backend

Created on 13 Jun 2017 · 24Comments · Source: gruntwork-io/terragrunt

We are using Terragrunt to create the infrastructure for an AWS service. Our setup scripts allow us to configure a remote S3 backend or just use a local one (for instance, to do a quick test that is not supposed to be shared with other developers).

While everything works fine when we use the S3 backend, Terragrunt seems not to be capable of initializing the local backend.

We are using Terraform 0.9.8 and Terragrunt 0.12.20.

main.tf contains the following configuration section for the backend:

terraform {
  backend "local" {}
}

And terraform.tfvars just contains:

terragrunt = {
}

When running

terragrunt init
terragrunt get
terragrunt plan

the following error is obtained:

[terragrunt] [SomePath\terraform] 2017/06/13 11:11:01 Running command: terraform --version
[terragrunt] 2017/06/13 11:11:01 Reading Terragrunt config file at SomePath/terraform/terraform.tfvars
[terragrunt] 2017/06/13 11:11:01 Running 'init' manually is not necessary: Terragrunt will call it automatically when needed before running other Terraform commands
[terragrunt] [SomePath\terraform] 2017/06/13 11:11:01 Running command: terraform --version
[terragrunt] 2017/06/13 11:11:01 Reading Terragrunt config file at SomePath/terraform/terraform.tfvars
[terragrunt] 2017/06/13 11:11:01 Running command: terraform get
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: git::https://SomeUrl/scm/ct/terraformmodules.git?ref=v0.0.6
Get: file://SomePath/terraform/.terraform/modules/13a6f4a5ee3a99f75f6182d39132639f/modules/n_list
Get: file://SomePath/terraform/.terraform/modules/fbeb1acb7f91b66962a4d6b122b47dce/modules/n_list
Get: file://SomePath/terraform/.terraform/modules/52edc4f80b92e0ca511331074ba5648a/modules/n_list
Get: file://SomePath/terraform/.terraform/modules/d8d44e535eee15bb64a069f15f76dceb/modules/n_list
Get: file://SomePath/terraform/.terraform/modules/0295f53ae0c7fa1e039102d1173dd61d/modules/n_list
[terragrunt] [SomePath\terraform] 2017/06/13 11:11:02 Running command: terraform --version
[terragrunt] 2017/06/13 11:11:02 Reading Terragrunt config file at SomePath/terraform/terraform.tfvars
[terragrunt] 2017/06/13 11:11:02 Running command: terraform plan
Backend reinitialization required. Please run "terraform init".
Reason: Initial configuration of the requested backend "local"

The "backend" is the interface that Terraform uses to store state,
perform operations, etc. If this message is showing up, it means that the
Terraform configuration you're using is using a custom configuration for
the Terraform backend.

Changes to backend configurations require reinitialization. This allows
Terraform to setup the new configuration, copy existing state, etc. This is
only done during "terraform init". Please run that command now then try again.

If the change reason above is incorrect, please verify your configuration
hasn't changed and try again. At this point, no changes to your existing
configuration or state have been made.

Failed to load backend: Initialization required. Please see the error message above.
[terragrunt] 2017/06/13 11:11:03 exit status 1

Is this a bug, or are we not configuring the local backend properly?

The error occurs when cloning a fresh copy of the repository (i.e., no .terraform directory is present).

enhancement help wanted

Source

betabandido

👍2

All 24 comments

Ah, I think the issue is that Terragrunt doesn't try to configure a backend unless you have a remote_state block defined. See if defining an empty block does the trick:

terragrunt = {
  remote_state {
    backend = "local"
    config {}
  }
}

brikis98 on 13 Jun 2017

Oh, that was exactly the issue. I assumed remote_state shouldn't be used (as I wanted to use the local backend). But, after adding the empty block as you suggested, it is now working. Thanks!

It might be worth to add a note to the documentation mentioning the need to use the remote_state configuration item even for a local backend.

betabandido on 13 Jun 2017

Yea, in Terraform 0.9, the naming convention changed from "remote state" to "backends". We have not updated our naming convention in Terragrunt yet. Readme PR is very welcome :)

brikis98 on 13 Jun 2017

Would be great to also change the keyword supported by Terragrunt from remote_state to backend for consistency with Terraform.

conorgil on 14 Jun 2017

Hi all,

I have the same issue. Even if I specify the following:

terragrunt = {
  remote_state {
    backend = "local"
    config {}
  }
}

No .tfstate file will be created.

I am running the following:

terragrunt version v0.12.25
Terraform v0.9.11

Thanks

alternico on 14 Aug 2017

Using latest v0.13.2 terragrunt and terraform v0.10.4
terragrunt will not init/create the local backend in the specified path.

remote_state { backend = "local" config { path = "/somelocalpath/remote-state/na4-perfstage1" } }

Pretty much no matter what I do I can't get the local file to exist. I've looked for terraform issues around this - and didn't see any, however, I think it's a terraform problem, as I see the command [terragrunt] 2017/09/08 18:01:55 Running command: terraform init -backend-config=path=mypath/blah

jedierik20 on 9 Sep 2017

👍1

@alternico and @jedierik20 In your Terraform code (the .tf files) you still need an empty block like this:

terraform {
  backend "local" {}
}

Without that, even if you specify the Terragrunt remote_state config, it won't do anything, as Terraform itself will ignore it.

brikis98 on 9 Sep 2017

I am running into the same problem, not wanting to use remote state as we are on OpenStack not AWS

cyberious on 26 Oct 2017

As @brikis98 mentioned above --- did you add the empty block at the top of your TF files?
terraform {
backend "local" {}
}

I had to do this even when I was using the local state.

jedierik20 on 26 Oct 2017

yes and the remote block in my terragrunt config section

cyberious on 26 Oct 2017

when I have the remote backend in the terraform.tfvars either parent or include source it causes an error.

[terragrunt] 2017/10/27 12:46:06 runtime error: invalid memory address or nil pointer dereference

I am using the pattern of folders with terraform.tfvars and including a parent folder with the common variables

_child_

```terragrunt = {

include {
path = "${find_in_parent_folders()}"
}

}


_parent_

terragrunt = {

terraform {

source = "${path_relative_from_include()}/../sources/generic"

extra_arguments "common_vars" {
  commands           = [
    "apply",
    "plan",
    "import",
    "push",
    "refresh",
    "plan",
    "destroy"
  ]

  required_var_files = [
    "${get_tfvars_dir()}/../common.tfvars"
  ]
}

}
}
```

cyberious on 27 Oct 2017

Please set the TERRAGRUNT_DEBUG to true so we can see the stack trace from your error.

brikis98 on 28 Oct 2017

This is not working for me too, and the steps above are not really clear too. My understanding so far is that, to use a local backend, I have to:
1) Define a terraform { backend "local" {} } block in my TF files
2) Add a

terragrunt = {
  remote_state {
    backend = "local"
    config {}
  }
}

block in a terraform.tfvars and... something else?

ColOfAbRiX on 12 Dec 2017

@ColOfAbRiX Looks mostly correct, except you should set a path in your terragrunt config {} block.

brikis98 on 12 Dec 2017

Thanks @brikis98. Anyway I can't make it working. I'm migrating an existing infrastructure to terragrunt and at the moment, to learn, I'm using all local directories. I'm running terraform v0.11.1 and terragrunt v0.13.23 on CentOS 7.3. The directory structure is as you recommend:

.
├── modules
│   └── network
│       ├── inputs.tf
│       └── main.tf
└── live
    ├── test
    │   └── network
    │       └── terraform.tfvars
    └── terraform.tfvars

I then have the terraform { backend "local" {} } in ./modules/network/main.tf and this code in the parent ./live/terraform.tfvars:

terragrunt {
  remote_state {
    backend = "local"
    config { path = "/home/user/terraform.tfstate" }
  }
}

and this in the module ./live/test/network/terraform.tfvars:

terragrunt {
  terraform {
    source = "../../../modules//network"
  }
  include {
    path = "${find_in_parent_folders()}"
  }
}

And this is what I get:

$ TERRAGRUNT_DEBUG=true terragrunt apply
[terragrunt] [/home/user/terraform/live/test/network] 2017/12/12 18:42:46 Running command: terraform --version
[terragrunt] 2017/12/12 18:42:46 Reading Terragrunt config file at /home/user/terraform/live/test/network/terraform.tfvars
[terragrunt] 2017/12/12 18:42:46 Cleaning up existing *.tf files in /home/user/.terragrunt/uWoy5G8HnUMi9iTnxFm02FQuNks/7VUZV-UT9PfkDWR3uowb8V2xUoI
[terragrunt] 2017/12/12 18:42:46 Downloading Terraform configurations from file:///home/user/terraform/components into /home/user/.terragrunt/uWoy5G8HnUMi9iTnxFm02FQuNks/7VUZV-UT9PfkDWR3uowb8V2xUoI using terraform init
[terragrunt] 2017/12/12 18:42:46 *json.SyntaxError unexpected end of JSON input
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/remote/terraform_state_file.go:74 (0x4cb64c)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/remote/terraform_state_file.go:66 (0x4cb55f)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/remote/terraform_state_file.go:51 (0x4cb2aa)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/remote/remote_state.go:68 (0x4c849c)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:473 (0x45e40c)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:283 (0x45ce78)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:256 (0x45cd6a)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:402 (0x45d77e)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/download_source.go:363 (0x460fd9)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/download_source.go:92 (0x45f54b)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/download_source.go:54 (0x45f0ff)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:212 (0x45c8a3)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:196 (0x45c740)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:177 (0x45c42c)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/vendor/github.com/urfave/cli/app.go:502 (0x49cc24)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/vendor/github.com/urfave/cli/app.go:268 (0x49a95d)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/main.go:20 (0x4010c4)
/usr/local/go/src/runtime/proc.go:183 (0x42a864)
/usr/local/go/src/runtime/asm_amd64.s:2086 (0x458e51)

[terragrunt] 2017/12/12 18:42:46 Unable to determine underlying exit code, so Terragrunt will exit with error code 1

ColOfAbRiX on 12 Dec 2017

That error suggests your terraform.tfstate file has invalid JSON. What does it contain?

brikis98 on 12 Dec 2017

Yes, the JSON file was valid. I also tried without a tfstate. But now I think I fixed it. I had to remove the ~/.terragrunt directory and now it seems it works. I'll do more investigation to confirm.

ColOfAbRiX on 13 Dec 2017

Hi,
for me, still does not work.
This is what I see in the logs:

2017/12/22 17:01:27 Initializing remote state for the local backend
2017/12/22 17:01:27 Running command: terraform init -backend-config=path=/data/terraform/terraform-infra_live/vcenter/terraform.tfstate -from-module=file:///data/terraform/terraform-infra_modules/vcenter/template /home/setup/.terragrunt/DX9GJdzVaSoEUGOIi-P2RsYstFs/6xikLtVRGymAZdhW17xxh85vWGQ

alternico on 22 Dec 2017

Something I'd like to clarify if someone has a moment. I'm setting up a new project with terragrunt. Initially I want to use local state files, but will eventually I will switch to remote state. So I'm trying to modularize the stuff required to make local state work so that I can "swap" it out when ready to do so.

However, the problem that I'm coming across is that the only way to make it work is to have state related changes in both the "live" and the "modules/components" repo for the project.

Am I missing something. Is there a way to get the

terraform {
  backend "local" {}
}

part OUT of the module/component and into the live repo so that when a switch from local state to remote state is done it is all done in the "live" repo instead of having to make coordinating changes in multiple places?

Right now what I'm doing feels dirty and actually prohibits me from having a mix of remote and local state on a particular project (which I admittedly would probably never do)

FYI, I'm successfully using extra_arguments to generate the -state argument and put the state files into a separate directory of the "live" repo which makes for a nice "remote'ish" solution for when you want to experiment with something but still want to keep your .tfstate files apart from your .tf files.

john-mcgowan-wowza on 12 Jan 2018

This is blocking me from using terragrunt. My infrastructure modules shouldn't be hard-linked to a specific backend type. Some live infrastructure scenarios will need remote and some will need local. (e.g. We shouldn't be spinning S3 buckets for each person's local minikube cluster).

Looking for workarounds.

josdotso on 18 Feb 2019

👍1

Interested in this as well with the same use-case as @john-mcgowan-wowza

As we do our initial development, I want to specify backend "local" {} but transition to "s3" in live; having to touch my module repositories to do that feels very wrong.