Terraform: Plan errors when it uses 'data.terraform_remote_state.foo' that is yet to be created/applied

[stephenchu]

Terraform Version

0.8.7 and 0.9.0-beta
(although I believe it all started when Terraform debuted "data" data "terraform_remote_state" resources)

Affected Resource(s)

• data "terraform_remote_state"

Terraform Configuration Files

Suppose I have 2 different tfstate files in S3: one for creating my vpc, and the other for creating a security group inside of it:

vpc/main.tf:

provider "aws" {
  region = "${var.region}"
}

resource "aws_vpc" "vpc" {
  cidr_block           = "${lookup(var.cidr_blocks, var.region)}"
  enable_dns_support   = true
  enable_dns_hostnames = true
}

output "vpc_id" { value = "${aws_vpc.vpc.id}" }

security-group/main.tf:

provider "aws" {
  region = "${var.region}"
}

data "terraform_remote_state" "vpc" {
  backend = "s3"
  config { ... }
}

resource "aws_security_group" "ssh" {
  name        = "SSH"
  vpc_id      = "${data.terraform_remote_state.vpc.vpc_id}"
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Expected Behavior

Terraform used to allow me to do terraform plan for my vpc and then immediately followed by terraform plan on my security-group, without having to first apply my vpc plan.

This allows me to visually inspect the plan for all components in my dependency tree. My code here only shows a 2-deep tfstate dependency tree, but imagine a tree of: kafka > zookeeper > common-security-groups -> vpc. Having terraform plan work only dependency N if-and-only-if N-1 has already applied seems to conflict somewhat with my understanding of multiple tfstate files usage.

I believe this was the behavior before data "terraform_remote_state" was introduced.

Actual Behavior

Now, if I do both terraform plan on vpc then immediately on security-group, the second plan will fail with:

data.terraform_remote_state.vpc: Refreshing state...
Error running plan: 1 error(s) occurred:

* aws_security_group.ssh: 1 error(s) occurred:

* aws_security_group.ssh: Resource 'data.terraform_remote_state.vpc' does not have attribute 'vpc_id' for variable 'data.terraform_remote_state.vpc.vpc_id'

I also tried 0.9-beta and the remote state revamp does not yield different result.

References

I think this enhancement (#8001) is also trying to illustrate the same idea, but because it is marked as "enhancement" I don't think it captures the fact that this used to work in previous versions of Terraform.

[apparentlymart]

Hi @stephenchu! Sorry this isn't behaving the way you expected.

The primary motivation for the data source idea was to allow Terraform to fetch data to use during planning, but as you have seen this means that the data must be available at that point out else Terraform can't fetch it.

The assumption is that the configurations would be applied in order, with each using the result of the previous. However, you should be able to get the result you were expecting by passing -refresh=false to the plan command, which will prevent Terraform from trying to read any data -- including the remote state -- before making the plan.

In this case the plan should include an entry for reading the remote state, so you can see that it wasn't done already. Note though that this will not work if you are using data source results in places that Terraform needs to be able to resolve during plan, which includes provider arguments and (in 0.9) the count argument on resources.

[stephenchu]

No apologies accepted because you do not need to apologize for anything @apparentlymart! Any help is much appreciated here, and thanks for such prompt response.

I just tried the -refresh=false flag for plan. It however is (on 0.9-beta) still giving me the same error message.

[apparentlymart]

Oh, I think this is a slightly different issue than I thought... there is some special behavior for the remote state data source to allow it to dynamically expose values of different types, which might be making this act differently than for a "normal" static resource attribute.

I don't have a workaround ready for that, I'm afraid. :(

[ankitkl]

Guys, Any update i stuck at same stage.

provider "aws" {
  region = "${var.region}"
}
data "terraform_remote_state" "vpc" {
  backend = "s3"
  config {
     bucket     = "my-terraform-ojec"
      key        = "phase6/terraform.tfstate"
      region     = "ap-southeast-1"
 }
}
resource "aws_subnet" "foo" {
   vpc_id = "${data.terraform_remote_state.vpc.vpc_id}"
   cidr_block = "10.0.19.0/24"
}


#############Output#######
output "vpc_id" {
  value = "${aws_vpc.default.id}"
}

[ankitkl]

Hello guys its working .... here is my configuration.
As recently added to Root Outputs Only on https://www.terraform.io/docs/providers/terraform/d/remote_state.html, any module outputs need to be explicitly exposed.

VPC module root folder output.tf
#############Output#######
output "awsvpc2" {
  value = "${aws_vpc.default.id}"
}

data "terraform_remote_state" "vpc" {
  backend = "s3"
  config {
     bucket     = "my-terraform-ojec"
      key        = "phase6/terraform.tfstate"
      region     = "ap-southeast-1"
 }
}
resource "aws_subnet" "foo" {
   vpc_id = "${data.terraform_remote_state.vpc.awsvpc2}"
   cidr_block = "10.0.19.0/24"
}

[stephenchu]

@ankitkl Your HCL configs are virtually the same between your last two posts (barring a awsvpc2 output rename). It's also not clear that your VPC module uses any modules inside.

In my original post, my vpc/main.tf and security-group/main.tf don't use any Terraform modules at all. My output (vpc_id) is at root level. I don't see my reported problem is due to "root outputs only."

However, from your page, I did find this:

If this root level output hadn't been created, then a remote state resource wouldn't be able to access the value output on the module.

... which is exactly my grief on this issue: I used to be able to plan without having to first applyied _all_ of the dependent infrastructure.

[felipe-vvoosh]

I'm having the same issue... Is this fixable?

[felipe-vvoosh]

@stephenchu, did you manage to find a workaround? Thanks.

[bengaywins]

@apparentlymart or @stephenchu are there any intentions/plans to change this behavior for running plan?

[xxthegonzxx]

👍

[hawkesnc]

👍

[SandyKapil]

awsvpc2

How do you figure out what are the atrributes which you can refer it form your state??
because I keep getting this error:

Resource 'data.terraform_remote_state.vpc' does not have attribute 'type' for variable 'data.terraform_remote_state.vpc.type'

I have tried few other combinations as well like subnet_id and vpc_id but nothing is working and it is getting a bit urgent now to sort it out as it's been a while am working on it..

any small help would be highly appreciated..

thanks in advance.

[adamrt]

@stephenchu I was having this issue today. To fix the issue, I added the workspace attribute to the data block and it fixed it. For me, I have a staging and prod workspace, in addition to the default workspace, which I don't use. I think by default it was attempting to use the default workspace.

This seems like the workspace functionality hasn't been integrated into the terraform_remote_state feature of terraform yet.

Anyway, I hope this helps.

data "terraform_remote_state" "persistent" {
  backend = "s3"
  workspace = "${terraform.workspace}"
  config {
    bucket  = "bucketname"
    key     = "terraform.tfstate"
    region  = "us-east-1"
  }
}

[PicoSushi]

Same issue here:

# remote output.tf
output "vpc_id" {
  value = "${aws_vpc.my_vpc.id}"
}

# main file
data "terraform_remote_state" "remote" {
  backend = "s3"
  config {
    bucket = "terraform-state"
    key    = "shared/terraform.tfstate"
    region = "ap-northeast-1"
  }
}

locals {
  vpc_id = "${data.terraform_remote_state.immutable.output.vpc_id}"
}

Causes * local.vpc_id: local.vpc_id: Resource 'data.terraform_remote_state.remote' does not have attribute 'output.vpc_id' for variable 'data.terraform_remote_state.remote.output.vpc_id' .

Version:
Terraform v0.11.11

[adamrt]

@PicoSushi are you using workspaces? If so, check out my previous comment that fixed it for me.

[PicoSushi]

@adamrt Yes, I'm using workspace for local terraform and not using for remote one.

And, I've tried specifying these pattern, but they returned same error message as I wrote.
workspace = "${terraform.workspace}"
workspace = "default"
(no specify)

[adamrt]

@PicoSushi

vpc_id = "${data.terraform_remote_state.immutable.output.vpc_id}"

You are referencing immutable here but are using the name remote in the data block declaration. Might just be a copy/paste issue when commenting, otherwise change it to:

data "terraform_remote_state" "immutable" {...}

[PicoSushi]

Ah, that was copied from old commit. Already that line was changed to vpc_id = "${data.terraform_remote_state.remote.output.vpc_id}", but did not solved.

[kchugalinskiy]

@PicoSushi

tl;dr

Try vpc_id = "${data.terraform_remote_state.remote.vpc_id}"

long explanation

Seems like documentation is totally wrong for terraform 0.11.13, as in code one may see the line: d.UnsafeSetFieldRaw(key, val) which leads us here.

p.s.

I was surprised that documentation is about non-released 0.12 version. Seems like an operational issue. In my opinion explicit documentation version selection should be added, because problems like this may drive deep into the madness

[ebalakumar]

@PicoSushi

tl;dr

Try vpc_id = "${data.terraform_remote_state.remote.vpc_id}"

long explanation

Seems like documentation is totally wrong for terraform 0.11.13, as in code one may see the line: d.UnsafeSetFieldRaw(key, val) which leads us here.

p.s.

I was surprised that documentation is about non-released 0.12 version. Seems like an operational issue. In my opinion explicit documentation version selection should be added, because problems like this may drive deep into the madness

I added remote . Its not working .

[kchugalinskiy]

@ebalakumar my point is to remove output, not to add remote. By the way, I didn't see your specific case, I cannot be more precise.

[PicoSushi]

@kchugalinskiy
I've tried vpc_id = "${data.terraform_remote_state.remote.output.vpc_id}" but error occured as below.

* local.vpc_id: local.vpc_id: Resource 'data.terraform_remote_state.remote' does not have attribute vpc_id' for variable 'data.terraform_remote_state.remote.vpc_id'

Feeling odd.

note: Terraform updated

Terraform v0.11.13

• provider.aws v1.57.0
• provider.template v2.0.0

[kchugalinskiy]

@PicoSushi

It's not a misprint, you have to try vpc_id = "${data.terraform_remote_state.remote.vpc_id}" instead of vpc_id = "${data.terraform_remote_state.remote.output.vpc_id}". Documentation lies you :-) I've got the same issue on my project

[PicoSushi]

oh, I've confused. But, fixed to vpc_id = "${data.terraform_remote_state.remote.vpc_id}" and same error caused.

[vonglasow]

Hello, I have tried all solutions proposed here and cannot succeed to get the value. I have created a small project to try to reproduce the behavior. You can found the code here https://github.com/vonglasow/terraform-remote-state

If you have any idea to fix it. It will be really useful.

[GreenyMcDuff]

I saw @adamrt's , post, but that didn't fix it for me. The only thing I could do to get it working was:

locals {
  environment = "${terraform.workspace}"
}

data "terraform_remote_state" "this" {
  backend   = "s3"
  # workspace = "${local.environment}"

  config {
    bucket = "my-unique-bucket"
    key    = "${local.environment}/my/key/terraform.tfstate"
    region = "${var.aws_region}"
  }
}

Which feels horrible :( but works :)

[vonglasow]

@GreenyMcDuff thanks for your example, it works for me too. I have just change it to use directly terraform.workspace on the key but it works. I have fix it in my repo and now we have a working example. https://github.com/vonglasow/terraform-remote-state

[adamrt]

I misunderstood this issue from the beginning, sorry for that.

I had a different issue than the original poster. His issue was with planning, while referencing remote data which didn't exist yet. My issue was workspace related. How can it get remote data that doesn't exist?

Unless I'm still misunderstanding, it seems like this _should_ fail. There is no data to retrieve, therefore planning cannot proceed.

[nitramiz]

IIUC, the thing that usually saves me in such situations is adding an empty default output for the specific remote_state:

output "internal_ip" {
  value = google_compute_address.internal[0].address
}

root output:

output "internal_ip" {
  value = module.test.internal_ip
}

data "terraform_remote_state" "def" {
  backend   = "gcs"
  workspace = terraform.workspace

  config = {
    bucket = "name"
    prefix = "dir"
  }

  defaults = {
    internal_ip = ""
  }
}

[teamterraform]

Hi folks!

We have added some guidelines situations such as this one. Please see our best practice documentation on data-only module composition.

There have been many changes to terraform and data sources specifically since this issue was originally opened, and the new conversations seem to be about different use cases, so I am going to close it. Anyone experiencing a similar issue should open a new GitHub issue and fill out the issue template. Thank you!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.