Terraform: Dynamically reference terraform remote state output variables

[sfresher]

Terraform Version

Terraform v0.11.7
+ provider.aws v1.31.0
+ provider.template v1.0.0

Terraform Configuration Files

data "terraform_remote_state" "vpc" {
  backend = "s3"
}


resource "aws_security_group" "allow_443" {
  name        = "allow_443_${var.vpc_tag}"
  vpc_id      = "${data.terraform_remote_state.vpc.vpc_var.tag_id}"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags {
    Name = "allow-443-${var.vpc_tag}"
  }
}

The terraform remote state produces two output variables, vpc_VPCA_id and vpc_VPCB_id respectively.

Steps to Reproduce

We would like a way to dynamically reference remote state outputs. e.g.
When "${var.vpc_tag}" == "VPCA", then
"${data.terraform_remote_state.vpc.vpc_var.tag_id}" can be resvoled as "${data.terraform_remote_state.vpc.vpc_VPCA_id}"

[mildwonkey]

Hi!

Please note that we use GitHub issues for tracking bugs and enhancements rather than for questions. While we may be able to help with certain simple problems here it's generally better to use one of the community forums where there are far more people ready to help, whereas the GitHub issues here are generally monitored only by our few core maintainers.

Having said that, I think what you are looking for is a conditional statement. Something along the lines of:

resource "aws_security_group" "allow_443" {
  name        = "allow_443_${var.vpc_tag}"
  vpc_id      = "${ var.vpc_tag == "VPCA" ? data.terraform_remote_state.vpc.vpc_VPCA_id : data.terraform_remote_state.vpc.vpc_var.tag_id }"
  ...
}

Hope this helps!

[sfresher]

Thank you. I also use the conditional statement as a workaround. However, it's not really a neat solution if the variables can have lots of values (VPCA, VPCB, VPCC, VPCD ... VPCZ). It'll end up with a very long list of nested conditions, and also have to update the code each time if there is a new value/conditional.

So I think it'll be nice to have a feature to dynamically reference terraform remote state output variables.

[apparentlymart]

Hi @sfresher!

The most general answer to this problem is to replace your separate output values with a single output that contains a map value:

output "vpc_ids" {
  value = {
    VPCA = "${aws_vpc.a.id}"
    VPCB = "${aws_vpc.b.id}"
    VPCC = "${aws_vpc.c.id}"
    # etc, as required
  }
}

Then in your other configuration you can reference these using the index operator:

resource "aws_security_group" "allow_443" {
  name        = "allow_443_${var.vpc_tag}"
  vpc_id      = "${data.terraform_remote_state.vpc.vpc_ids[var.vpc_tag]}"
  ...
}

[sfresher]

@apparentlymart Excellent! Thank you. That works. Sorry that I wasn't aware of that you can output a map value.

On the other side, it'll be awesome to add support for variables referrences in the names of a resource/data. e.g.

data "aws_security_group" "group_for_a" {
...
}

data "aws_security_group" "group_for_b" {
...
}

# can use var.mname is to reference to above data  sources dynamically
"${data.aws_security_group.group_for_var.mname.id}"

[apparentlymart]

@sfresher I think you're describing the use-case of a forthcoming planned feature being discussed in #17179, where you can make a resource (or data) block itself behave as a map, as long as the configurations are similar enough to combine together:

# Not yet implemented, and details may change before release

data "aws_security_group" "allow_443" {
  # Create one instance of this data resource for each element of the map
  for_each = data.terraform_remote_state.vpc.vpc_ids

  # For each instance, use the value from the map (the VPC id)
  vpc_id = each.value
  tags = {
    Name = "allow_443"
  }
}

...
  something = data.aws_security_group.allow_443[var.vpc_tag].id
...

With that said, in this case if you already know which of the VPCs you want (in var.vpc_tag) you could just request it directly and not fetch the other ones at all:

data "aws_security_group" "allow_443" {
  vpc_id = data.terraform_remote_state.vpc.vpc_ids[var.vpc_tag]
  tags = {
    Name = "allow_443"
  }
}

...
  something = data.aws_security_group.allow_443.id
...

(The above examples are using the first-class expressions syntax coming in Terraform 0.12, but the second example should work in Terraform 0.11 too if you wrap the two expressions in ${ ... } interpolation delimiters.)

It is not possible to dynamically generate the resource names _themselves_ in expressions, because Terraform needs to know all of the inter-resource dependencies _before_ evaluating any expressions in order to construct the dependency graph that is used to evaluate them. Using maps makes the map as a whole behave as a node in the dependency graph, thus avoiding that problem.

[mildwonkey]

It _looks_ like this can be closed, but please yell here if I'm mistaken and I'll reopen!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.