Terraform: data resource attributes cannot be interpolated into count

[MarsuperMammal]

I am attempting to use the new data source for aws_availability_zone to inform values of count to create more flexible modules for creating networks in different regions. Unfortunately this doesn't appear to work, and I'm curious if it's a known/intended behavior, and to understand why these resources can't interact as I'd expect.

Terraform Version: 0.7.0-rc2

Affected Resource(s)

• data.aws_availability_zones
• aws_*
  

  Code

provider "aws" {
  region = "us-east-1"
}

data "aws_availability_zones" "available" {}

resource "aws_instance" "web" {
  count = "${length(data.aws_availability_zones.available.instance)}"
  ami = "ami-408c7f28"
  instance_type = "t2.micro"
  tags {
    Name = "HelloWorld"
  }
}

Result

Errors:

  * 1 error(s) occurred:

* module root: 1 error(s) occurred:

* aws_instance.web: resource count can't reference resource variable: data.aws_availability_zones.available.instance

Reference Test

provider "aws" {
  region = "us-east-1"
}

data "aws_availability_zones" "available" {}

output "azs" {
  value = "${length(data.aws_availability_zones.available.instance)}"
}

Test Result

data.aws_availability_zones.available: Refreshing state...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
azs = 4

[apparentlymart]

Hello! :)

This behavior is unfortunately intended for the moment. The ability to use data resource attributes in count didn't make it into 0.7, but there are tickets for this and we would like to support it in future... just that it requires some work on Terraform's internals to make it work.

[MarsuperMammal]

Is there a known mechanism by which I could work around the issue? I had tried the obvious depends_on clause, but lacking a real understanding of what functional order is causing the issue I haven't been able to think of any more suitable options.

My only thoughts would be possibly encapsulating the data source in a module, and passing it's response as an output used by the resources, but I'm not sure if that would work, and haven't yet tested it.

Outside of that, I would have to just do the legwork outside of terraform and render values in, basically replicated the data source outside of terraform, which obviously makes me sad.

Can you at least provide references to tickets or issues that discuss this internal behavior?

[apparentlymart]

@MarsuperMammal it looks like what you were trying to do here is create one EC2 instance for each availability zone available to your account.

Assuming I understood correctly, a possible workaround could be to use an autoscaling group, in which case Terraform sees only one resource (the autoscaling group) and AWS itself creates the appropriate number of instances. Here's an example of what I mean, though I didn't test this yet since I don't have my AWS credentials to hand at the moment:

provider "aws" {
  region = "us-east-1"
}

data "aws_availability_zones" "available" {}

resource "aws_launch_configuration" "web" {
  name = "web"
  image_id = "ami-408c7f28"
  instance_type = "t2.micro"
}

resource "aws_autoscaling_group" "web" {
  launch_configuration = "${aws_launch_configuration.web.name}"
  desired_capacity = "${length(aws_autoscaling_group.available.names)}"
  availability_zones = ["${aws_autoscaling_group.available.names}"]
  # ... and any other settings you need ...
}

The immediate downsides of the above configuration are:

• AWS autoscaling _attempts_ to balance instances across AZs, but doesn't guarantee that the result will be exactly evenly balanced. Thus you might end up with two instances in one AZ and none in another.
• The above assumes you're using EC2-Classic and not VPC. For VPC you'd actually need a list of subnets rather than availability zones, and we don't have a resource for that. (Most of our data sources are for single items rather than lists, because right now Terraform's handling of dynamic resource sets is rather limited, as this very issue illustrates.)

Dynamic count (that is, a count that isn't known until after we've built and walked the configuration graph) was originally intended as part of the data sources feature, but it proved more complex to implement than originally thought and so it was deferred for now.

The reason for the complexity is that there is a bit of a chicken-and-egg problem in that we currently do the "expand" operation (turning a single configuration block into a set of objects based on the count argument) during graph construction, which happens before we "walk" the graph. Data source fetching happens during the "refresh" phase, which is run on an already-constructed graph where it's too late for us to dynamically populate the count. To fix this requires some reorganization of Terraform's workflow, the design for which is not yet obvious.

I was actually mistaken when I said there was an open ticket for this in my last comment. I forgot that this feature was originally being tracked in the data source issue, but that was closed (with a reduced scope) already. I expect we'll open a new ticket for this once a more concrete design is devised, but for now I suppose this ticket will serve to represent the missing feature.

[MarsuperMammal]

The use of the ec2 resource leveraging the data source for count was merely a simple example resource. The code itself is actually the example ref directly from the ec2_instance reference page.

The idiom itself has desired implementations in many places, where less clear solutions, such as using autoscaling groups, aren't available. For instance, if you desired to deploy a VPC with a web and app subnet tier, to later be leveraged in autoscaling, across all available AZs, and would have a desire to reuse the same VPC module between regions (where the total potential number of availability zones could differ) the model becomes much more cumbersome.

There are some nasty pieces to that logic as well, as you're using the available AZs at apply time to determine your network, and could destroy instances during a subsequent apply in an AZ which is functioning to serve traffic but not listed as available for the launch of new instances, but edge cases will always exist, and with sufficient ephemerality that behavior could be tolerated.

The end result is just a +1 to the feature addition of dynamic count, because regardless of the individual example used to justify it's value, it's really a case of access to addition information allows for the declaration of more elegant and flexible environments.

[lbernail]

I just tried to use the aws_availability_zones data source to create a subnet in each AZ and run into a similar issue with 0.7.0

data "aws_availability_zones" "available" {}

resource "aws_subnet" "public" {
  vpc_id     = "${aws_vpc.main.id}"
  count      = "${length(data.aws_availability_zones.available.names)}"
  cidr_block = "${cidrsubnet(var.cidr_block, var.subnet_bits, count.index) }"
  availability_zone       = "${data.aws_availability_zones.available.names[count.index]}"
  map_public_ip_on_launch = "true"
}

I get the following error:

* strconv.ParseInt: parsing "${length(data.aws_availability_zones.available.names)}": invalid syntax

However if I use a list variable (instead of the data source), it works as expected:

variable "azs" {
  type = "list"
  default = [ "eu-west-1a", "eu-west-1b", "eu-west-1c" ]
}

resource "aws_subnet" "public" {
  vpc_id     = "${aws_vpc.main.id}"
  count      = "${length(var.azs)}"
  cidr_block = "${cidrsubnet(var.cidr_block, var.subnet_bits, count.index) }"
  availability_zone       = "${var.azs[count.index]}"
  map_public_ip_on_launch = "true"
}

[drshade]

+1 on this. Shouldn't the data.aws_availability_zones.available.names data source work exactly like the list?

[dbrandt]

I think I'm starting to see a pattern when it comes to many of the interpolation-related tickets.

It seems (I haven't started digging through the source code yet) that Terraform can't construct its execution plan without having all the data ready at the planning stage. The aws_availability_zone data source has to execute to contain values, which it won't do until the apply-phase. Hence you can't base something like a count on interpolation from a data source like this. It might also explain why you can't interpolate one variable from another (if interpolations are indeed performed in the execution phase).

This seems like a fundamental weakness (not necessarily a flaw) in the data flow for a lot of us Terraform users that would like to see discovered data steer the execution a bit.

This is probably not the best place for my pontifications, but here it is anyway.

[apparentlymart]

I expect that making data sources be interpolated into count would come after the introduction of the feature described in #4149, since that should enable the refactoring required to make this work. At present Terraform handles counts too "early" for the data sources to have had a chance to run, so some restructuring is required to make both this and the "partial apply" case of dynamic counts work as desired here.

I totally agree that this should work and have made some early steps on prototyping partial apply (in #8521) to start down the road to this being possible.

[jc-m]

@apparentlymart #4149 is almost one year in the making, and it seems that a lot of users are blocked by this issue (me included). Wouldn't it be easier at this point to treat data sources differently (specially remote_state ones) and try to perform the resolution phases before the plan and apply phase instead of trying to try to solve the more ambitious chicken and egg problem ?

[lbernail]

I run into this limitation again today in a different scenario:

I usually create VPC in an dedicated terraform stack and use remote states to retrieve the resources created in the VPC:

data "terraform_remote_state" "vpc" {
  backend = "s3"
  config {
    bucket = "${var.state_bucket}"
    key    = "${var.vpc_state_key}"
    region = "${var.region}"
  }
}

Sometimes I would like to use information from this state file as a count parameter. For instance:

resource "aws_efs_mount_target" "efsmp" {
  count = "${length(data.terraform_remote_state.vpc.private_subnets})"
  file_system_id = "${aws_efs_file_system.efs.id}"
  subnet_id = "${var.subnets[count.index]}"
}

If I'm correct this was possible before terraform 0.7 and the transformation of the remote state to a data source

It is not a very big deal but it would be really helpful if this was solved

[tomgoren]

I think this is related, if not a duplicate of https://github.com/hashicorp/terraform/issues/1497.

[kostyrev]

yep, it's the same issue

[MarsuperMammal]

Closing as a fundamental duplicate of #1497.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.