Terraform: How to interpolate data source name ?

[arunvelsriram]

data "aws_iam_policy_document" "make_objects_public" {
  # if policy_name == <this-policy-name>, then generate policy
  count = "${var.policy_name == "make_objects_public" ? 1 : 0}"

  statement {
    ...
  }
}

resource "aws_s3_bucket_policy" "bucket_policy" {
  # if policy_name != "", then add the generated policy
  count = "${var.policy_name != "" ? 1 : 0}"

  bucket = "${var.bucket_name}"
  policy = "${data.aws_iam_policy_document.<policy-name-goes-here>.json}"
}

How to interpolate the variable policy_name while fetching the policy generated by  aws_iam_policy_document ?

policy = "${data.aws_iam_policy_document."${var.policy_name}".json}"

policy = "${"${format("%s", "data.aws_iam_policy_document.${var.policy_name}.json")}"}"

policy = "${format("%s", "$${data.aws_iam_policy_document.${var.policy_name}.json}")}"

This looks very simple to me but couldn't get anything to work. Does terraform support this kind of operation ?

[apparentlymart]

Hi @arunvelsriram,

Unfortunately dynamic resource names are not supported, since Terraform needs to be able to understand all of the dependencies before processing begins, and thus interpolation at this point in the process is not possible.

We don't usually recommend such dynamic configuration (switching between several resources based on variables) and instead suggest decoupling things so that they can be written with simple, static configuration.

For example, if you make each of your pre-written policies available as a module, and the bucket policy also a module, the calling configuration can then compose these things itself, statically:

module "policy" {
  # policy-specific module; only contains the policy data source
  source = "../policies/make_objects_public"

  # (any other arguments the policy needs...)
}

module "s3_bucket" {
  # S3 bucket module creates a bucket and attaches a policy to it
  source = "../s3_bucket" # general resource for S3 buckets with attached policies

  name   = "example"
  policy = "${module.policy.policy_json}" # an output from the policy module above
}

Each caller that uses the s3_bucket module can choose which of the policy modules is appropriate, or it can create its own custom policy directly for unusual cases. This keeps the policy selection decoupled from the bucket setup, giving more flexibility and making the relationships between these components more explicit.

[apparentlymart]

Hi again @arunvelsriram,

This morning I saw your similar question on Stack Overflow and posted the same suggestion there since such answers are easier for others to find than github issues.

I also added some extra color there about the design principle at play in this solution. We've been looking recently at designing a set of improvements to the configuration language to improve usability and to make it easier to produce larger systems from smaller components, and improving the usability of the dependency injection technique is one of the goals; it's proven itself a good design pattern in a number of cases already, but with today's language it can get pretty verbose for non-trivial modules with lots of "dependencies".

[arunvelsriram]

Interesting! Thank you for the wonderful de-coupled approach.

I have implemented the same. As you said now every caller of s3_bucket module can use one of existing policies or set a custom policy if needed.

Sharing the detailed implementation as it will be useful for others.

Modules:

# modules/s3_bucket/main.tf

resource "aws_s3_bucket" "bucket" {
  bucket = "${var.bucket_name}"
  policy = "${var.policy}"
  # ... other arguments
}

# modules/policies/make_objects_public/main.tf

data "aws_iam_policy_document" "make_objects_public" {
  statement {
    # ...
  }
}

output "policy_json" {
  value = "${data.aws_iam_policy_document.make_objects_public.json}"
}

Modules usage:

# staging/s3_bucket/bucket_using_policy_module.tf

module "policy" {
  source = "modules/policies/make_objects_public"

  # Other arguments if needed
}

module "example_s3_bucket" {
  source = "modules/s3_bucket"

  bucket_name = "example"
  policy = "${module.policy.policy_json}"

}

# staging/s3_bucket/bucket_using_custom_policy.tf  

data "aws_iam_policy_document" "custom_policy" {
  statement {
    # ...
  }
}

module "another_s3_bucket" {
  source = "modules/s3_bucket"

  bucket_name = "another"
  policy = "${data.aws_iam_policy_document.custom_policy.json}"
}

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.