Terraform: Can't use for_each with an object that has sensitive values.

[WilliamABradley]

With Terraform 0.14, some of my Terraform modules have become broken. A key one is a module that creates multiple secrets in Secrets Manager. With this new update, it no longer works, because it takes a map of strings, if one of the values of the map is sensitive, it won't work. I even tried to rewrite it to use the keys function (The keys aren't sensitive), and it still fails.

Terraform Version

Terraform v0.14.0

Terraform Configuration Files

variable "prefix" {
  type    = string
  default = ""
}

variable "tags" {
  type    = map(any)
  default = {}
}

variable "secrets" {
  type = map(string)
}

variable "version_stages" {
  type    = list(string)
  default = ["AWSCURRENT"]
}

resource "aws_secretsmanager_secret" "secret" {
  for_each = toset(keys(var.secrets))

  name                    = "${var.prefix}${each.value}"
  recovery_window_in_days = 0
  tags                    = var.tags
}

resource "aws_secretsmanager_secret_version" "secret_version" {
  for_each = toset(keys(var.secrets))

  secret_id     = aws_secretsmanager_secret.secret[each.value].id
  secret_string = var.secrets[each.value]

  version_stages = var.version_stages
}

Debug Output

N/A

Crash Output

N/A

Expected Behavior

Create multiple secrets based on keys.

Actual Behavior

  on .terraform/modules/core.kubernetes_secrets/modules/secretsManager/store/main.tf line 4, in resource "aws_secretsmanager_secret" "secret":
   4:   for_each = keys(var.secrets)

Sensitive variable, or values derived from sensitive variables, cannot be used
as for_each arguments. If used, the sensitive value could be exposed as a
resource instance key.

Steps to Reproduce

1. terraform init
2. terraform plan

Additional Context

References

[Nosmoht]

Hi All,

we face the same issue using Google KMS Crypto Key output as input for another resource. Any idea how to fix this?

[WilliamABradley]

Perhaps there needs to be a way to unmark a value, declaring that I'm ok with this value is not sensitive to me.

E.g. desensitize(toset(keys(var.secrets)))

[danieldreier]

I've restated a simpler reproduction case in https://github.com/danieldreier/terraform-issue-reproductions/tree/master/27180 to try and abstract this from an AWS resource into a trivial null_resource case. If I'm understanding correctly, the issue being reported is that when sensitive = true is explicitly set, there is no way to use keys that are not considered sensitive to the user for iteration. Again, if I understand correctly, this only happens if you use the new sensitive feature - is that right? It doesn't sound to me like existing workflows are being broken, so much as that you would like to use this new feature and cannot because you you need to iterate over a non-sensitive part of a sensitive value. Can you confirm that?

If I'm understanding that right, this sounds like a valid and reasonable enhancement request rather than a bug, but I'm open to pushback if I'm not understanding the request correctly

[umglurf]

Hi @danieldreier

We also have this issue and I can say our existing workflow is being broken. I've created a small example with null_resource to show how. This is very similar to our setup that is broken with 0.14. In the example, I've set token as a variable just to show how it breaks, but it actually is an output from a resource that set it correctly as sensitive.

variable "token" {
  type      = string
  default   = "foo"
  # for reproducing issue, comment out for 0.13
  sensitive = true
}

locals {
  configs = {
    test = {
      token = var.token
    }
  }
}

module "test" {
  source   = "./modules/test"
  projects = ["project1"]
  configs  = local.configs
}

# test module
variable "projects" {
  type = list(string)
}

variable "configs" {
  type = map(object({
    token = string
  }))
}

resource "null_resource" "project" {
  for_each = { for e in setproduct(var.projects, keys(var.configs)) : "${e[0]}-${e[1]}" => e }
  # do something with token here
}

[danieldreier]

@umglurf I am surprised by that behavior; according to our docs:

There is also experimental behavior that will extend this sensitivity-awareness to attributes providers define as sensitive. You can enable this feature by activating the experiment in the terraform block:

terraform {
  experiments = [provider_sensitive_attrs]
}

If you enable this experiment, attributes that are defined by a given provider as sensitive will have the same sensitivity-tracking behavior as sensitive input values and outputs. For example, the vault_generic_secret data source has an attribute data that is sensitive according to this provider's schema.

Assuming you do not have this experiment enabled intentionally, it sounds to me like you are describing a case in which the experimental behavior is enabled without the flag being set. Our intent had been to ship that feature as experimental to avoid upgrade issues like what you're running into here.

I'm going to dig in using the aws_secretsmanager_secret resource to see if I can reproduce a behavior in which provider-defined sensitivity is used even without the experimental flag set, and then will update this issue with my findings.

[umglurf]

@danieldreier I was looking through our code again, and it the value comes from a module output with sensitive set to true, not from a resource output. However I would still argue that the existing workflow is broken by 0.14 as shown in my example.

[danieldreier]

Ah, that makes more sense to me. That use case makes sense. Thanks for digging in and explaining it. That does seem more bug-like than feature-enhancement-like because it breaks a workflow that previously worked, without an obvious workaround.

@WilliamABradley and @Nosmoht are you also seeing this for previously-defined sensitive module outputs, or are you seeing it directly from resources without explicitly having enabled the provider_sensitive_attrs experiment?

[danieldreier]

@umglurf regardless of what @WilliamABradley says here, I think that the specific issue you're describing - how the 0.14 sensitive values feature interacts with existing sensitive module outputs - ought to be a separate issue. Do you mind filing a new bug report and linking to this one? I'd like to separate out that issue from the desire to iterate over keys in sensitive variables. They are related but different needs.

[pselle]

@WilliamABradley The current behavior for Terraform 0.14 is to disallow values that contain _any_ sensitive-marked data, however, we could be more nuanced in this behavior and allow cases where the object may contain sensitive values, but have those values in the _values_ of the object but not the keys.

I've linked the PR I'm drafting that will address this issue, however this will only address cases like so:

variable "foo" {
  default = "cat"
  sensitive = true
}

locals {
  baz = {
    "a" = var.foo
    "b" = "dog"
  }
}

resource null_resource foo {
  for_each = local.baz # will now be allowed
}

The use case you have here (and I imagine not-uncommon) is leveraging functions to create values from sensitive values, and those functions currently return the transformed values as sensitive, without defining specific locations within the value as sensitive, per se ("why is the keys() function returning a sensitive value, when only the values of the map were sensitive?"). To wit: some fixes found, but I'm investigating more, hopefully without having to answer "If you use expressions involving sensitive values, you can't use for_each" (the current diagnostic you have).

Unfortunately, I would recommend a work-around of using count rather than for_each, but I found and fixed a bug with that (this fix is unreleased): https://github.com/hashicorp/terraform/pull/27238.

[WilliamABradley]

So the issue was that a module output was sensitive, passing to another module output, which is required to be sensitive, because the object contains sensitive data:

Kubernetes Admin Module:

output "kubeconfig_data" {
 value = ...
 sensitive = true
}

Kubernetes Module:

output "cluster_secrets" {
  value = {
    kubeconfig = module.admin.kubeconfig_data
  }
  # Requires sensitive = true in 0.14.0, as it contains sensitive values.
  sensitive = true
}

To pass to secrets manager module:

module "cluster_config" {
  source = "./modules/secretsManager/store"

  secrets = {
    cluster_secrets = jsonencode(module.kubernetes.cluster_secrets)
  }
}

So the issue that breaks the flow is that it is forcing the object to be sensitive, when only the values are, which means they can't be used in for_each.

@pselle I will try using count instead once your fix is released :)

[pselle]

@WilliamABradley I wanted to share some information that might help with your current usage, that although using a function like keys() with sensitive arguments means the result of that function (currently) will be sensitive (and thus can't be used with for_each, you could achieve the same results with a for expression: toset([for k,v in var.foo : k]) (which transforms the object into a list of keys and creates a set with the non-sensitive result of that for expression to pass to toset).

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.