Terraform: feature: extend sensitive attribute to all objects and most sub-objects

[mengesb]

It's great that we now have sensitive = (BOOL) on outputs, however it would be most excellent to add sensitive to all the things.

sensitive on top level resources such as null_resource and the likes, as well as internal things such as provisioner "remote-exec". Basically, sensitive for ALL THE THINGS!

Terraform Version

0.7.0

Affected Resource(s)

• all
  

  Terraform Configuration Files

Desired example syntax:

resource "null_resource" "knife-rb" {
  # Generate knife.rb
  sensitive = true
  provisioner "local-exec" {
    command = <<-EOC
      [ -f .chef/knife.rb ] && rm -rf .chef/knife.rb || echo OK
      tee .chef/knife.rb <<EOF
      ${data.template_file.knife-rb.rendered}
      EOF
      EOC
  }
}

or

resource "null_resource" "knife-rb" {
  # Generate knife.rb
  provisioner "local-exec" {
    sensitive = true
    command = <<-EOC
      [ -f .chef/knife.rb ] && rm -rf .chef/knife.rb || echo OK
      tee .chef/knife.rb <<EOF
      ${data.template_file.knife-rb.rendered}
      EOF
      EOC
  }
}

Of course there are more resources than null_resource that this could be beneficial for.

Debug Output

N/A (feature request)

Panic Output

N/A (feature request)

Expected Behavior

Able to suppress output due to the sensitive flag

Actual Behavior

Syntax error when key added to resource:

  * aws_instance.chef-frontends #0: : invalid or unknown key: sensitive

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. Add sensitive = true to any resource other than an output
2. Run terraform apply

Important Factoids

N/A (feature request)

References

N/A (feature request)

[mwz]

+1 for sensitive for ALL THE THINGS!

[apparentlymart]

Hi @mengesb! Thanks for opening this issue.

With our focus elsewhere right now we've not yet had a chance to delve into the details of this problem, but we've done some initial investigation/planning a few months ago which I will try to summarize here:

---

The way we are likely to approach this problem is to treat sensitive as a sort of "infectious" trait that follows values around as they move through expressions, using conservative static analysis techniques.

The general idea here would be that sensitivity would be established in one of three ways:

• A variable can be marked as sensitive by the user
• An output can be marked as sensitive by the user
• A resource attribute can be marked as sensitive by the provider developer.

With these "sensitivity roots" established, any other attribute that refers to one of them would then itself become sensitive automatically, without explicit specification. This would then propagate transitively.

This high-level idea would then allow sensitivity to be handled "everywhere" without adding any new configuration constructs. There are, however, some details that we need to figure out first, and so we'll need to do another design/prototyping pass to figure out the right approach for those. The Terraform team at HashiCorp is currently focused on configuration language syntax improvements for the next major release, so further design work in this area is paused for the moment.

Some examples of the tricky details we're already aware of are:

• The next major release of Terraform will include the ability to refer to an entire resource instance or module instance in an expression, returning an object value. In order to properly propagate the sensitive trait, we'd probably need to conservatively treat _everything_ in that object is sensitive, to avoid leakage when users do expressions like aws_instance.example[(any non-constant value)]. This is likely to make large parts of the diff be marked as <sensitive> whenever whole objects are accessed, which will reduce the effectiveness of the terraform plan output.
• The terraform_remote_state backend should ideally be able to detect sensitive attributes recorded in the state and propagate their sensitive trait, but this would add a dynamic component to what would otherwise be a static analysis problem.

---

In the mean time, we could potentially add a force_sensitive argument within the lifecycle block of resource and data blocks that would for now just cause Terraform to mask out the direct planned attributes of that resource but could later be one more way to introduce a "sensitivity root". However, I expect that's a pretty complicated change in itself (the configuration-level change is simple, but its effect on the rest of Terraform Core would be large) and so it's very possible that the design and implementation work for this would be of similar size to what I described above (remaining questions notwithstanding), so we'll look at this again once we're able to redirect our focus back to this problem and see if it's viable to implement this more iteratively.

[mengesb]

the majority of my concern was textual output; however I certainly don't ignore the value of it stemming to the sensitive nature of the backend output! Hope the planning goes well!

[bionicles]

+1 for this, our bitbucket pipelines env var password for our lambda to access our database shows up in the terraform plan when we set the environment variables for AWS lambda ... not ideal, how do you formulate a workaround or fix ? i guess KMS + decrypt inside lambda?

[pselle]

👋 I want to let this thread know that I've added this issue to the 0.14 milestone as I believe the upcoming changes that add a sensitive attribute to variables, and experimental support for extending sensitivity to provider-defined sensitive attributes, resolve this issue.

Please refer to the upgrade guide: https://github.com/hashicorp/terraform/blob/v0.14/website/upgrade-guides/0-14.html.markdown#sensitive-values-in-plan-output for some details on this feature, which you can test in the 0.14 beta versions.

[pselle]

Defining input variables as sensitive in order to redact the values from UI output is now available in 0.14! A recent blog post has more information on this feature, and I'll be closing this one as a result!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.