Terraform-provider-aws: aws_kms_secret data source produces dynamically-named attributes, breaks count support

[RulerOf]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

$ terraform -v
Terraform v0.11.7
+ provider.aws v1.25.0

Affected Resource(s)

• aws_kms_secret

Terraform Configuration Files

provider "aws" {
  region  = "us-east-1"
}

resource "aws_kms_key" "test_key" {
  description = "test key"
  is_enabled = true
}

resource "random_string" "gibberish" {
  length = 16
  special = false
}

data "aws_kms_ciphertext" "test_ct" {
  key_id = "${aws_kms_key.test_key.key_id}"
  plaintext = "my plain text"
}

data "aws_kms_secret" "test_secret" {
  count = 2
  secret {
    name    = "${random_string.gibberish.result}"
    payload = "${data.aws_kms_ciphertext.test_ct.ciphertext_blob}"
  }
}

Expected Behavior

Terraform should create a resource that can be consumed through an attribute name that can be interpolated. This data source takes an argument value and turns it into an attribute, which means that any attempt to interpolate the resulting attribute requires interpolation of attribute names which terraform does not support.

Actual Behavior

This causes normal terraform code conventions like count to be effectively useless with this provider, as you could count the data source, but then have to explicitly define every resource that consumes the decrypted secrets because of the way the resulting attributes come out, as illustrated with the state from the POC code above:

AndrewBlskysMBP:kms andrewbobulsky$ terraform console
> data.aws_kms_secret.test_secret
data.aws_kms_secret.test_secret: data variables must be four parts: data.TYPE.NAME.ATTR in:

${data.aws_kms_secret.test_secret}
> data.aws_kms_secret.test_secret.1
Resource 'data.aws_kms_secret.test_secret' not found for variable 'data.aws_kms_secret.test_secret.1'
> data.aws_kms_secret.test_secret.1.*
Resource 'data.aws_kms_secret.test_secret.1' does not have attribute '*' for variable 'data.aws_kms_secret.test_secret.1.*'
> data.aws_kms_secret.test_secret.*.*
Resource 'data.aws_kms_secret.test_secret' does not have attribute '*' for variable 'data.aws_kms_secret.test_secret.*.*'
> data.aws_kms_secret.test_secret.*.secret
[
  [
    {
      "context" = {}
      "grant_tokens" = []
      "name" = "JDh2cT5730rWif6g"
      "payload" = "AQICAHiaF7Yp1rgVkRvVMhwWY41jeFFyo38RNJ9aRgU8huNLxwFIDhFB3MRF+qvxzqA9RxwHAAAAazBpBgkqhkiG9w0BBwagXDBaAgEAMFUGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMPZGJFCV561KG7fYvAgEQgCgUyOZ0CVZTlccWvz44daZRyVdOirs5WS6fLXSqyhkw2NJVEBmfpgcC"
    },
  ],
  [
    {
      "context" = {}
      "grant_tokens" = []
      "name" = "JDh2cT5730rWif6g"
      "payload" = "AQICAHiaF7Yp1rgVkRvVMhwWY41jeFFyo38RNJ9aRgU8huNLxwFIDhFB3MRF+qvxzqA9RxwHAAAAazBpBgkqhkiG9w0BBwagXDBaAgEAMFUGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMPZGJFCV561KG7fYvAgEQgCgUyOZ0CVZTlccWvz44daZRyVdOirs5WS6fLXSqyhkw2NJVEBmfpgcC"
    },
  ],
]
> data.aws_kms_secret.test_secret.0.JDh2cT5730rWif6g
my plain text

The only thing I'm able to bring up has _everything but_ the decrypted data in it.

Steps to Reproduce

1. Create a new terraform project with the code above in the file main.tf
2. terraform apply
3. Try some method of interpolating the resulting attributes

Important Factoids

This particular data source is the only one I've seen that creates dynamically-named attributes, and is also the only occurrence of the UnsafeSetFieldRaw() function(?) in the entire project. This behavior doesn't seem right to me.

As a suggestion, based on the existing architecture of the data source, perhaps the data source should return a result attribute that is a map of names and decrypted secrets, such that the following interpolation would work on a single-count aws_kms_secret:

lookup(aws_kms_secret.test_secret.result, "my-secret-name")

This may not be a great idea due to the potential to create a list of maps when used with count, but it has the advantage of allowing the existing behavior to continue functioning.

Edit: I changed the POC code while I was writing this issue to source the name of the secret using the result attribute of a random_string resource because it does an even better job at showing how the aws_kms_secret data source can create completely-inaccessible attributes.

[bflad]

This discusses the future plan for this data source: #5144 (/ht @iancward)

[bflad]

Releasing tomorrow with version 1.29.0 of the AWS provider, a new aws_kms_secrets data source is will be available that does not use the dynamic attribute behavior. It instead uses a map attribute, which should work in this situation.

The existing aws_kms_secret data source needed to be deprecated anyways due to this behavior for us to support some upcoming changes in Terraform 0.12. The old data source itself and its documentation page will link to migration instructions, which are pretty straightforward. 😄

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!