Terraform-provider-aws: kms:DescribeKey on default S3 KMS key should not be required

[FireballDWF]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

Terraform 0.10.8
plugin.terraform-provider-aws_v1.9.0_x4

Affected Resource(s)

Please list the resources as a list, for example:

• aws_s3_bucket_object

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

provider aws {
  version = "~> 1.9"
}
resource "aws_s3_bucket_object" "emr_server_cert_object" {
  key                    = "${var.tag_application_name}-${var.name_suffix}-emr_server-cert.zip"
  bucket                 = "${var.app_bucket}"
  source                 = "${path.module}/emr_server_cert.zip"
  server_side_encryption = "aws:kms"
  kms_key_id             = "${var.kms_key_arn}"
}

Debug Output

https://gist.github.com/55e126e62d3c205c1082891c700fdac5

Expected Behavior

terraform-provider-aws should have only called kms:DescribeKey on the value of the var.kms_key_arn which was "arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f"

Actual Behavior

Instead it called it on arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7 which is the default S3 KMS Key. No where in the bucket is the default S3 key used, and the role being used does not have or need access to the default S3 KMS key. The error received is:

default S3 KMS key (alias/aws/s3): AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. Setup IAM role which has an explict deny on kms:DescribeKey for the default KMS S3 key.
2. Setup an S3 bucket with a default encryption policy aws:kms with the same kms_key_arn as will be used in the aws_s3_bucket_object.
3. Use aws_s3_bucket_object to put a file in that same bucket and specify the kms_key_id (same as when setting up the default bucket encryption).
4. Observe the error

[FireballDWF]

I'm working on getting a gist of the trace.log, but in the meantime here is a grep showing some relevant lines:

$ grep 'arn:aws:kms:us-east-1:253038043104:key/' trace.log 
2018-02-19T16:33:49.420-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: <ServerSideEncryptionConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ApplyServerSideEncryptionByDefault><SSEAlgorithm>aws:kms</SSEAlgorithm><KMSMasterKeyID>arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f</KMSMasterKeyID></ApplyServerSideEncryptionByDefault></Rule></ServerSideEncryptionConfiguration>
2018-02-19T16:33:49.420-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4:           KMSMasterKeyID: "arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f",
2018-02-19T16:33:49.736-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id: arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f
2018-02-19T16:33:49.737-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4:   SSEKMSKeyId: "arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f",
2018-02-19T16:33:49.838-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: 2018/02/19 16:33:49 [DEBUG] [aws-sdk-go] {"__type":"AccessDeniedException","Message":"User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7"}
2018-02-19T16:33:49.838-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: 2018/02/19 16:33:49 [DEBUG] [aws-sdk-go] DEBUG: Validate Response kms/DescribeKey failed, not retrying, error AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7
2018/02/19 16:33:49 [ERROR] root.emr-cluster-test: eval: *terraform.EvalRefresh, err: aws_s3_bucket_object.emr_server_cert_object: Failed to describe default S3 KMS key (alias/aws/s3): AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7
2018/02/19 16:33:49 [ERROR] root.emr-cluster-test: eval: *terraform.EvalSequence, err: aws_s3_bucket_object.emr_server_cert_object: Failed to describe default S3 KMS key (alias/aws/s3): AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7

[FireballDWF]

Editted Debug Output section to link to gist of trace file

[ashishapy]

I am also getting same error in exact scenario.

Any workaround or solution to this bug?

[netdisciple]

I also have this issue.

[brandonstevens]

I'm able to reproduce this on the latest release (0.24.0). Looks like it's due to how TF tries to keep state consistent when the default S3 key is used. AWS automatically sets the key ID and subsequent reads will return it. Without this code, Terraform plans would think the key changed from "" to the default key ARN. Here is the offending code block (https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_s3_bucket_object.go#L297):

if resp.SSEKMSKeyId != nil {
    // retrieve S3 KMS Default Master Key
    kmsconn := meta.(*AWSClient).kmsconn
    kmsresp, err := kmsconn.DescribeKey(&kms.DescribeKeyInput{
        KeyId: aws.String("alias/aws/s3"),
    })
    if err != nil {
        return fmt.Errorf("Failed to describe default S3 KMS key (alias/aws/s3): %s", err)
    }

    if *resp.SSEKMSKeyId != *kmsresp.KeyMetadata.Arn {
        log.Printf("[DEBUG] S3 object is encrypted using a non-default KMS Key ID: %s", *resp.SSEKMSKeyId)
        d.Set("kms_key_id", resp.SSEKMSKeyId)
    }
}

I'm wondering if we can simply remove this condition and make kms_key_id a computed value.

[Ilhicas]

On version 0.12.6 still getting the same error as above. A role that only has permission to a given key is being assumed by terraform to Put an encrypted object at S3, resulting in DescribeKey being triggered against the arn for default key.

[asahin-icg]

strange, seems still this isssue isn't solved.

[yetanotherbug]

Hitting this issue right now, too: terraform v0.12.28, AWS provider v2.69.0. This prevents us from managing certain configuration blobs using Terraform. Please fix this; the arguments in favour of this behaviour are not good enough to convince our core infra team to grant more permissions on KMS keys.