Terraform: aws_s3_bucket_policy marked as modified on every apply regardless of changes

[kerin]

Terraform Version

Terraform v0.8.8

Affected Resource(s)

• aws_s3_bucket_policy

Terraform Configuration Files

resource "aws_s3_bucket_policy" "source" {
    bucket = "${aws_s3_bucket.source.id}"
    policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyIncorrectEncryptionHeaderInSource",
      "Effect": "Deny",
      "Action": "s3:PutObject",
      "Resource": "${aws_s3_bucket.source.arn}/*",
      "Principal": "*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "AES256"
        }
      }
    },
    {
      "Sid": "DenyUnEncryptedObjectUploadsInSource",
      "Effect": "Deny",
      "Action": "s3:PutObject",
      "Resource": "${aws_s3_bucket.source.arn}/*",
      "Principal": "*",
      "Condition": {
        "Null": {
          "s3:x-amz-server-side-encryption": true
        }
      }
    }
  ]
}
EOF
}

Debug Output

https://gist.github.com/kerin/c27c23765dc0366147961b49059788c3

Expected Behavior

Terraform should not see modifications to unchanged resources.

Actual Behavior

Terraform thinks there's a modification to be applied on each plan or apply.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform plan
2. Observe planned changes to bucket policy
3. terraform apply
4. terraform plan
5. Observe that Terraform plans to apply the same modification again.

Important Factoids

None of these attempts to work around the issue have had any effect:

• Removing all linebreaks and spaces from the policy document
• Using a separate JSON document via $file()
• Replacing references to resource ARNs with static values

[bdashrad]

Try defining the policy with aws_iam_policy_document. I've found this is usually the result of formatting.

[kevinburke]

Try defining the policy with aws_iam_policy_document. I've found this is usually the result of formatting.

Can you explain further? If so it seems like we should open a separate issue for that; a policy that is consistently generating a different file in AWS than the one specified.

[bdashrad]

If you are parsing in json as a policy, terraform is not formatting it. When you run a plan, terraform takes the policy from the AWS api and compares it to what you have. AWS sometimes formats the JSON, so if the policy you are creating has different formatting, terraform will show that it needs to be changed. If you use the aws_iam_policy_document data source, it renders the JSON in the same format as AWS, and you're less likely to see errors.

[ju2wheels]

@bdashrad That wont help here. In my case, my policy uses principals and you cannot create an actual aws_iam_policy_document using the policy and using aws_iam_policy_document data source results in the same issue @kerin reported.

This still affects 0.9.6. Here is another example to reproduce the issue:

variable "aws_elb_account_ids" {
  description = "AWS ELB account IDs to allow access to the Application ELB S3 bucket for access logs."
  type        = "map"

  default     = {
    ap-northeast-1 = "582318560864"
    ap-northeast-2 = "600734575887"
    ap-south-1     = "718504428378"
    ap-southeast-1 = "114774131450"
    ap-southeast-2 = "783225319266"
    ca-central-1   = "985666609251"
    cn-north-1     = "638102146993"
    eu-central-1   = "054676820928"
    eu-west-1      = "156460612806"
    eu-west-2      = "652711504416"
    sa-east-1      = "507241528517"
    us-east-1      = "127311923021"
    us-east-2      = "033677994240"
    us-gov-west-1  = "048591011584"
    us-west-1      = "027434742980"
    us-west-2      = "797873946194"
  }
}

data "aws_iam_policy_document" "my_s3_alb_access" {
  statement {
    actions = [
      "s3:PutObject"
    ]

    principals {
      type        = "AWS"
      identifiers = [
        "${var.aws_elb_account_ids["ap-northeast-1"]}",
        "${var.aws_elb_account_ids["ap-northeast-2"]}",
        "${var.aws_elb_account_ids["ap-south-1"]}",
        "${var.aws_elb_account_ids["ap-southeast-1"]}",
        "${var.aws_elb_account_ids["ap-southeast-2"]}",
        "${var.aws_elb_account_ids["ca-central-1"]}",
        "${var.aws_elb_account_ids["eu-central-1"]}",
        "${var.aws_elb_account_ids["eu-west-1"]}",
        "${var.aws_elb_account_ids["eu-west-2"]}",
        "${var.aws_elb_account_ids["sa-east-1"]}",
        "${var.aws_elb_account_ids["us-east-1"]}",
        "${var.aws_elb_account_ids["us-east-2"]}",
        "${var.aws_elb_account_ids["us-west-1"]}",
        "${var.aws_elb_account_ids["us-west-2"]}"
      ]
    }

    effect    = "Allow"
    resources = ["${aws_s3_bucket.my_bucket.arn}/alb_logs/${var.my_module_instance_name}/*"]
  }
}

resource "aws_s3_bucket_policy" "my_s3_access" {
  bucket = "${aws_s3_bucket.my_bucket.id}"
  policy = "${data.aws_iam_policy_document.my_s3_alb_access.json}"
}

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.