Terraform: Error deleting S3 Bucket with Versioning Enabled: BucketNotEmpty

[osterman]

Terraform Version

Terraform v0.6.16

Affected Resource(s)

aws_s3_bucket

Terraform Configuration Files

resource "aws_s3_bucket" "storage" {
  bucket = "storage.${var.dns_zone}"
  acl = "private"                                                                                                                                                                                                                          
  force_destroy = true
  versioning {
    enabled = true                                                                                                                                                                                                                         
  }
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect":"Allow",
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::storage.${var.dns_zone}", "arn:aws:s3:::storage.${var.dns_zone}/*"],
      "Principal": "*",
      "Condition": {
        "StringEquals": {
          "aws:sourceVpce": "${aws_vpc_endpoint.s3.id}"
        }
      }
    }
  ]
}
EOF

  tags {
    Name = "${var.namespace}-storage"
    Namespace = "${var.namespace}"
    Role = "storage"
    Stage = "${var.stage}"
  }

  lifecycle {
    create_before_destroy = true
  }
}

Expected Behavior

Because force_destroy = true is enabled, it should also delete the versions of the objects.

Actual Behavior

Error applying plan:

1 error(s) occurred:

* aws_s3_bucket.storage: Error deleting S3 Bucket: BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket.
    status code: 409, request id: 9AE6E70339A97B62

Steps to Reproduce

1. Define a bucket resource
2. terraform apply
3. Upload some other assets to the bucket
4. terraform destroy
   

   Important Factoids

• Running on AWS.

• Bucket was created using aws cli because we also use it to store terraform remote state.

  References

• The ability to force delete a bucket was requested here: https://github.com/hashicorp/terraform/issues/1977

• The ability to force delete a bucket was implemented here: https://github.com/hashicorp/terraform/pull/2007 (no mention of versioning support)

[grebois]

+1

[Ninir]

Hi @osterman

Just tried using 0.7.13 and it works pretty nicely. To reproduce it, I launched your configuration, upload a file to S3, then update it locally, re-upload it to have 2 versions.
Then, I terraform destroy it and everything was removed.

Could you check again please?

Thanks!

[pgporada]

I am unable to delete a bucket with versions on Terraform 0.8.4 while force_destroy = true on the aws_s3_bucket resource. I get the same error as @osterman.

[iknite]

Same behaviour here,

Terrraform v0.8.7

variable "TFSTATE_BUCKET_NAME" {}

resource "aws_s3_bucket" "tfstate_log" {
    bucket = "${var.TFSTATE_BUCKET_NAME}-log"
    acl = "log-delivery-write"
    force_destroy = true
}

resource "aws_s3_bucket" "tfstate" {
    bucket = "${var.TFSTATE_BUCKET_NAME}"
    acl = "private"
    force_destroy = true

    versioning {
        enabled = true
    }

    logging {
        target_bucket = "${aws_s3_bucket.tfstate_log.id}"
        target_prefix = "log/"
    }

}

Do you really want to destroy?
  Terraform will delete all your managed infrastructure.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

aws_s3_bucket.tfstate_log: Refreshing state... (ID: ***)
aws_s3_bucket.tfstate: Refreshing state... (ID: ***)
aws_s3_bucket.tfstate: Destroying...
Error applying plan:

1 error(s) occurred:

* aws_s3_bucket.tfstate: Error deleting S3 Bucket: BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket.
    status code: 409, request id: *************

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

[bonovoxly]

Terraform v0.8.7 here. Same error.

[sebbonnet]

force_destroy is an attribute that needs to be applied first against your s3 resource (e.g. terraform apply).
If the attribute is not set to true in your state file for your s3 resource, the force delete logic won't be applied when destroying the bucket (e.g. terraform destroy).
This works for me on terraform v0.8.8.

[igor-kupczynski]

I can confirm what @sebbonnet says in version 0.9.2

If you create a bucket without force_destroy = true then even if you add it later you can't force destroy it, as the resource in the state file has force_destroy = false. Of course you can force_destroy, then apply and then delete.

~ aws_s3_bucket.my_bucket
    force_destroy: "false" => "true"

- aws_s3_bucket.my_bucket

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.