Terraform: Error refreshing state: state data in S3 does not have the expected content. Comes up when i try to spin up a new tf plan

[00subra8]

I have amnually deleted my S3, .tfstate file, lambda, api gateway and cloudfront entries and am trying to spin up a new instance of all the above:

the terraform plan is fine, but once I approve:

Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Error refreshing state: state data in S3 does not have the expected content.

This may be caused by unusually long delays in S3 processing a previous state
update. Please wait for a minute or two and try again. If this problem
persists, and neither S3 nor DynamoDB are experiencing an outage, you may need
to manually verify the remote state and update the Digest value stored in the
DynamoDB table to the following value:

Note: am not using dynamo Db
Also it seems to work fine if the .tfstate file is not deleted

[jbardin]

Hi @00subra8,

That is somewhat unexpected if you're not using DynamicDB at all. The only place that the backend gets a digest for comparison is from DynamoDB, so we shouldn't see the error if it didn't exist. Can you provide some more detail about the backend configuration? What is the .tfstate file you mention?

[00subra8]

I believe the .tfstate file stores the current state of my terraform plan. I find it in S3 bucket under my project folder. I have deleted this and torn down the whole proj - S3, api gateway, cloudfront, lambda and am trying to respawn the whole proj. This is when i get the above error.

[00subra8]

ok i restored my tf state to an old version and now everything works well. The error i kept getting without tfstate was:

to manually verify the remote state and update the Digest value stored in the
DynamoDB table to the following value:

so my new question is : when u teardown do we also delete this digest entry. would be very helpful if there is anyway to completely destroy via terraform and not tear down in stages from aws UI.

[nherds]

Like as been refered here:
https://github.com/hashicorp/terraform/issues/15380#issuecomment-310800265

Removing the md5 item from the already existing dynamo table solved for me.

Hope that helps!
;)

[anuj-modi]

I believe the exact issue is that at this line. The backend client simply logs a warning when the MD5 Put fails rather than indicating a failure anywhere. This should NOT fail silently. Since at this point the infrastructure changes have already succeeded, I think this should either be a time based retry or a message to the user the hash not being successfully pushed. Maybe both.