Terraform: Deprecate s3 remote backend lock table with new strong consistency

[tomelliff]

Current Terraform Version

0.13.5

Proposal

Terraform currently requires the usage of a DynamoDB table for state file locking due to consistency requirements that have previously been impossible in S3 directly as it has eventual consistency on read after write. The GCS backend, however, is able to avoid this extra mechanism and just write the lock file to the GCS bucket as GCS provides strong read after write consistency.

AWS have announced that S3 will now support strong read after write consistency: https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/

I think this means that the DynamoDB lock table is now unnecessary and the same approach as the GCS backend can be used here to write the lock file directly to S3.

This should simplify the usage of Terraform's S3 backend as it removes an extra component.

I'd propose aligning the S3 backend with the GCS backend so it uses the lock file in S3 and marks the dynamodb_table argument as deprecated in 0.14 releases. I'd then propose removing the dynamodb_table argument altogether in 0.15/1.0.

References

• https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/
• https://www.terraform.io/docs/backends/types/s3.html
• https://www.terraform.io/docs/backends/types/gcs.html

[tomelliff]

If there's interest in this I might have some time to contribute the change on Friday.

[pkolyvas]

@tomelliff Thanks for the proposal!

The AWS team here at HashiCorp maintain our S3 remote state backend. They're probably a little busy with re:invent going at the moment so I'm not sure they'll have a chance to review and weigh this by Friday, but I'll bring it to their attention.

[tbugfinder]

I'm curious if this could make it into 0.15.0 :-)