Terraform: timeout (signature mismatch) error on creation of cloudfront distribution

[tobowers]

Terraform Version

v0.6.16

Affected Resource(s)

• aws_cloudfront_distribution
  

  Terraform Configuration Files

https://gist.github.com/tobowers/f46289da2bc9e451ec4b0dd3987d717d

Debug Output

https://gist.github.com/tobowers/523e4a1b502948f1a352370a3eb95796

Expected Behavior

Terraform should have waited for the cloudfront distribution to become ready (10-15 minutes as the documentation suggests).

Actual Behavior

At the 6 minute mark, the AWS signature is no longer valid and AWS rejects terraform's request, and the apply is halted

Steps to Reproduce

1. terraform apply

[solvip]

Confirmed - we experience this with v0.6.16. We can supply our code and plans on request.

[tobowers]

@solvip would love to see how you worked around this.

[solvip]

@tobowers Unfortunately, the workaround so far has been to create the distribution manually. I suspect this is a regression, as the documentation specifies that creating the distribution will take a long time.

[mwalton236]

I'm not sure if it's related but I'm also seeing 503 for post requests to http://cloudfront.amazonaws.com/doc/2016-01-28/. It appears that this is only listening on https now.

[mwalton236]

try adding minimum_protocol_version = "TLSv1" to the viewer_certificate configuration. The default set by terraform is SSLv3, however that does not work via the API.

[martinthegeek]

Adding minimum_protocol_version = "TLSv1" resolved this problem for me. Thanks! Guess it still deserves better error reporting.

[tomelliff]

I've just come across this and tried again with 0.6.15 and that has the same issue. Am I missing a trick somewhere to prevent Terraform's request timing out?

[mwalton236]

for a work around, try adding minimum_protocol_version = "TLSv1" to the viewer_certificate configuration.

I've got a ticket open with AWS. They think it's related to this https://aws.amazon.com/security/security-bulletins/CVE-2014-3566-advisory/.

If the minimum protocol version is set to the terraform's default value of SSLv3 (which should be valid if you're using a custom domain), then the cloudfront API returns a 503, and doesn't create the distribution. After 5 mins the timeout error occurs.

If you set the minimum_protocol_version to at least "TLSv1" everything seems to work.

[tomelliff]

Thanks @mwalton236 was scratching my head at that. If I get a chance this weekend I'll raise a PR to fix that.

[steve-jansen]

Thank you @mwalton236!! Was also scratching my head watching Terraform tell me "Still creating...", and eventually hitting the 6 minute mark.

[dmportella]

@tobowers is this still a required change?

See below the aws api specs for this

Minimum SSL Protocol Version

Specify the minimum version of the SSL protocol that you want to use—SSLv3 or TLSv1—for HTTPS connections between viewers and CloudFront. CloudFront will serve your objects only to viewers that support the SSL version that you specify and later versions. The TLSv1 protocol is more secure, so we recommend that you specify SSLv3 only if your users are using browsers or devices that don't support TLSv1. Note the following:

• If the option that you select for SSL Certificate is Default CloudFront Certificate, the minimum SSL protocol version is TLSv1 and can't be changed.
• If the option that you select for SSL Certificate is Custom SSL Certificate and if the option that you select for Custom SSL Client Support is Only Clients that Support Server Name Indication (SNI), CloudFront uses TLSv1, which is the minimum allowed SSL protocol for SNI.
• If you selected Custom SSL Certificate and you selected Only Clients that Support Server Name Indication (SNI), CloudFront uses TLSv1, which is the minimum allowed SSL protocol for SNI.

---

The question is: Should we just default the value to TLS1v and Update the documentation?

[tobowers]

@dmportella well I actually stopped using terraform so not required for me :). It's been a while. However, defaulting to TLSv1 seems like a good idea to me.

[tomelliff]

Just took a quick look at this after the notification in this issue popped up and it oddly looks like AWS is reporting that distributions using the default *.cloudfront.net certificate have a minimum protocol version of SSLv3 because when I change the default to "TLSv1" and run the tests I get this back:

=== RUN   TestAccAWSCloudFrontDistribution_customOrigin
--- FAIL: TestAccAWSCloudFrontDistribution_customOrigin (57.97s)
    testing.go:280: Step 0 error: After applying this step, the plan was not empty:

        DIFF:

        UPDATE: aws_cloudfront_distribution.custom_distribution
          viewer_certificate.1540335073.acm_certificate_arn:            "" => ""
          viewer_certificate.1540335073.cloudfront_default_certificate: "" => "true"
          viewer_certificate.1540335073.iam_certificate_id:             "" => ""
          viewer_certificate.1540335073.minimum_protocol_version:       "" => "TLSv1"
          viewer_certificate.1540335073.ssl_support_method:             "" => ""
          viewer_certificate.2654384193.acm_certificate_arn:            "" => ""
          viewer_certificate.2654384193.cloudfront_default_certificate: "true" => "false"
          viewer_certificate.2654384193.iam_certificate_id:             "" => ""
          viewer_certificate.2654384193.minimum_protocol_version:       "SSLv3" => ""
          viewer_certificate.2654384193.ssl_support_method:             "" => ""

and when I caught one of the tests running using the CLI I can see this:

...
                "ViewerCertificate": {
                    "CloudFrontDefaultCertificate": true, 
                    "MinimumProtocolVersion": "SSLv3", 
                    "CertificateSource": "cloudfront"
                }, 
                "CustomErrorResponses": {
                    "Quantity": 0
                }, 
                "LastModifiedTime": "2017-05-03T15:11:01.806Z", 
                "HttpVersion": "HTTP2", 
                "Id": "E39YMW8C82TYCJ", 
                "ARN": "arn:aws:cloudfront::423845407043:distribution/E39YMW8C82TYCJ", 
                "Aliases": {
                    "Items": [
                        "mysite.803670144339039499.example.com", 
                        "yoursite.803670144339039499.example.com"
                    ], 
                    "Quantity": 2
                }
...

The code is a little complicated and would take more time than I have right now to dig into what's going on there and how to deal with it.

I did also notice that none of the full acceptance tests cover anything other than using the *.cloudfront.net cert which looks like it could do with expanding but then there's a bunch of extra unit style tests against cloudfront_distribution_configuration_structure so that might help cover some things there.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.