Terraform: S3 Backend TLS verification error with MacOS Catalina 10.15 Beta (19A512f) - x509: certificate signed by unknown authority

Created on 24 Jul 2019  ·  9Comments  ·  Source: hashicorp/terraform


I know that Terraform might not support OSX beta versions. But I foresee this bug coming up in future releases perhaps - this is why I report it.

Terraform Version

Terraform v0.12.5

Terraform Configuration Files

terraform {
  backend "s3" {
    bucket = "xxxxxxx-terraform"
    key    = "terraform.tfstate"
    region = "eu-central-1"
  }
}

Debug Output

Verifying that the S3 bucket remote state bucket exists
Configuring the terraform backend
2019/07/24 14:38:22 [INFO] Terraform version: 0.12.5  
2019/07/24 14:38:22 [INFO] Go runtime version: go1.12.4
2019/07/24 14:38:22 [INFO] CLI args: []string{"/Users/fabian/bin/terraform", "init", "-backend-config=profile=xx", "-backend-config=region=eu-central-1", "-backend-config=bucket=xxxxxxxeu-central-1-terraform", "-backend-config=key=terraform.tfstate"}
2019/07/24 14:38:22 [DEBUG] Attempting to open CLI config file: /Users/fabian/.terraformrc
2019/07/24 14:38:22 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/07/24 14:38:22 [INFO] CLI command args: []string{"init", "-backend-config=profile=xx", "-backend-config=region=eu-central-1", "-backend-config=bucket=xxxxxxxeu-central-1-terraform", "-backend-config=key=terraform.tfstate"}

Initializing the backend...
2019/07/24 14:38:23 [TRACE] Meta.Backend: merging -backend-config=... CLI overrides into backend configuration
2019/07/24 14:38:23 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value 922486822
2019/07/24 14:38:23 [TRACE] Preserving existing state lineage "3a7be645-6cb6-7535-f67e-261d13311219"
2019/07/24 14:38:23 [TRACE] Preserving existing state lineage "3a7be645-6cb6-7535-f67e-261d13311219"
2019/07/24 14:38:23 [TRACE] Meta.Backend: working directory was previously initialized for "s3" backend
2019/07/24 14:38:23 [TRACE] backendConfigNeedsMigration: given configuration matches cached configuration, so no migration is required
2019/07/24 14:38:23 [TRACE] Meta.Backend: using already-initialized "s3" backend configuration
2019/07/24 14:38:23 [INFO] Setting AWS metadata API timeout to 100ms
2019/07/24 14:38:23 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2019/07/24 14:38:23 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2019/07/24 14:38:23 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/07/24 14:38:23 [DEBUG] [aws-sdk-go] DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.20.19 (go1.12.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.5
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJYYYYYYYYYYYYYy/20190724/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=1a0e01e041b5678c7bfa57b1a1d24a9ab3333c51b6f128946ef4007515834e7a
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20190724T123823Z
Accept-Encoding: gzip

Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2019/07/24 14:38:24 [DEBUG] [aws-sdk-go] DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 412
Content-Type: text/xml
Date: Wed, 24 Jul 2019 12:38:23 GMT
X-Amzn-Requestid: edc3430e-ae0f-11e9-a298-c791ec8cb03d


-----------------------------------------------------
2019/07/24 14:38:24 [DEBUG] [aws-sdk-go] <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:iam::123456789000:user/terraform-admin</Arn>
    <UserId>AIDAICZZZZZZZZZZZZZZZZZZ</UserId>
    <Account>123456789000</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>edc3430e-ae0f-11e9-a298-c791ec8cb03d</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>
2019/07/24 14:38:24 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.20.19 (go1.12.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.5
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJYYYYYYYYYYYYYy/20190724/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=1f17e53eb5415e8a59a50cd3272725c55b143d23523a9b2aa2c79640353f258a
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190724T123824Z
Accept-Encoding: gzip


-----------------------------------------------------
2019/07/24 14:38:24 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 0/5, error RequestError: send request failed
caused by: Get https://xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request s3/ListObjects, attempt 1
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.20.19 (go1.12.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.5
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJYYYYYYYYYYYYYy/20190724/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=c426127dd7b52c313c0208dc5c6a93734c7ee2688403bcd99f17cee146df10cd
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190724T123825Z
Accept-Encoding: gzip


-----------------------------------------------------
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 1/5, error RequestError: send request failed
caused by: Get https://xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request s3/ListObjects, attempt 2
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.20.19 (go1.12.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.5
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJYYYYYYYYYYYYYy/20190724/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=c426127dd7b52c313c0208dc5c6a93734c7ee2688403bcd99f17cee146df10cd
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190724T123825Z
Accept-Encoding: gzip


-----------------------------------------------------
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 2/5, error RequestError: send request failed
caused by: Get https://xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request s3/ListObjects, attempt 3
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.20.19 (go1.12.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.5
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJYYYYYYYYYYYYYy/20190724/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=c426127dd7b52c313c0208dc5c6a93734c7ee2688403bcd99f17cee146df10cd
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190724T123825Z
Accept-Encoding: gzip


-----------------------------------------------------
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 3/5, error RequestError: send request failed
caused by: Get https://xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request s3/ListObjects, attempt 4
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.20.19 (go1.12.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.5
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJYYYYYYYYYYYYYy/20190724/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=c426127dd7b52c313c0208dc5c6a93734c7ee2688403bcd99f17cee146df10cd
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190724T123825Z
Accept-Encoding: gzip


-----------------------------------------------------
2019/07/24 14:38:25 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 4/5, error RequestError: send request failed
caused by: Get https://xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority
2019/07/24 14:38:26 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request s3/ListObjects, attempt 5
2019/07/24 14:38:26 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.20.19 (go1.12.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.5
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJYYYYYYYYYYYYYy/20190724/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=96d7c9fd32df782c852769b904c4b4915943d9fbb57d7fa119522cc994a042ab
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190724T123826Z
Accept-Encoding: gzip


-----------------------------------------------------
2019/07/24 14:38:26 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 5/5, error RequestError: send request failed
caused by: Get https://xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority

Error: Failed to get existing workspaces: RequestError: send request failed
caused by: Get https://xxxxxxxeu-central-1-terraform.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority

Crash Output

Expected Behavior


S3 Backend works

Actual Behavior


S3 Backend fails with message x509: certificate signed by unknown authority

Steps to Reproduce


Using a Makefile, following are the relevant steps that are executed. It fails during terraform init. 

prep: set-env ## Prepare a new workspace (environment) if needed, configure the tfstate backend, update any modules, and switch to the workspace
    @echo "$(BOLD)Verifying that the S3 bucket remote state bucket exists$(RESET)"
    @aws --profile $(AWS_PROFILE) s3api head-bucket --region $(REGION) --bucket $(BUCKET) > /dev/null 2>&1
    @if [ $$? -ne 0 ]; then \
        echo "$(BOLD)S3 Bucket was not found, creating new bucket with versioning enabled to store tfstate$(RESET)"; \
        aws --profile $(AWS_PROFILE) s3api create-bucket \
            --bucket $(BUCKET) \
            --acl private \
            --region $(REGION) \
            --create-bucket-configuration LocationConstraint=$(REGION); \
        echo; \
        aws --profile $(AWS_PROFILE) s3api put-bucket-versioning \
            --bucket $(BUCKET) \
            --versioning-configuration Status=Enabled; \
        aws --profile $(AWS_PROFILE) s3api put-bucket-encryption \
            --bucket $(BUCKET) \
            --server-side-encryption-configuration '{"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}]}';
     fi
    #####
    @echo "$(BOLD)Configuring the terraform backend$(RESET)"
    @echo "yes" | terraform init \
        -backend-config="profile=$(AWS_PROFILE)" \
        -backend-config="region=$(REGION)" \
        -backend-config="bucket=$(BUCKET)" \
        -backend-config="key=terraform.tfstate" || exit 1;
    #####
    @if [ ! -d terraform.tfstate.d/aws_$(REGION) ]; then \
        echo "$(BOLD)Configuring the terraform workspace$(RESET)"; \
        terraform workspace new aws_$(REGION)_$(ENV); \
     fi
    #####
    @echo "$(BOLD)Switching to workspace $(REGION)_$(ENV)$(RESET)"
    @echo "yes" | terraform workspace select aws_$(REGION)_$(ENV) || exit 1;
    #####
    @echo "$(BOLD)Updating TF modules$(RESET)"
    @terraform get -update=true || exit 1;
    @echo

Additional Context

References

backens3
Source
picture nodomain
👍1

Most helpful comment

My incredibly weird workaround (not sure why it works):

Depending on which resource fails (in this case registry.terraform.io for the comment above, or the s3 endpoint for the original issue post), you need to check which root CA is trusted for the certificate presented by that domain, and do a weird dance in Keychain

For instance, for the s3 endpoint, you can use curl or the browser to verify that the root CA trusted for it is "Baltimore CyberTrust Root".

  • Search for it in Keychain
  • Double click on it
  • Expand the "trust" section. The first dropdown should be set to "always trust". Change it to "never trust". Close the dialog, and authenticate to save the changes.
  • Double click on it again, expand trust and change it to "use system defaults" (my attempts to change it directly to this value, without going through "never trust" always failed silently. I blame the Keychain ui for being weird)

I encountered this problem with many programs written in Go using the standard TLS library.

This issue might describe a less "magic" approach to my workaround https://github.com/golang/go/issues/24652

picture signaleleven on 8 Apr 2020
👍6

All 9 comments

Unfortunately the S3 backend does not seem to support the insecure option like the AWS provider so I have no workaround.

nodomain picture nodomain on 24 Jul 2019

This happens to me on Catalina. Where does terraform/go take its certificates from?

```
2019/10/10 16:43:19 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 4/5, error RequestError: send request failed
caused by: Get https://XXXXX.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority
2019/10/10 16:43:20 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request s3/ListObjects, attempt 5
2019/10/10 16:43:20 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: XXXXX.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.22.0 (go1.12.9; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.9
Authorization: AWS4-HMAC-SHA256 Credential=XXXXXX/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=XXXXX
X-Amz-Content-Sha256: XXXX
X-Amz-Date: 20191010T144320Z
Accept-Encoding: gzip

2019/10/10 16:43:20 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 5/5, error RequestError: send request failed
caused by: Get https://girogate-staging-environments.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority```

alnr picture alnr on 10 Oct 2019
👍2

This happens to me on Catalina. Where does terraform/go take its certificates from?

2019/10/10 16:43:19 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 4/5, error RequestError: send request failed
caused by: Get https://XXXXX.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority
2019/10/10 16:43:20 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request s3/ListObjects, attempt 5
2019/10/10 16:43:20 [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: XXXXX.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go/1.22.0 (go1.12.9; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.9
Authorization: AWS4-HMAC-SHA256 Credential=XXXXXX/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=XXXXX
X-Amz-Content-Sha256: XXXX
X-Amz-Date: 20191010T144320Z
Accept-Encoding: gzip


-----------------------------------------------------
2019/10/10 16:43:20 [DEBUG] [aws-sdk-go] DEBUG: Send Request s3/ListObjects failed, attempt 5/5, error RequestError: send request failed
caused by: Get https://girogate-staging-environments.s3.eu-central-1.amazonaws.com/?prefix=env%3A%2F: x509: certificate signed by unknown authority```

so, who should we reach to for the Catalina ca-certificate issue?

fenglinnorway picture fenglinnorway on 13 Oct 2019

I have a similar issue, independent from any provider, even for a terraform init:

$ TF_LOG=TRACE terraform init
2019/12/16 15:33:10 [INFO] Terraform version: 0.12.18
2019/12/16 15:33:10 [INFO] Go runtime version: go1.12.13
2019/12/16 15:33:10 [INFO] CLI args: []string{"/Users/peter/test/terraform", "init"}
2019/12/16 15:33:10 [DEBUG] Attempting to open CLI config file: /Users/peter/.terraformrc
2019/12/16 15:33:10 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/12/16 15:33:10 [INFO] CLI command args: []string{"init"}

2019/12/16 15:33:10 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
Initializing the backend...
2019/12/16 15:33:10 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2019/12/16 15:33:10 [DEBUG] New state was assigned lineage "c16f61f9-2783-137e-b138-6a9efdfc0f7f"
2019/12/16 15:33:10 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2019/12/16 15:33:10 [TRACE] Meta.Backend: instantiated backend of type <nil>
2019/12/16 15:33:10 [DEBUG] checking for provider in "."
2019/12/16 15:33:10 [DEBUG] checking for provider in "/Users/peter/test"
2019/12/16 15:33:10 [DEBUG] checking for provisioner in "."
2019/12/16 15:33:10 [DEBUG] checking for provisioner in "/Users/peter/test"
2019/12/16 15:33:10 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2019/12/16 15:33:10 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2019/12/16 15:33:10 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2019/12/16 15:33:10 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2019/12/16 15:33:10 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2019/12/16 15:33:10 [TRACE] statemgr.Filesystem: read nil snapshot
2019/12/16 15:33:10 [DEBUG] checking for provider in "."
2019/12/16 15:33:10 [DEBUG] checking for provider in "/Users/peter/test"

2019/12/16 15:33:10 [DEBUG] plugin requirements: "random"=""
Initializing provider plugins...
2019/12/16 15:33:10 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2019/12/16 15:33:10 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Checking for available provider plugins...
2019/12/16 15:33:11 [DEBUG] Failed to request discovery document: Get https://registry.terraform.io/.well-known/terraform.json: x509: certificate signed by unknown authority

Registry service unreachable.

This may indicate a network issue, or an issue with the requested Terraform Registry.


Error: registry service is unreachable, check https://status.hashicorp.com/ for status updates

Sending a request to the above mentioned url via curl works fine:

$ curl https://registry.terraform.io/.well-known/terraform.json
{"modules.v1":"/v1/modules/","providers.v1":"/v1/providers/"}

We also checked the routing table for misconfigurations, but everything looks fine.
Running terraform init from different networks (corporate, mobile,..) yielded the same results.
Running terraform from a fresh user account on my macOS 10.15.2 works fine for any reason.

What we did find out:

  1. we run into this issue when performing a terraform release build from source (make bin)
  2. we don't run into this issue when performing a terraform dev build (make dev)

Any suggestions what might result in such a different behavior?

phjun picture phjun on 16 Dec 2019
👍2

My incredibly weird workaround (not sure why it works):

Depending on which resource fails (in this case registry.terraform.io for the comment above, or the s3 endpoint for the original issue post), you need to check which root CA is trusted for the certificate presented by that domain, and do a weird dance in Keychain

For instance, for the s3 endpoint, you can use curl or the browser to verify that the root CA trusted for it is "Baltimore CyberTrust Root".

  • Search for it in Keychain
  • Double click on it
  • Expand the "trust" section. The first dropdown should be set to "always trust". Change it to "never trust". Close the dialog, and authenticate to save the changes.
  • Double click on it again, expand trust and change it to "use system defaults" (my attempts to change it directly to this value, without going through "never trust" always failed silently. I blame the Keychain ui for being weird)

I encountered this problem with many programs written in Go using the standard TLS library.

This issue might describe a less "magic" approach to my workaround https://github.com/golang/go/issues/24652

signaleleven picture signaleleven on 8 Apr 2020
👍6

Thanks @signaleleven works for me.

betorvs picture betorvs on 8 Apr 2020

Thanks for writing it down. It helped me as well some time ago.

dmitrievav picture dmitrievav on 8 Apr 2020

Wow. Thanks.
On my machine there was a _specific_ trust setting for a certain domain. Removing this, everything works again. Crazy :D

I am going to close this now. Thanks @signaleleven!

nodomain picture nodomain on 11 May 2020
👍1

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashibot[bot] picture hashibot[bot] on 11 Jun 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

rjinski picture rjinski  ·  3Comments
larstobi picture larstobi  ·  3Comments
ronnix picture ronnix  ·  3Comments
shanmugakarna picture shanmugakarna  ·  3Comments
c4milo picture c4milo  ·  3Comments