Terraform: S3 backend ignores explicit access_key and secret_key

[mogul]

Terraform Version

Terraform v0.12.28
+ provider.archive v1.3.0
+ provider.cloudfoundry (unversioned)
+ provider.local v1.4.0
+ provider.random v2.3.0

Terraform Configuration Files

% ls -a

.       ..   backend.tfvars  main.tf

% cat main.tf

terraform {
  backend "s3" {}
}

% cat backend.tfvars

bucket                = "[s3 bucket name]"
region                = "[bucket's region]"
key                   = "[a path]"
encrypt               = true
AWS_ACCESS_KEY_ID     = "[access key ID]"
AWS_SECRET_ACCESS_KEY = "[secret access key]"

% env | grep AWS

# No results...

% ls ~/.aws/credentials

ls: /Users/[redacted]/.aws/credentials: No such file or directory

% terraform init -backend-config=backend.tfvars

Initializing the backend...

Error: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

Debug Output

% TF_LOG=trace terraform init -backend-config=backend.tfvars

2020/07/22 13:52:27 [INFO] Terraform version: 0.12.28  
2020/07/22 13:52:27 [INFO] Go runtime version: go1.13.12
2020/07/22 13:52:27 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init", "-backend-config=backend.tfvars"}
2020/07/22 13:52:27 [DEBUG] Attempting to open CLI config file: /Users/[redacted]/.terraformrc
2020/07/22 13:52:27 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/07/22 13:52:27 [DEBUG] checking for credentials in "/Users/[redacted]/.terraform.d/plugins"
2020/07/22 13:52:27 [DEBUG] checking for credentials in "/Users/[redacted]/.terraform.d/plugins/darwin_amd64"
2020/07/22 13:52:27 [INFO] CLI command args: []string{"init", "-backend-config=backend.tfvars"}

Initializing the backend...
2020/07/22 13:52:27 [TRACE] Meta.Backend: merging -backend-config=... CLI overrides into backend configuration
2020/07/22 13:52:27 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value 704415183
2020/07/22 13:52:27 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/07/22 13:52:27 [DEBUG] New state was assigned lineage "b0896985-b65e-32ea-9038-2f05244702cc"
2020/07/22 13:52:27 [TRACE] Meta.Backend: moving from default local state only to "s3" backend
2020/07/22 13:52:27 [INFO] Setting AWS metadata API timeout to 100ms
2020/07/22 13:52:29 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2020/07/22 13:52:29 [INFO] Attempting to use session-derived credentials

Error: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

Crash Output

None, there was no crash.

Expected Behavior

The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY credential values specified in backend.tfvars should have been used to initialize the s3 backend, as documented.

Actual Behavior

The supplied credentials were ignored, and I was shown an irrelevant error message about the (implicitly used) AWS Provider.

I tried changing to using the keys access_key and secret_key in backend.tfvars and got the same result.

If I set the credentials in environment variables like so...

AWS_ACCESS_KEY_ID="[redacted]" AWS_SECRET_ACCESS_KEY="[redacted]" terraform init -backend-config=backend.tfvars

...it works fine.

Steps to Reproduce

1. terraform init -backend-config=backend.tfvars

Additional Context

Nothing, this is the simplest repro case I could come up with.

References

This seems relevant: https://github.com/hashicorp/terraform/issues/13589#issuecomment-293662383

Unfortunately the backend had its own code to configure the aws client, taken from the old s3 remote state which looks like it didn't properly configure the profile.

The S3 backend in the next Terraform release will be sharing the configuration code with the aws provider which solves this issue.

It seems like the S3 backend is not properly initializing the shared AWS configuration code with the credentials that were made available via the .tfvars file.

[onlydole]

Hello, @mogul!

I believe you need to set your credentials with these names via:

https://www.terraform.io/docs/providers/aws/index.html

access_key = "my-access-key"
secret_key = "my-secret-key"

[mogul]

Thanks @onlydole, but no such luck.

I tried changing to using the keys access_key and secret_key in backend.tfvars and got the same result.

The docs say both are accepted.

[onlydole]

Interesting 🤔 - thank you for filing an issue!

[jbardin]

Hi @mogul

Sorry about the confusion here. AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are supported, but they represent the standard AWS environment variables, not configuration attributes. To write those values into the configuration, you need to use access_key and secret_key.

[mogul]

Per my report:

I tried changing to using the keys access_key and secret_key in backend.tfvars and got the same result.

[jbardin]

@mogul,

Sorry, can you provide the exact config format and trace log output when using access_key and secret_key? Those should both be working correctly, which I confirmed locally just to double check.

[mogul]

Argh, I've tried to reproduce it and now it worked fine. 🤦 Must be that PEBKAC... Sorry for the distraction!

[mogul]

Reopening as the documentation point still stands: It should be clearer that the AWS_ options are referring to env vars, not keys for use in a .tfvars file.

[bflad]

Does the documentation present in the latest codebase, but not releasing until 0.13, satisfy the ask here? https://github.com/hashicorp/terraform/blob/master/website/docs/backends/types/s3.html.md#credentials-and-shared-configuration

[mogul]

Yes, definitely!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.