Terraform-provider-aws: Error: No valid credential sources found for AWS Provider.

[hashibot[bot]]

_This issue was originally opened by @florin-waterford as hashicorp/terraform#24768. It was migrated here as a result of the provider split. The original body of the issue is below._

---

Terraform randomly fails to read the AWS credentials from ~/.aws.

Terraform Version

Terraform v0.12.24

(macOS 10.15)

Terraform Configuration Files

variable "account" {
  default = "123123123123"
}

variable "region" {
  default = "eu-east-1"
}

variable "bucket-region" {
  default = "us-east-1"
}

Debug Output

$ terraform init

2020/04/26 18:20:39 [INFO] Terraform version: 0.12.24  
2020/04/26 18:20:39 [INFO] Go runtime version: go1.13.8
2020/04/26 18:20:39 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init"}
2020/04/26 18:20:39 [DEBUG] Attempting to open CLI config file: /Users/USER/.terraformrc
2020/04/26 18:20:39 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/04/26 18:20:39 [INFO] CLI command args: []string{"init"}
2020/04/26 18:20:39 [TRACE] ModuleInstaller: installing child modules for . into .terraform/modules
Initializing modules...
2020/04/26 18:20:39 [DEBUG] Module installer: begin module1
2020/04/26 18:20:39 [TRACE] ModuleInstaller: Module installer: module1 <nil> already installed in ../modules/module1
2020/04/26 18:20:39 [TRACE] modsdir: writing modules manifest to .terraform/modules/modules.json

Initializing the backend...
2020/04/26 18:20:39 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value 782968548
2020/04/26 18:20:39 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/04/26 18:20:39 [DEBUG] New state was assigned lineage "12312312-1231-1231-1231-123123123123"
2020/04/26 18:20:39 [TRACE] Meta.Backend: moving from default local state only to "s3" backend
2020/04/26 18:20:39 [INFO] Setting AWS metadata API timeout to 100ms
2020/04/26 18:20:40 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2020/04/26 18:20:40 [INFO] Attempting to use session-derived credentials

Error: No valid credential sources found for AWS Provider.
        Please see https://terraform.io/docs/providers/aws/index.html for more information on
        providing credentials for the AWS Provider

Expected Behavior

It should read credentials from ~/.aws.

Actual Behavior

It can't find the credentials.

Steps to Reproduce

terraform init

Additional Context

This breaks every once in a while. I'm not sure when. Removing the .terraform directory doesn't help. Reinstalling Terraform doesn't fix it. It works if I copy the working project in another directory and run terraform init, it works, but I'd rather not do that every time TF breaks.

Setting the AWS_PROFILE, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables fixes this, but it should not be needed.

[rkmudunuru]

I am also facing the same issue while performing terraform init using AWS role for s3 backend. Below is my configuration

terraform version
Terraform v0.12.24
+ provider.aws v2.59.0
+ provider.template v2.1.2

Backend Configuraton : terraform.tf

provider "aws" {
  region  = "us-east-1"
  profile = "test-profile"
}

terraform {
  backend "s3" {
    encrypt = true
    bucket = "test-bucket"
    dynamodb_table = "test-ddb"
    region = "us-east-1"
    key = "terraform.tfstate"
  }
}

test-profile resides in the location ~/.aws

Below is the error when I try to initialize

$terraform init
Initializing modules...

Initializing the backend...

Error: No valid credential sources found for AWS Provider.
        Please see https://terraform.io/docs/providers/aws/index.html for more information on
        providing credentials for the AWS Provider

[rkmudunuru]

I managed to get around this issue by export AWS_SDK_LOAD_CONFIG=1

[florin-waterford]

Fixed: I had AWS_PROFILE set in my environment variables and the other variables were missing. The keys were stored in my ~/.aws/credentials config in the [default] section and TF didn't follow the source_profile = default under my profile's section in the credentials file.

The fix is one of:

• set all AWS environment variables
• unset all AWS environment variables, in which case TF will read the keys from the default profile
• copy the keys from default into each profile in the credentials file

[dkciscosystems]

Solved: See, terraform init would not look into the provider configuration or credentials file as it is supposed to initialize the backend within the terraform { } block. So it would rather look for the credentials inside the block itself. Do this:
Add the secret and access key inside the block as you would add it in the provider block. This works!
Make sure you don't push it to github my mistake :)

terraform {
backend "s3" {
access_key = <>
secret_key =<>
bucket = <>
key = "/terraform.tfstate"
region = <>
# dynamodb_table = ""
encrypt = true

}
}

[bflad]

Hi folks 👋 A lot of the issues described above should be resolved with ensuring that if you are using the Terraform S3 Backend, that you are separately configuring it as compared to the Terraform AWS Provider. While the two share similar authentication mechanisms for consistency purposes, they are separately configurable. Other issues, such as the Terraform S3 Backend not correctly reading its profile configuration, should be resolved as of Terraform 0.13.0-beta2. Similar enhancements and bug fixes will also be applied to the Terraform AWS Provider with the upcoming version 3.0.0 release in the coming weeks.

[bflad]

The authentication changes applied to the Terraform S3 Backend as part of Terraform CLI 0.13.0-beta2 have also been merged into the Terraform AWS Provider and will release with version 3.0.0, likely in the next two weeks. Please follow the v3.0.0 milestone for tracking the progress of that release. If you are still having trouble after updating when its released, please file a new issue. Thanks!

[hashibot[bot]]

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!