Terraform: shared_credentials_file doesn't work on AWS provider

[dlanger]

Using the shared_credentials_file parameter in the AWS provider gives me permission errors when I'm trying to use a role defined in that file.

In order to get the functionality to work, I had to not use that parameter, and export AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY instead - and even this I had trouble recreating a few hours later.

---

I have ~/.aws/credentials set up to enable me to use STS to assume different roles and move between different accounts, assuming I've used MFA when logging in:

[default]
aws_access_key_id = AAA
aws_secret_access_key = BBB

[dev]
role_arn=arn:aws:iam::###:role/CCC
source_profile=default
mfa_serial = arn:aws:iam::###:mfa/EEE
region=us-east-1

I've verified that this works using the Python AWS CLI.

According to the docs I should be able to set shared_credentials_file and profile in the provider:

provider "aws" {
    shared_credentials_file = "~/.aws/credentials"
    profile = "dev"
}

and have things work. When I do this, however, I get this error:

$ terraform plan
Refreshing Terraform state prior to plan...

Error refreshing state: 1 error(s) occurred:

* 1 error(s) occurred:

* No valid credential sources found for AWS Provider. 
  Please see https://terraform.io/docs/providers/aws/index.html for more information on 
  providing credentials for the AWS Provider

This led to some investigation, and it turns out that (at least on my system) something is up when I use ~ to refer to my home directory:

$ echo ~        
/home/dan

$ strace -f -e openat terraform plan 2>&1 | grep ".aws"
[pid  5739] openat(AT_FDCWD, "~/.aws/credentials", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

$ strace -f -e openat terraform plan 2>&1 | grep ".aws"
[pid  5886] openat(AT_FDCWD, "/home/dan/.aws/credentials", O_RDONLY|O_CLOEXEC) = 7

Between the two terraform plan calls I edited my tf file to use the full path. Even once I switched to using /home/dan/.aws/credentials for shared_credentials_file, I got the same error as above.

After some more trial and error, the winning combination ended up using this provider definition:

provider "aws" {
    profile = "dev"
    region = "us-east-1"
}

and manually exporting AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY into the environment.

Happy to add some logs if this will help, but I'm not sure which ones to add.

On the latest Terraform:

$ terraform --version
Terraform v0.6.12

[atward]

source_profile/role_arn is not supported by aws-sdk-go. This is a dupe of #1275.

[dlanger]

Alas - we should update the docs to remove references to this.

[ngamradt]

I am using terraform v0.6.15 and I am running into the same problem with the tilde (~) character to point to my AWS credentials file. The folowing should work, but does not:

provider "aws" {
region = "us-east-1"
shared_credentials_file = "~/.aws/credentials"
}

When I hardcode the full path, things work:

provider "aws" {
region = "us-east-1"
shared_credentials_file = "/home/user/.aws/credentials"
}

Is there a fix planned for this specific issue? I have found a work-around for the time-being, but it is ugly.

[Kuberchaun]

Same issue here on Terraform v0.6.16. It's completely reasonable to have ~/.aws/credentials and not have to path the entire thing out.

[dlanger]

We've had good luck using this wrapper in the interim.

[murphyslaw]

In case you are using the default AWS credentials located at ~/.aws/credentials, I think you can get it to work by only setting the profile attribute, but remove the shared_credentials_file attribute. It is slightly hidden in the documentation https://www.terraform.io/docs/providers/aws/#shared_credentials_file.

provider "aws" {
  region = "us-east-1"
  profile = default

  # note: Do not set the path to the credentials file, in order to use the default AWS credentials.
  # shared_credentials_file = "~/.aws/credentials"
}

[dlanger]

@murphyslaw That didn't work for me when I last tried it (as the link above mentions, the underlying SDK doesn't/didn't support STS), but things may have changed since then. Did you try it and have it work?

[hamstah]

Last time I tried, the assume role didn't work either.

Here is how I work around it

I have a script I use that uses boto to get a temporary token with sts and add/replace it in my awscli credentials file under a different profile.

The file has profile-a I added myself, then when I assume a role for account B it creates an entry profile-b-assumed in the credentials file.

Then in terraform I use the `profile = "profile-b-assumed" to use it.

[murphyslaw]

Yes, I was using this on my local machine yesterday. As I said, I only put in the profile attribute, but removed the shared_credentials_file entry. I'm using terraform version 0.6.16 on MacOS.

[xpaulz]

aws-sdk-go now supports assume role and the rest of ~/.config by default as of 27 days ago... https://github.com/aws/aws-sdk-go/issues/472

However, it looks like terraform won't just magically start working; it seems the credential providers chain is not the same as the "default credentials chain provider" for func getCredentials in
https://github.com/hashicorp/terraform/blob/master/builtin/providers/aws/auth_helpers.go

may need to be refactored to use aws/session instead?

[dlanger]

The ~ seems like a (fixed) dupe of https://github.com/hashicorp/terraform/issues/11268 - no idea on the state of shared_credentials_file though.

[luntzel]

@murphyslaw I had to put profile = "default" (with quotes) to get this to work. Thanks!

[zsalamov]

I have same problem with Terraform v0.8.6
By default it goes and takes default profile, but when specifying shared_credentials_file with profile ( according docs profile works only with shared_credentials_file) I got:
No valid credential sources found for AWS S3 remote
Exporting works fine:
export AWS_SHARED_CREDENTIALS_FILE=/Users/salamov/.aws/credentials
export AWS_PROFILE=STAGE

[joaquinrinaudo-olx]

I'm running into a similar problem a similar problem on v0.8.6. When using cross account access, I get "You are not authorized to perform this operation." since it's trying to create the resources on the base account rather than on the one from the assume-role.

aws sts get-caller-identity outputs the correct AWS account.

~/.aws/credentials:

[default]
aws_access_key_id=ZZZZZ
aws_secret_access_key=YYYY
aws_session_token=XXXXX`

[JohnBloom]

I am having the same issue with v0.9.5:

provider "aws" {
shared_credentials_file = "/home/john/.aws/credentials"
region = "${var.region}"
profile = "myprofile"
}

I have tried different combinations of properties in the aws provider but the error is the same.

[oli-g]

This is still not working for me with 0.9.4, even if this PR on AWS Go SDK is merged already since months.

[damianh]

Not working 0.9.8 Might be picnic; still investigating.

[ngamradt]

Unless you have a specific need for Terraform (working with different cloud
vendors for one product). I would highly recommend switching to
CloudFormation if you are dealing with AWS. It too isn't perfect, but less
buggy than Terraform and more tightly integrated with AWS.

On Thu, Jun 8, 2017 at 11:08 AM, Damian Hickey [e-mail]
wrote:

Not working 0.9.8

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/terraform/issues/5610#issuecomment-307132977,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABYKA1JAr9pXzYYRLde9UC-feizCeo47ks5sCA6GgaJpZM4HvnhP
.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.