Terraform: Terraform fails to authenticate using STS temporary credentials - InvalidClientTokenId

[cabarria]

I'm running Terraform v0.6.15. I have exported the following keys following an STS call for credentials:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN (Can use AWS_SECURITY_TOKEN as well)

the aws-cli works, but Terraform complains with the following:

Refreshing Terraform state prior to plan...

Error refreshing state: 1 error(s) occurred:

1 error(s) occurred:

InvalidClientTokenId: The security token included in the request is invalid
status code: 403, request id:

No matter what I do... I've tried AWS_TOKEN as the key as well, Terraform fails to use temporary credentials. If I use a Permanent Access Key and Secret Access Key it works. I thought this issue had been solved.

[cabarria]

Looks like this is failing because the temporary credentials are being requested using the GetFederationToken API. http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html

Those temporary credentials are not allowed to make IAM API or STS calls. There must be some code in Terraform checking for that regardless of whether the user intends to work with IAM or not.

I think this should be fixed... Terraform shouldn't presume to know that the credentials provided are not adequate to perform whatever actions the user needs.

[radeksimko]

Hi @cabarria
I was able to reproduce this. You're right in a way that Terraform currently uses iam:GetUser to validate credentials.

We may be able to swap this with a fairly new API endpoint sts:GetCallerIdentity which wasn't available before. It seems that this API method is always available, even in regions that have explicitly disabled STS support.

I need to confirm this is expected and correct behaviour w/ AWS support and then we should be good to make that change.

This also makes me think that our recent changes for allowed_account_ids & forbidden_account_ids won't work yet for federated accounts due to this validation we have in place. they will mostly work because credentials validation ignores certain error codes coming from iam:GetUser.

[radeksimko]

I just verified that sts:GetCallerIdentity works on EC2 instances too.

I also approached the AWS support in regards to disabled STS regions and the representative told me to wait until they hear confirmation back from the service team. I think I will prepare a PR in the meantime though.

[radeksimko]

See PR that is fixing this at #6536

[cabarria]

Thanks for the quick response!!!

[riponbanik]

is it being fixed on terraform v0.7.13. Stil having issue

export AWS_ACCESS_KEY=$aws_access_key
export AWS_SECRET_KEY=$aws_secret_key
export AWS_SECURITY_TOKEN=$aws_security_token

terraform apply -var ami_id=$ami_id -var aws_access_key=$aws_access_key -var aws_secret_key=$aws_secret_key -var aws_region=$AWS_DEFAULT_REGION

[lewismarshall]

Still having the issue in version v0.8.5 although it works with just terraform plan with skip_credentials_validation = true:

provider "aws" {
  access_key = ""
  secret_key = ""
  skip_credentials_validation = true
}

I'm using MFA and grabbing STS creds from:

local creds_json=$(aws sts assume-role --output json --role-arn ${role_arn} --profile ${profile} --role-session-name ${profile} )

export AWS_ACCESS_KEY_ID=$(echo ${creds_json} | jq -r '.Credentials.AccessKeyId')
export AWS_SECRET_ACCESS_KEY=$(echo ${creds_json} | jq -r '.Credentials.SecretAccessKey')
export AWS_SECURITY_TOKEN=$(echo ${creds_json} | jq -r '.Credentials.SessionToken')
export AWS_SESSION_EXPIRE=$(echo ${creds_json} | jq -r '.Credentials.Expiration')

Error:

Error applying plan:

1 error(s) occurred:

* aws_instance.example: Error launching source instance: AuthFailure: AWS was not able to validate the provided access credentials
        status code: 401, request id: 25c3671f-2e13-4f02-a7ba-825d20ea441c

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure

[lewismarshall]

OK so It works without skip_credentials_validation once export AWS_SESSION_TOKEN="${AWS_SECURITY_TOKEN}" is also set!

[satyamuralidhar]

https://github.com/immurali5/Terraform

[kcd83]

I think the use of AWS_SESSION_TOKEN instead of AWS_SECURITY_TOKEN could be documented here https://www.terraform.io/docs/providers/aws/index.html

But thanks for this, it resolved my issue

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.