Terraform: AWS assume role not working

[hkalyana]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version -0.8.0

Run terraform -v to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.

Affected Resource(s)

Please list the resources as a list, for example:

• aws_provider

Copy-paste your Terraform configurations here - for large Terraform configs,

provider "aws"
{
    region = "${var.region}"
    assume_role {
    role_arn = "arn:aws:iam::<trustedaccount>:role/trustedrole"
    }
 }

Expected Behavior

Expected the resources to be created on the trusting account

Actual Behavior

Received error The role "arn:aws:iam:::role/trustedrole" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:
1) Created iam role in the trusting account with the necessary permission .
2) Created a policy on the trusted account with sts assume role permission and pointing the arm of the trusting account in the resource
3)Attached the policy to the user
4)Used the same role ARN of the trusting account in the terraform assume role

Important Factoids

The procedure I followed is working for me when I try to switch the roles in AWS GUI and I am able to create resources on the trusting account .

References

[aaomoware]

I am facing the same issue. Below is my configuration for the aws provider.

provider "aws" {
token = "${var.aws_token}"
region = "${var.aws_region}"
access_key = "${data.vault_generic_secret.aws_access_key.data["value"]}"
secret_key = "${data.vault_generic_secret.aws_secret_key.data["value"]}"

assume_role {
    role_arn = "arn:aws:iam::0000000000:role/MyRoleNAme"
}

}

Like @hkalyana I can assume the same role and create resources from the AWS GUI. The aws_*_keys are correct.

[aaomoware]

@hkalyana

I found a way round it. The terraform documentation on this issue is unclear.

follow: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html

remove the assume_role block; it's useless, for now.
assume_role {
role_arn = "arn:aws:iam::0000000000:role/MyRoleNAme"
}

the aws provider required attributes should be populated with the corresponding outputs you get when temporary credentials are requested from sts.amazonaws.com.

there are various ways you can make the call to sts. Anisble has a cool module for this specific purpose
http://docs.ansible.com/ansible/sts_assume_role_module.html

hope it helps you resolve your issue.

[alefauch]

Having exactly the same problem here.

Role delegation is working fine in both AWS Web Interface and aws CLI but not with terraform.

@aaomoware could you provide more details on your workaround ? If I get your post right, looks like you have to use an external tool to get temporary credentials and then feed them to terraform.

[aaomoware]

@alefauch

• hosts: localhost
  tasks:

  • name: get sessions
    sts_assume_role:
    mfa_token: "{{mfa_token}}"
    region: "eu-west-1"
    role_arn: "arn:aws:iam::0000000000000:role/RoleName"
    mfa_serial_number: "arn:aws:iam::000000000000:mfa/username"
    role_session_name: "NameofTheCreatedSession"
    register: assume_role

  • name: write credentials to vault
    shell: vault write secret/{{item.path}} value={{item.value}}
    environment:
    VAULT_ADDR: "{{ansible_env.VAULT_ADDR}}"
    VAULT_TOKEN: "{{ansible_env.VAULT_TOKEN}}"
    with_items:

    • { path: "RUDEBOY/AWS/access_key", value: "{{assume_role.sts_creds.access_key}}" }
    • { path: "RUDEBOY/AWS/secret_key", value: "{{assume_role.sts_creds.secret_key}}" }
    • { path: "RUDEBOY/AWS/session_token", value: "{{assume_role.sts_creds.session_token}}" }

I use vault to store my credentials....hence im writing to vault. Terraform supports reading/writing from and to vault.

You can use the aws api, wrapped in a script, to accomplish the above steps; i use ansible because it's easier, works well, and it's already in use in my project to configure instances brought up by terraform

You can add the ansible run to a cron job, every 1hr. The max duration for the temporary keys is 3600(1hr)

Notes:
http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html
http://docs.ansible.com/ansible/sts_assume_role_module.html

[jfriis]

Has anyone actually gotten assume_role to work? I'm trying to use it and have been seeing the same error as above. Would love a working example of the feature

[alefauch]

@jfriis

I have a working solution with Terraform and AWS with assume role.

Note that I'm not using Terraform to get a session token or to assume the role.

First you want to have a session, especialy if you have MFA activated like me.

aws sts get-session-token --profile <YOUR_AWS_SESSION_PROFILE> --duration-seconds <YOUR_AWS_SESSION_DURATION> --serial-number <YOUR_AWS_MFA_SERIAL> --token-code <YOUR_AWS_MFA_VALUE> --output text

is the same one that you have in ~/.aws/config usually "default"
is the session duration in seconds, I use 36000 (10 hours)
is the serial of you MFA if you use one this should look like arn:aws:iam::ACCOUNT_ID:mfa/YOUR_USERNAME
is the current MFA value of your token (usually 6 digits)

Note that you can skip --serial-number and --token-code if you are not using a MFA.

This command will return to you some tokens:
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN

You have to save them and source them in order to use them. The easy way is to store them in a file like this and the "source" the file in question in order to have those variables set in your environment (this is for Linux):

export AWS_ACCESS_KEY_ID=AAAAADDDDDAAAAA
export AWS_SECRET_ACCESS_KEY=BLABLABLA
export AWS_SESSION_TOKEN=LONNNGGGG_TOKEN

Then you can call the assume role:

aws sts assume-role --role-arn <YOUR_AWS_ASSUMRED_ROLE_ARN> --role-session-name <YOUR_AWS_ASSUMRED_ROLE_SESSION_NAME> --duration <YOUR_AWS_ASSUMED_ROLE_DURATION> --output text

: Will look like arn:aws:iam::ACCOUNT_ID:role/YOUR_ROLENAME
: you can put YOUR_ROLENAME as value
: is the assume role credentials duration in seconds, I use 3600 (1 hour) which is the maximum allowed value.

This command will return to you the same tokens but with new values:
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN

Like the previous ones, you have to save them and source them in order to use them. The easy way is to store them in a file and the "source" the file in question in order to have those variables set in your environment.

Note that you should save those credentials in a separate file than the get-session-token credentials.

Your Terraform provider configuration should look this this:

provider "aws" {
  region     = "${var.region}"
  access_key = "${var.aws_access_key}"
  secret_key = "${var.aws_secret_key}"
  token = "${var.aws_token}"
}

You have to populate the variables using the assume-role environment variables AWS_ACCESS_KEY_ID for access_key, AWS_SECRET_ACCESS_KEY for secret_key and AWS_SESSION_TOKEN for token.

Now you should be able to lauch Terraform without errors.

The reason you have to save those assume-role and get-session-token credentials separately is that if your get-session-token is still valid, you can directly source the get-session-token variables and call assume-role with them.

Hope I was clear enough ;)

[david-kli]

@jfriis In case you were still looking for confirmation, I was able to successfully use the assume role functionality with new 0.9 release.

I have a single entry in the aws credentials file which contains a personal IAM key from a Jump style account (no other AWS resources are created or allowed here except IAM resources).
~/.aws/credentials

[my-iamjump-account]
aws_access_key_id = <ACCESS_KEY>
aws_secret_access_key = <SECRET_KEY>
region = us-east-1

In my terraform project, I created a backend.tf file which contained the new remote backend config and the AWS provider config. The S3 bucket is created in the destination AWS account and all resources specified in the project were properly created.

terraform {
  backend "s3" {
    bucket = "my-terraform-bucket"
    key = "my-project/terraform.tfstate"
    region = "us-east-1"
    profile = "my-iamjump-account"
    role_arn = "arn:aws:iam::<DESTINATION_AWS_ACCOUNT_NUMBER>:role/user/myadminrole"
 }
}

provider "aws" {
  profile = "my-iamjump-account"
  region = "us-east-1"
  assume_role {
    role_arn = "arn:aws:iam::<DESTINATION_AWS_ACCOUNT_NUMBER>:role/user/myadminrole"
    session_name = "terraform"
  }
}

This setup allowed me to terraform init and terraform apply without any AWS environment variables configured.

[agrmckinnon]

@david-kli
In the example above is the role being assumed in the same AWS account as the user?

I have been able to successfully do this with 0.9 (great), but cannot do this if the user's access keys belong to Account A and the role to be assumed is in Account B I receive the following

(these are 'bona fide' accounts and roles - which allow the role to be assumed successfully via the AWS Console - switch roles functionality)

* module.aws_cloudtrail.provider.aws.${var.alias}: 
The role "arn:aws:iam::<ACCOUNT_B_ACCTNO>:role/<ROLE2ASSUME>" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

My use case is that I have a hub and spoke scenario where I want to use the hub account to build out infrastructure on spoke accounts using a user from the hub account who has permissions to assume a role in each spoke account.

Would appreciate if you or anyone can confirm if you can / cannot do this across account.

[david-kli]

@agrmckinnon
My example had the user and role in separate accounts.

I use a similar hub and spoke setup which requires a minimal amount of terraforming using the root account during initial account setup. After that, the build of the build out is accomplished with assumed roles.

The aws jump account in my setup only contains IAM users and group definitions. I create one group here per unique remote account & role combination. The group policies contain the sts:assumeRole for the destination account/role arn. Each remote account has the roles defined with a trust relationship defined back to the jump account.

Your error message makes me wonder if something is getting lost with the provider usage inside a module. You mentioned you can assume the role with the AWS console. Can you also perform similar actions with the aws cli? It might also be worthwhile to work up a test which does not use a provider inside in a module.

[agrmckinnon]

@david-kli
Thank you for the quick feedback and support - After a thorough review and a minor tweak I have this working now

In my case we also had the role available to the user for console access and I had added the condition to the trust policy

,
    "Condition": {"Bool": {"aws:MultiFactorAuthPresent": "true"}}

This enforced that the user had to have MFA'd via the console before assuming the role. This had the detrimental effect of receiving the error, as defined in the previous post from me, when assuming the role via terraform.

After removing the condition from the trust policy I can successfully assume the role on the account on the spoke account via Terraform.

Our goal is to have our production activities undertaken by an automation tool using instance profiles and assuming roles.

For development there will be an element of running terraform with assume roles from an engineers own credentials.

In the short/medium term I plan to separate the engineers user into two users to avoid future confusion

• One to access the console with MFA
• One to access the api's without MFA

Thanks Again

[FransUrbo]

I'm having problem getting assume role to work on v0.9.4:

~/.aws/config

Doesn't exist - but I'm hoping to put only the region here and get rid of it from the *.tf files

~/.aws/credentials:

[root]
# Account: ACCOUNT1_ID
aws_access_key_id = ACCOUNT1_ACCESSKEY
aws_secret_access_key = ACCOUNT1_SECRETKEY

[core]
# Account: ACCOUNT2_ID
aws_access_key_id = ACCOUNT2_ACCESSKEY
aws_secret_access_key = ACCOUNT2_SECRETKEY

[test]
# Account: ACCOUNT3_ID
role_arn = arn:aws:iam::ACCOUNT3_ID:role/Cross_Account
source_profile = core

provider.tf

provider "aws" {
  region  = "eu-west-1"
  profile = "core"
  alias   = "core"
}

provider "aws" {
  region  = "eu-west-1"
  profile = "test"
}

remote.tf

data "terraform_remote_state" "base" {
  backend   = "s3"
  config {
    bucket  = "my-terraform-bucket"
    key     = "terraform-base.tfstate"
    region  = "eu-west-1"
    profile = "root"
  }
}

data "terraform_remote_state" "core" {
  backend   = "s3"
  config {
    bucket  = "my-terraform-bucket"
    key     = "terraform-core.tfstate"
    region  = "eu-west-1"
    profile = "root"
  }
}

data "terraform_remote_state" "test" {
  backend   = "s3"
  config {
    bucket  = "my-terraform-bucket"
    key     = "terraform-test.tfstate"
    region  = "eu-west-1"
    profile = "root"
  }
}

The following shell commands work just fine:

$ aws iam list-users --profile root
$ aws iam list-users --profile core
$ aws iam list-users --profile test

So the aws command sees nothing wrong with my credentials file.

However, terraform_0.9.4 refresh gives me:

data.terraform_remote_state.core: Refreshing state...
data.terraform_remote_state.test: Refreshing state...
data.terraform_remote_state.base: Refreshing state...
data.aws_caller_identity.core: Refreshing state...
Error refreshing state: 1 error(s) occurred:

* provider.aws: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

For testing, I have disabled the MultiFactorAuthPresent requirement in arn:aws:iam::ACCOUNT3_ID:role/Cross_Account.

[drdamour]

i'm getting what @FransUrbo is getting in tf 9.4....is MFA supposed to work (will TF prompt my for my MFA token?)

[ashb]

@drdamour No, at current it will not :(

[FransUrbo]

For me, it doesn't work even if i disable MFA everywhere...

[drdamour]

is the problem people are running into possibly that you can't assume the role if you are already in the role? that's what we run into from time to time.

I can assume roles from my main session...and thus assume_role works fine...but if i use aws cli to assume the role then running terraform fails on teh assume_role call...as my new role does not have permissions to assume itself.

seems like terraform should check if the session role is already the role defined in assume_role before it tries to assume it...however i don't know if the aws api allows you to get your current role..and i don't know go so can't really test...anyone got some skills?

looks like you CAN get the current role http://docs.aws.amazon.com/cli/latest/reference/iam/get-role.html

[FransUrbo]

@drdamour If you look at my code at the top, you'll see that's not what's happening for me. At least, I don't think so.

I have my core account, which then assumes a role in the other account.

[ojkelly]

In case anyone comes across this, I've adapted a work around from kops

Save the following to a file called terraform-mfa.

Then make it executable with $ chmod +x terraform-mfa.

Then put it somewhere in your $PATH: $ mv terraform-mfa /usr/local/bin

Then call terraform-mfa instead of terraform.

#!/usr/bin/env bash

# From https://github.com/kubernetes/AWS/issues/226#issuecomment-278879348
# and https://github.com/kubernetes/AWS/blob/master/docs/mfa.md

# set -euo pipefail

main() {
    local role_arn="${AWS_MFA_ROLE_ARN:-}"
    local serial_number="${AWS_MFA_ARN:-}"
    local token_code

    if [ -z "${role_arn}" ]; then
        echo "Set the AWS_MFA_ROLE_ARN environment variable" 1>&2
        return 1
    fi

    if [ -z "${serial_number}" ]; then
        echo "Set the AWS_MFA_ARN environment variable" 1>&2
        return 1
    fi

    echo -n "Enter MFA Code: "
    read -s token_code

    # NOTE: The keys should not be exported as AWS_ACCESS_KEY_ID
    # or AWS_SECRET_ACCESS_KEY_ID. This will not work. They
    # should be exported as other names which can be used below. This prevents
    # them from incorrectly being picked up from libraries or commands.
    temporary_credentials="$(aws \
        sts assume-role \
        --role-arn="${role_arn}" \
        --serial-number="${serial_number}" \
        --token-code="${token_code}" \
        --role-session-name="terraform-access"
    )"

    unset AWS_PROFILE

    export "AWS_ACCESS_KEY_ID=$(echo "${temporary_credentials}" | jq -re '.Credentials.AccessKeyId')"
    export "AWS_SECRET_ACCESS_KEY=$(echo "${temporary_credentials}" | jq -re '.Credentials.SecretAccessKey')"
    export "AWS_SESSION_TOKEN=$(echo "${temporary_credentials}" | jq -re '.Credentials.SessionToken')"

    exec terraform "$@"
}


main "$@"

[catsby]

Hey all – this issue has been migrated to the new AWS provider repo, here: https://github.com/terraform-providers/terraform-provider-aws/issues/472

I've posted a follow up comment there with an example config, if anyone can take a look, thanks!

https://github.com/terraform-providers/terraform-provider-aws/issues/472#issuecomment-308311071

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.