Terraform-provider-aws: AssumeRoleTokenProviderNotSetError when using assume_role with mfa enabled

[chrishowell]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform version: 0.12.10
Go runtime version: go1.13.1
provider.aws ~> 2.32.0

Affected Resource(s)

• provider "aws"

Terraform Configuration Files

env

AWS_SDK_LOAD_CONFIG=1
AWS_PROFILE=bar-admin

~/.aws/config

[default]
region = eu-west-1
output = json

[profile bar-default]
aws_access_key_id=<key>
aws_secret_access_key=<secret>

[profile bar-admin]
role_arn=arn:aws:iam::<account_id>:role/admin
source_profile=bar-default
mfa_serial=arn:aws:iam::<account_id>:mfa/ch

resource "aws_iam_group" "admins" {
  name = "admins"
}

resource "aws_iam_role" "admin_mfa_role" {
  name = "admin"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::${data.aws_caller_identity.current.account_id}:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "Bool": {
          "aws:MultiFactorAuthPresent": "true"
        }
      }
    }
  ]
}
EOF
}

resource "aws_iam_group_policy_attachment" "admins_assume_role" {
  group      = "${aws_iam_group.admins.name}"
  policy_arn = "${aws_iam_policy.assume_role.arn}"
}

resource "aws_iam_policy" "assume_role" {
  name = "assume_role"

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": "arn:aws:iam::*:role/*"
  }
}
EOF
}

resource "aws_iam_user" "ch" {
  name = "ch"
}

resource "aws_iam_user_login_profile" "ch_login" {
  user    = "${aws_iam_user.ch.name}"
  pgp_key = "keybase:chrishowell"
}

resource "aws_iam_user_group_membership" "ch_groups" {
  user = "${aws_iam_user.ch.name}"

  groups = [
    "${aws_iam_group.admins.name}"
  ]
}

Debug Output

https://gist.github.com/chrishowell/ddd169c24ba4f0fcaba70a3e2f624a5a

Panic Output

N/A

Expected Behavior

As of terraform-provider-aws_v2.32.0 I believe assume_role with mfa enabled should work.

Actual Behavior

Error: error creating EC2 Metadata session: AssumeRoleTokenProviderNotSetError: assume role with MFA enabled, but AssumeRoleTokenProvider session option not set.

Steps to Reproduce

1. terraform plan

Important Factoids

Running locally on Mac OSX Catalina
Brew install of Terraform
No ~/.aws/configuration file present

References

• #10379
• #5592

[lachlancooper]

I can reproduce this issue exactly as described.

I noticed there's a difference in behaviour between specifying the profile in an env var (AWS_PROFILE=bar-admin) vs. the provider config directly (profile = "bar-admin"). Using the env var results in the above error creating EC2 Metadata session message, which (since we're running locally, not on EC2) is perhaps a clue to why this is failing. I think this is closely related to https://github.com/hashicorp/aws-sdk-go-base/issues/7

Specifying the profile in provider config still doesn't work though, and I can't see any indication in the trace logs that terraform is even trying to assume a role. The only error I get is:

Error refreshing state: AccessDenied: Access Denied
    status code: 403, request id: 795ACFB7B863F762, host id: XRoHa+G5vU0TKxQkWwyld0p6g/u5Er8T/rd3NjU1rd7odmsFi1JGiZXb3/q42O2DYOCaByEmoFA=

[sean-kang]

Any update on this issue? I am having the exactly same problem. I think that using assume_role with MFA is probably the most common way of access control in big organizations. In my opinion, specifying profile should be enough for Terraform to recognize the use of assume_role with MFA since the configuration is already in ~/.aws/config and ~/.aws/credentials.

[ryan-gerstenkorn-sp]

As a workaround for anyone running into this when using aws-vault (maybe others?) it seems you can remove mfa_serial from the role profile while having it still in the base profile and it will work.

So for example:

[profile base]
credential_process = sh -c 'aws-vault exec base --json 2> $(tty)'
mfa_serial = arn:aws:iam::<redacted>:mfa/<user_name>
region = us-east-1

[profile assumed_role]
role_arn = arn:aws:iam::<redacted>:role/<role_to_assume>
source_profile = base
region = us-east-1

Note that the assumed_role does not have a mfa_serial option set.

And then for your provider something like this should work:

provider "aws" {
  region  = "us-east-1
  profile = "assumed_role"
}

One problem is this role doesn't seem to work with aws-vault now. To fix that you will likely want to make an assumed_role profile and an assumed_role_tf profile to support both.

It almost seems like in this specific case if an error was not raised, this would simply work.

[mikkopiu]

One problem is this role doesn't seem to work with aws-vault now. To fix that you will likely want to make an assumed_role profile and an assumed_role_tf profile to support both.

To add (for aws-vault users) there's also include_profile so you can do this:

[profile base]
credential_process = sh -c 'aws-vault exec base --json 2> $(tty)'
mfa_serial = arn:aws:iam::<account a>:mfa/<user_name>
region = us-east-1

[profile assumed_role]
role_arn = arn:aws:iam::<account b>:role/<role_to_assume>
source_profile = base
region = us-east-1
# For aws-vault only:
include_profile = base

This way aws-vault exec will know about the mfa_serial and prompt for MFA accordingly, and Terraform won't see the mfa_serial in assumed_role and lets aws-vault do the work.

[alexander-wiechert]

this does not work for me. Below is my ~/.aws/credentials file.

`[default]
region = eu-central-1
aws_access_key_id = xxx
aws_secret_access_key = xxx
mfa_serial = arn:aws:iam::acount_id_1:mfa/username
credential_process = sh -c 'aws-vault exec default --json 2> $(tty)'

[profile_2]
role_arn = arn:aws:iam::account_id_2:role/SuperAdmin
source_profile = default
include_profile = default`

In terraform I setup the provider like this
provider "aws" { region = eu-central-1 profile = "profile_2" }

If I issue the terraform command I got this:

terraform init --backend-config=backends/mb.tfbackend
Initializing modules...

Initializing the backend...

Error: No valid credential sources found for AWS Provider.
Please see https://terraform.io/docs/providers/aws/index.html for more information on
providing credentials for the AWS Provider

Additionaly I had to issue

aws-vault add default
for another test I tried this
aws --profile profile_2 s3 ls

An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::account_1:user/ai75169 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::account_2:role/SuperAdmin