Aws-vault: IAM calls dont work

[myoung34]

Not using assume role, my user has basically god mode permissions:

s3, ec2, all my calls work, but IAM calls do not

$ aws-vault exec work --debug -- aws ec2 describe-instances | head -n 3
2018/01/22 13:58:23 [keyring] Considering backends: [kwallet secret-service file]
2018/01/22 13:58:23 Loading config file /home/marc/.aws/config
2018/01/22 13:58:23 Parsing config file /home/marc/.aws/config
2018/01/22 13:58:23 Looking for sessions for work
2018/01/22 13:58:23 Looking up all keys in keyring
2018/01/22 13:58:23 Session "work session (1516665429)" expires in 3h58m45.39440184s
2018/01/22 13:58:23 Using session ****************ARBA, expires in 3h58m45.391367038s
2018/01/22 13:58:23 Setting subprocess env: AWS_DEFAULT_REGION=us-east-1, AWS_REGION=us-east-1
2018/01/22 13:58:23 Setting subprocess env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
2018/01/22 13:58:23 Setting subprocess env: AWS_SESSION_TOKEN, AWS_SECURITY_TOKEN
{
    "Reservations": [
        {

iam calls:

$ aws-vault exec work --debug -- aws iam get-user
2018/01/22 13:57:38 [keyring] Considering backends: [kwallet secret-service file]
2018/01/22 13:57:38 Loading config file /home/marc/.aws/config
2018/01/22 13:57:38 Parsing config file /home/marc/.aws/config
2018/01/22 13:57:38 Looking for sessions for work
2018/01/22 13:57:38 Looking up all keys in keyring
2018/01/22 13:57:38 Session "work session (1516665429)" expires in 3h59m30.499477845s
2018/01/22 13:57:38 Using session ****************ARBA, expires in 3h59m30.496242238s
2018/01/22 13:57:38 Setting subprocess env: AWS_DEFAULT_REGION=us-east-1, AWS_REGION=us-east-1
2018/01/22 13:57:38 Setting subprocess env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
2018/01/22 13:57:38 Setting subprocess env: AWS_SESSION_TOKEN, AWS_SECURITY_TOKEN

An error occurred (InvalidClientTokenId) when calling the GetUser operation: The security token included in the request is invalid

$ cat ~/.aws/config 
[profile work]
region = us-east-1

[myoung34]

without using aws-vault, using the access key id/secret that i used to create the aws-vault profile 'work'

$ AWS_ACCESS_KEY_ID=redact AWS_SECRET_ACCESS_KEY=redact aws iam get-user
{
    "User": {
        "UserName": "marc", 
        "PasswordLastUsed": "2018-01-22T11:47:29Z", 
        "CreateDate": "2016-08-29T13:00:40Z", 
        "UserId": "redact", 
        "Path": "/", 
        "Arn": "arn:aws:iam::redact:user/marc"
    }
}

[0xdabbad00]

Use --no-session. You can't call IAM functions with session tokens from STS which aws-vault uses.

[FernandoMiguel]

Scott I wonder if that is the same issue with rotating access keys I have on another issue here.

[0xdabbad00]

For documentation sake, this is referenced here: https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html

The temporary security credentials created by GetSessionToken can be used to make API calls to any AWS service with the following exceptions:
You cannot call any IAM APIs unless MFA authentication information is included in the request.
You cannot call any STS API except AssumeRole or GetCallerIdentity

[myoung34]

that worked with that flag @0xdabbad00 , im goinmg to do due diligence and assume roles in the mean time like i should be doing.

[lox]

Even better, add MFA to the user! I wonder if we could figure out a way to add a warning in this scenario, it's such a weird edge case of STS,

[FernandoMiguel]

He is adding mfa. He promised me
Only took two months of nagging

Hahaha

Thanks for the amazing tool guys!

[0xdabbad00]

When you make the IAM call from a session token, you have to use MFA at that time, not when you get the session token. So let's say you're running a script that makes a couple of IAM calls, you'd need to enter an MFA token each time, which would remove a lot of the value of running aws-vault in the first place.

To warn the user, you'd probably have to have aws-vault rewire the IAM end-point to be able to identify calls to it and then take action of some sort (ex. prompt the user for the MFA). It's an awkward thing to deal with. My current solution strategy though has not been ideal which has been to spend 5 minutes yelling "Why doesn't this work" and then remembering this gotcha and adding --no-session every time I bump into it.

[FernandoMiguel]

So when does this happen?
I don't think I ever see it, except for --rotate (changing access keys, and that's broken)
And I spend the all day inside aws-vault subshell or running --server (daemon agent)

[lox]

My current solution strategy though has not been ideal which has been to spend 5 minutes yelling "Why doesn't this work" and then remembering this gotcha and adding --no-session every time I bump into it.

👌🏻

Just tested it locally, I have two profiles:

[profile readonly]
region=us-east-1

[profile admin]
region=us-east-1
source_profile=readonly
mfa_serial=arn:aws:iam::1234567890:mfa/my.user.name
role_arn=arn:aws:iam::1234567890:role/assume-admin-access

With readonly, I can't do anything IAM-ish:

aws-vault exec readonly -- aws iam get-user 

An error occurred (InvalidClientTokenId) when calling the GetUser operation: The security token included in the request is invalid

But with the admin profile, which needs an MFA to get a session token, I can:

aws-vault exec admin -- aws iam get-user --user-name my.user.name
{
    "User": {
        "Path": "/",
         .....    
}
}

Subsequently, I am only prompted for a password for the admin session and it still works for IAM.

[lox]

(changing access keys, and that's broken)

It shouldn't be broken :( I've spent ages clearing out edge cases. It asked me for my MFA 3 times, but it worked :)

[FernandoMiguel]

https://github.com/99designs/aws-vault/issues/190

I'll give it another go tomorrow and also try no session flag

[FernandoMiguel]

FYI all my profiles have MFA, enforced by aws policy
Hence I would never find Marc issue