Aws-vault: Can aws-vault handle setting default region automatically?

[jleclanche]

I don't use aws credentials files on my dev machine, which means unless I explicitly do AWS_REGION=us-east-1 aws-vault exec example, the region is not set in the resulting session.

Is there a way to have aws-vault set the (default?) region on the profile at profile creation, so that unless explicitly overridden, it's just in there as well?

[FernandoMiguel]

$ cat ~/.aws/config
[profile default]
region = eu-west-1

easy fix

[jleclanche]

Right, but like I said I'm not using aws config files and I'd like to avoid introducing that dependency if possible.

In other words, it'd be nice to have aws-vault add example --region=us-east-1 as a parameter and have the region be handled that way.

[lox]

You did say credential files, and ~/.aws/config isn't a credential file, it's a config file. Can you explain a bit more about why you don't want config files on your system? They are a pretty integral part to configuring aws-vault.

[lox]

If anything, in future, aws-vault add will create and populate ~/.aws/config in future.

[jleclanche]

My bad, I meant config and credential files :)
It's not so much that I don't want them on my system, it's that it's extra work to manage. If I can have the region managed by aws-vault then the entire thing is handled without pre-generating a config file.

[jleclanche]

I tried setting ~/.aws/config to your recommendation and it did not help, by the way. It looks like AWS_SDK_LOAD_CONFIG needs to also be set, which kind of defeats the point (might as well just load AWS_REGION=us-east-1 AWS_DEFAULT_REGION=us-east-1 into the environment manually).

[lox]

I think @FernandoMiguel might have gotten confused between [default] and [profile blah].

aws-vault reads your profiles, as you specify one by name in aws-vault exec <profile>. So you can add a region per-profile:

[profile blah]
region = us-east-1

aws-vault exec blah -- env | grep AWS_REGION 
AWS_REGION=ap-southeast-2

[lox]

We don't actually read [default], as all aws-vault commands relate to profiles.

[FernandoMiguel]

FYI, all my profiles are sourced from something else, so some are sourced from default

[profile IAM]
mfa_serial = arn:aws:iamXXX
role_arn = arn:aws:iam::XXX
source_profile = default

[lox]

That would be [default] though, not [profile default] I think @FernandoMiguel

[FernandoMiguel]

just bad copy paste :p

[sharms]

I can confirm that setting region does not work except via environment:
Works:

AWS_REGION=us-gov-west-1 aws-vault exec profile-name bash

Fails:

cat ~/.aws/config
[default]
region = us-gov-west-1

Fails:

cat ~/.aws/config
[profile profile-name]
region = us-gov-west-1

Of interest - once the MFA process has completed, I can drop AWS_REGION off the command while the session is valid.

[lox]

That's very odd. Will try and reproduce.

[lox]

I can't reproduce this. This is what I did:

docker run --rm -it -v $PWD:/go/src/github.com/99designs/aws-vault -w /go/src/github.com/99designs/aws-vault golang:1.9

$ go run *.go add --backend file test 
Enter Access Key ID: test
Enter Secret Access Key: test
Enter passphrase to unlock /root/.awsvault/keys:
Added credentials to profile "test" in vault

$ export AWS_VAULT_FILE_PASSPHRASE=test
$ export AWS_VAULT_BACKEND=file

$ mkdir -p ~/.aws
$ echo "[profile test]" > ~/.aws/config
$ echo "region = us-gov-west-1" >> ~/.aws/config

$ cat ~/.aws/config
[profile test]
region = us-gov-west-1

$ go run *.go --debug --backend file exec --no-session test -- env | grep AWS
tes2017/09/28 00:24:54 Parsing config file /root/.aws/config
2017/09/28 00:24:54 Skipping session token and using master credentials directly
2017/09/28 00:24:54 Looking up keyring for test
2017/09/28 00:24:54 Parsing config file /root/.aws/config
2017/09/28 00:24:54 Setting subprocess env: AWS_DEFAULT_REGION=us-gov-west-1, AWS_REGION=us-gov-west-1
2017/09/28 00:24:54 Setting subprocess env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
2017/09/28 00:24:54 Executing subprocess env []
AWS_VAULT_FILE_PASSPHRASE=test
AWS_VAULT=test
AWS_DEFAULT_REGION=us-gov-west-1
AWS_REGION=us-gov-west-1
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test

$ rm ~/.aws/config
$ go run *.go --debug --backend file exec --no-session test -- env | grep AWS
2017/09/28 00:27:26 Skipping session token and using master credentials directly
2017/09/28 00:27:26 Looking up keyring for test
2017/09/28 00:27:26 Setting subprocess env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
2017/09/28 00:27:26 Executing subprocess env []
AWS_VAULT_FILE_PASSPHRASE=test
AWS_VAULT=test
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test

[lox]

Could I trouble @sharms and @jleclanche to try running those commands with --debug and pasting the results here so we can see what is going on? Sounds like it might not be finding your config?

[sharms]

Sure - here is what it looks like without AWS_REGION set:

aws-vault exec --debug --backend=file profile-name bash
2017/09/27 21:29:23 Parsing config file /Users/sharms/.aws/config
2017/09/27 21:29:23 Session not found in keyring
2017/09/27 21:29:23 Looking up keyring for profile-name
Enter passphrase to unlock /Users/sharms/.awsvault/keys:
Enter token for arn:aws-us-gov:iam::12345678:mfa/sharms: 222222
2017/09/27 21:29:39 Getting new session token for profile profile-name
aws-vault: error: Failed to get credentials for cloud-gov-govcloud: InvalidClientTokenId: The security token included in the request is invalid.
    status code: 403, request id: UUID

Repeated with AWS_REGION set:

AWS_REGION=us-gov-west-1 aws-vault exec --debug --backend=file profile-name bash
2017/09/27 21:31:37 Parsing config file /Users/sharms/.aws/config
2017/09/27 21:31:37 Session not found in keyring
2017/09/27 21:31:37 Looking up keyring for profile-name
Enter passphrase to unlock /Users/sharms/.awsvault/keys:
Enter token for arn:aws-us-gov:iam::12345678:mfa/sharms: 222222
2017/09/27 21:31:49 Getting new session token for profile profile-name
2017/09/27 21:31:50 Writing session for profile-name to keyring
2017/09/27 21:31:50 Using session ****************ZZZZ, expires in 3h59m59.814768977s
2017/09/27 21:31:50 Parsing config file /Users/sharms/.aws/config
2017/09/27 21:31:50 Setting subprocess env: AWS_DEFAULT_REGION=us-gov-west-1, AWS_REGION=us-gov-west-1
2017/09/27 21:31:50 Setting subprocess env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
2017/09/27 21:31:50 Setting subprocess env: AWS_SESSION_TOKEN, AWS_SECURITY_TOKEN
bash-3.2$

Configuration files:

bash-3.2$ cat ~/.aws/credentials
cat: /Users/sharms/.aws/credentials: No such file or directory
bash-3.2$ cat ~/.aws/config
[profile profile-name]
region = us-gov-west-1
mfa_serial = arn:aws-us-gov:iam::12345678:mfa/sharms

[profile other-profile]
region = us-east-1
bash-3.2$

[lox]

Thanks, could you try it with --no-session? That will skip creating the STS session and just use the credentials as is. You could create a test profile with dummy credentials if you felt uncomfortable with that.

[sharms]

No problem. With --no-session it works and gives no errors

[lox]

Ok that is a good lead. I will investigate. Thanks for your patience 🙇🏼

[lox]

Btw, what version are you on @sharms?

[sharms]

aws-vault --version
v4.0.0

[lox]

@sharms I have a theory what the problem is, are you able to test out the code in https://github.com/99designs/aws-vault/pull/163?

[lox]

This is now in master, so closing. Reopen if it's not fixed for you @sharms / @jleclanche

[jleclanche]

@lox can you clarify what landed? Are we able to attach regions to aws-vault sessions now?

[lox]

It will be out in the next release, which we will try and do today or tomorrow.

The change I made was to read the ~/.aws/config file to get the region to set on STS calls.

This is the relevant PR: https://github.com/99designs/aws-vault/pull/163

[lox]

It's a slightly speculative fix, I'd love to hear if it fixes your issue.

[jleclanche]

Well, it's certainly an improvement :) But it doesn't really change the original issue that I had, which is that I'd like to tell aws-vault "the session 'home' should have an AWS_DEFAULT_REGION of us-east-1, and the session 'dev' should have an AWS_DEFAULT_REGION of us-west-1".

[lox]

Hmmmm. Apologies if I misunderstood, I'd interpreted it as "the region in the profile in ~/.aws/config should be respected for STS sessions". In your example, do you have two entries in your ~/.aws/config for home and dev, with different regions? That's a standard config from our perspective.

[jleclanche]

Well as I mentioned, I do not have a ~/.aws/config file usually. aws-vault handles the use case of profile management quite nicely, except for attaching default regions to them.

[lox]

Ah right, apologies, yes, I have totally missed the point 😓

[lox]

Ah right, I got misdirected by @sharms question, which is fixed.

[lox]

I guess the difficulty with saving additional metadata in the keyring is that it's hard to know what the source of truth is. Debugging what was setting the region gets much harder, especially when you add in things like assuming roles with source_profile. I think it's probably not a complexity I want to introduce. aws-vault is designed to compliment the AWS cli configuration, not replace it. I'll leave this open and see what other folks think.

[sharms]

Just wanted to confirm @lox that the fixes did solve my issue, thanks!

[lox]

I'm going to close this one out. Happy to discuss more if you want @jleclanche.