Aws-cli: [V2] Save temporary credentials in .aws/credentials for v1 compatibility

[marcelboettcher]

I expected that aws2 configure sso would get secrets via aws2 sso get-role-credentials and save them in .aws/credentials
The aws cli v1 and all current tools out there could continue using those temp credentials.

[andyfase]

Cannot +1 this enough. Without providing a mechanism to add the temp credentials too ~/.aws/credentials it makes AWS SSO CLI support unusable for many as a mechanism for authentication and retrieving temp creds.

Many tools are used to provision infrastructure on-top of the various AWS SDK's, they all rely on creds in ~/.aws/credentials. Given we cannot control when or if they will be updated, adding a flag to the aws2 sso login command to allow for the creds to be stored in v1 format would open up the usefulness considerably more.

[sethbacon]

Agreed, this breaks lots...please allow a work around until other tools can implement fixes.

[loadfix]

This is required for the AWS Toolkits (PyCharm / VSC) to operate without manual intervention.

[jamesls]

Hi everyone, we do plan on addressing this.

It's worth mentioning that, long-term,the plan is that all SDKs and CLIs (including CLI v1) will support SSO credentials, so all of the official AWS SDKs/CLIs will "just work." We launched this in CLI v2 early to get customer feedback.

We did something similar years back when the CLI first added assume role support, with all the SDKs eventually adding support.

However, the shorter term option that someone suggested that I like is being able to export your current credentials as various formats. I forget the exact suggested command, but something like "aws configure export-creds" will do its normal credential loading, but then print them out in various formats. For example, it could print them out as export AWS... env vars that you could eval, and I would imagine we could also print out a format that could be the credentials file format that you could append to ~/.aws/credentials if you wanted.

The reason I like this approach of exporting your creds as env vars or config snippets, etc, is because it's decoupled from a specific credential provider so in theory will work for any new credential providers we add in the future. It will also work for cases today such as assume-role which also use a cache file instead of writing to ~/.aws/credentials. It could address things like https://github.com/aws/aws-cli/issues/3711 and https://github.com/aws/aws-cli/issues/4323.

Lastly, while I know it's convenient, I do want to mention that writing temporary credentials to ~/.aws/credentials should really be used as a last resort. The shared credentials file has no concept of expiry time (for now) and many SDKs never reread this file for the entire duration of their process. This means that at some point, when your credentials expire, your API calls will start failing on you. Having this built in to the SDKs credential providers, they can automatically refresh these credentials for you to ensure everything works smoothly.

It's be great to hear what others think of an "export credentials" command.

[pcolmer]

How about adding an exec command so that you go something like:

aws2 --profile my-sso-profile exec terraform

and aws2 would set up environment variables with all of the cred details and then run whatever comes after exec as a subprocess so that the environment variables are inherited.

[rnhurt]

I like @pcolmer's idea of an exec command but I worry that the sub-process might introduce some complexities with some tools. My idea (#4668) of exporting the AWS_* env vars seems closer to what @jamesls mentioned several days ago. I think I like exporting env vars better because it sets up a specific terminal for a particular role (Production_ReadOnly, Development_Admin, etc.) and allows you to use tools in that terminal without having to manage them on every single call.

Really, I'm OK with almost any way we can do this. I really, really want to use the new v2 command, but the fact that it doesn't play well with others is preventing me from fully embracing it right now.

[yawn]

If you just syscall exec whatever comes after your exec or -- (or whatever command signifier you choose) there shouldn't be any problems with tooling. We've been doing this with our custom STS workflows for years.

[sethbacon]

The main issue I have with this is that on Windows, the CLI2 profiles now don't work with AWS Tools for powershell. Hopefully this gets addressed very soon.

[benkehoe]

@sethbacon you could open an issue on the AWS Tools for PowerShell repo

[sethbacon]

https://github.com/aws/aws-tools-for-powershell/issues/128

[arohter]

You should be leveraging the credential_process spec to do this, as it's already widely used: https://github.com/aws/aws-cli/issues/4668#issuecomment-595403684

[tarasrng]

I was searching for a similar feature some time ago, with no luck. Here is my custom solution that does sso login and puts credentials into .aws/credentials.
https://github.com/tarasrng/aws-sso-credentials-fetcher

[willhughes-au]

I can't express how frustrating it is not to be able to get KeyID and Secret from the commandline when it's clearly already implemented and just not exposed anywhere.

The AWS SSO Portal will happily generate credentials on the website for you.

The client is generating them and storing them in the ~/.aws/sso/cache/ directory, but there doesn't appear to be a way to map those filenames to a given profile.

The solution given by @tarasrng above is just "grab the most recently modified JSON file there and hope that it's the right one". If you've got multiple consoles open and running commands, then there's no guarantees about who's going to be most recent.

I've tried looking through the codebase but can't find anything obvious so far.

Edit: Looks like awscli/customizations/sso/utils.py is responsible for that cache directory, and it uses Botocore's SSOTokenFetcher to manage it.

I got lost in the code there - it seems like it's encoding the start URL, which makes me think that perhaps that cache file is re-used for each profile, but I'm uncertain at the moment.

[felipesere]

Yeah, for our project it would be great to login via SSO, and then get credentials for different accounts/roles to use with tools like Terraform.

[pcolmer]

@felipesere the solution we've adopted for use with tools like Terraform is https://github.com/linaro-its/aws2-wrap

[felipesere]

@pcolmer That is brilliant! I'll take a closer look later today. Is it OK it port it to go to have a static binary I can commit into our toolchain repo? I am trying to minimize things people need to install to get up and running.

[pcolmer]

@felipesere

Is it OK it port it to go to have a static binary I can commit into our toolchain repo?

Absolutely!

[willhughes-au]

I gave up on trying to decode the AWS Source code and have written a script to brute-force finding the credentials from the local cache directory.

Call it like so:

$ ./find-aws-ssokey.sh mycorp-sso-test
Looking for Profile: mycorp-sso-test - "arn:aws:sts::12345:assumed-role/AWSReservedSSO_SsoPolicyProfile_123456789/my.sso.user"
Found credentials for mycorp-sso-test and wrote to ~/.aws_credentials.mycorp-sso-test

It takes a single parameter which is the AWS Profile name, for which it then determines the current Profile's ARN.
Looping through the cached JSON files, it uses each key in turn to call aws sts get-caller-identity on them in turn until it finds a matching ARN.
Note that if for some reason you have multiple profiles pointing to the same ARN you may run into issues, I'm not sure why you'd do that, however.

Note that it requires jq a python package for parsing json.

I'm specifically writing it so that I can stuff the values into docker containers, so I'm outputting to ~/aws_credentials.$ProfileName

See the Gist for the script: https://gist.github.com/willhughes-au/d74594e1c37259b003854dc02d8e144a

I'd appreciate any bugfixes/suggestions. It works for me, but as with any code off the internet - take care, vet that it does what you want, and don't blame me if it runs up your AWS bill or deletes production.

[jaknn]

@pcolmer aws2-wrap is a lifesaver until Boto3 and the other SDKs catch up with sso support and/or aws sso adds one of the export options discussed here. Kudos!

[benkehoe]

Throwing my hat into the ring. I wrote a tool, aws-sso-credential-process, that implements the credential process interface, so you can add it to your profiles in .aws/config and you don't have to modify how you invoke your scripts or where they look for credentials. It uses the SSO configuration in the profile you put it in. Once support is implemented in the SDKs, you'll simply remove the credential_process line from the profile. It also has the capability to pop up the browser to login if the session is expired, rather than requiring you to do aws sso login separately (something I'd love from the CLI itself, too).

For people who want the raw credentials directly, they can also use this tool on its own, because the output of a credential process is a JSON object (to stdout) containing the current credentials, including the ability to refresh the credentials and/or the token if needed.

[corrjo]

Thanks @benkehoe, this is exactly whats needed until sso credentials become a first class citizen in the sdks.