Boto3: refreshing sts role assumption credentials for long running operations

[kapilt]

boto3 looks like it wires in credential refreshing when its using instance roles by default. The issue is how how to setup the same behavior when doing sts role assumption, to refresh credentials (i'm initially starting with an instance role). Atm the only way to pass the sts assumed role credentials is in via explicitly creating a session/client with key/secret/token, but that results in the creation of a default Credentials class without the refresh behavior. There doesn't seem to be a clean way to provide a RefreshableCredential acquired with sts token assumption to a client.

[kapilt]

by clean way, i mean not poking at botocore session internals for example, this is what i'm currently doing to make this work https://gist.github.com/kapilt/ac8e222081f63ba64e93

[russellballestrini]

I'm not a core dev but this is how I'm dealing with AWS access/secret keys, regions, etc: https://github.com/russellballestrini/botoform/blob/master/botoform/util.py#L18-L87

Edit: This class makes it possible to switch between the AWS config profile/region programatically.

[mtdowling]

There is an AssumeRoleProvider for credentials in Botocore that might help here: https://github.com/boto/botocore/blob/develop/botocore/credentials.py#L589. I'll try to get more information on this and see if I can provide an example.

Please also feel free to chime in to the related discussion happening on Botocore about how we can improve assume role support: https://github.com/boto/botocore/issues/761

[BardiaAfshin]

+1

[BardiaAfshin]

@russellballestrini do you have a gist of consuming that code?
I want to use it for s3 client for downloading and uploading files, a process that goes on for hours if not days.

EDIT:
I don't see how the refresh is being auto refreshed.

[russellballestrini]

@BardiaAfshin the code that I linked was never tested with "assuming a role". That class just makes it possible to switch between the AWS config profile/region programatically.

Sorry about that.

[BardiaAfshin]

I see. I'm surprised to not see the auto refresh solved already.

[babartareen]

+1

[JoeyG1973]

Anybody have any clue how to get cert refresh with assume_role_with_saml?

[JoeyG1973]

Figured it out.

https://gist.github.com/JoeyG1973/69ae503f67ff7f07b498b2e53226e206

[cobusdebeer]

@kapilt Thank you! Your method works for me as well. In my case, the client application gets cognito credentials (expiring in 1 hr) and does an s3 upload that may run for hours. I used assume_role_with_web_identity() in the refresh, with WebIdentityToken = the cognito token

[Enrico2]

It's been more than a year since the last activity on this thread. Have people found a better workaround than @kapilt 's? Thanks.

[slothkong]

@BardiaAfshin, I have the same issue with S3. Any chance you found a solution?

[jappievw]

Found a solution which only uses the public api of Boto3 and Botocore. Gist can be found here. Feedback = welcome.

[sls-cat]

https://github.com/monkeysecurity/botor is really close to doing what you need. There is at least one additional incarnation of this https://github.com/Netflix-Skunkworks/cloudaux, but I found that some of them just refresh the STS token and don't cache the connections. It's not hard to test and fix. It is a python decorator that you wrap all your AWS SDK calls with. It creates tokens and refreshes them when they are almost expired. It also caches connections. I made minor modifications so that it works with or without assumed role. It's not seamless, but it it requires no changes to core AWS SDK and it is working well for our needs.

[DogFortune]

I have the same problem. In my case I would like to transfer files to S3 using AssumeRole. Since transfer stops when it expires, you need to update your credentials. Is there a better way to update my credentials without stopping the transfer?

[chrisdlangton]

I strongly suggest this issue be CLOSED - it is a non issue. i.e. chose the tool for the job.
AWS solve this use case, all you need to do is use an _instance profile_ and assign the role to it, Amazon will keep the credentials alive for you.

If you are not using a host managed by Amazon, and the 12 hour limit on STS Assume Role is too short for your use case, then STS Assume Role is not the tool for the job, you will have to do these problematic hacks to make the square peg fit in your round hole, not smart.

The appropriate solution might be using IAM keyId and secret, or federated through Cognito, or something else. Bottom line, if you need more than a 12 hour session for a long running use case STS Assume Role is an anti-pattern and should be avoided.

Note to maintainers; do not change botocore or boto3 to appease such feature requests that any AWS solution archetect, and all AWS documentation and materials discourage. Point these _issues_ to the appropriate AWS documentation to help the educational issues, and suggest the Labs and AWS certified developer associate / professional exam materials.

[kapilt]

There are many legitimate use cases for this, please don't make assumptions and try to impose them on others without understanding their use cases. sts role assumption is needed for crossing account boundaries even from an instance with profile, and long running operations will need refresh. static iam creds are much more an anti pattern from a security perspective. I think there is a bit more support in botocore for refresh, but some assembly still required, the cleanest I've seen is this one https://github.com/boto/botocore/issues/761#issuecomment-426037853

[chrisdlangton]

incorrect, iam key and secret is not poor security practice at all. As an Information Security professional the security risks are poor key management. sts assumerole also requires iam key and secret that has permissions for sts and iam actions when assuming outside an aws managed compute environment therefore the same inherent risks apply in both cases. there is actually more risk when developers poorly implement hacks to force sts to work outside its 12 hour limit.

Also all code samples provided, and the feature request itself, do not address the cross account boundary use case, it is clearly a request to keep alive an assumed role for longer running processes than 12hr limitation. A good developer may find a solution and contribute upstream but a great developer solves the root problem for their use case.

Security risks aside, as this is a pure right-tool-for-the-job statement, and preserving the library from being compromised. As developers must seek clarity from those with knowledge before writing poor code that compromises the project. I impose nothing more than as boto maintainers to consultant AWS about this feature request, and I simply explain the facts to consider.

[ericfrederich]

I'm looking for a solution to this as well. I have a Lambda which assumes roles in other accounts.

boto3.Session() can take a profile_name but this requires ~/.aws/config/~/.aws/credentials files to work they these files don't exist in Lambda.

I think it would be enough if boto3 provided a way to configure profiles programmatically.
Maybe that's already there? I haven't found it.

[ericfrederich]

Just found this.

Check out get_role_session from boto3_extensions ... (permalink at time of commenting)

This is exactly what I was mentioning. They're monkey-patching the configuration in memory instead of using a file. This is also a hack. There really should be a way to programmatically specify a profile.

[PSiAU]

We use role chaining which AWS sets an unchangeable hard session limit of 1 hour on, which makes credential refreshing necessary.

[chrisdlangton]

the only real question is;
should a framework be dogmatic?
should a framework transparently manage auth keep-alive for the user?
should a framework circumvent the vendors own restrictions that are based on security best-practices?

The answer to all of those I hope keeps these rubbish insecure ideas out of boto3 and let the users who want to be insecure do their own hacks that should not be in a framework like boto3.

[ericfrederich]

@chrisdlangton when using an IAM role assigned to an EC2 instance this framework does exactly what you mention. It manages keep-alive for the user. It is completely transparent. Refreshing tokens using that 169.254.169.254 metadata service.

Here we're asking that the functionality be extended to support using an assumed role as well, not just the one assigned to the instance.

I would argue that

• Long running jobs on an EC2 are a valid use case and not against security best-practices.
• Long running jobs on an EC2 which assume a role are a valid use case and not against security best-practices.

If you have any links / documentation saying otherwise, I'd be happy to read them.

[chrisdlangton]

@ericfrederich why be so petty as to make it sound like it was my opinion in context to what you describe? I responded to @PSiAU commenting on the 1hr session limit, don't go generalizing my response to the 1hr limitation to make your own arguments sound better. Come up with your own value proposition to make the case to add bespoke hacks to boto3 that fit into your model of how frameworks should work and don't go generalizing other peoples responses to specific comments you didn't even make.

p.s. if you're looking for a link about the 1hr session limit it's clear you're not educated or capable of finding the knowledge (ignoring you asked me and I didn't even make the point)

[jimcarreer]

Also +1, my team is facing a similar issue and ended up using the gist @kapilt provided. Assumed roles are generally the solution AWS Support recommends when performing cross account operations, and there are scenarios you need these sessions refreshed. The fact that RefreshableCredentials exist at all speaks to the admission of the validity of that use case. Also I'd argue periodic session refreshing is more secure than hard coded AKey/SKey values. If the later is leaked once you have a major security issue until the keys are changed manually, depending on your setup that may require redeployment of software. If session details created from assumed role are leaked, you have a major security issue until they expire or you revoke them. The argument that refreshing them is some how insecure, especially considering that this is very similar to how the credentials for instance profiles are rotated, doesn't make a lot of sense to me.

[kapilt]

Well reasoned and said but in general, Don’t feed the trolls

On Wed, May 22, 2019 at 3:51 PM jimcarreer [e-mail] wrote:

Also +1, my team is facing a similar issue and ended up using the gist
@kapilt https://github.com/kapilt provided. Assumed roles are generally
the solution AWS Support recommends when performing cross account
operations, and there are scenarios you need these sessions refreshed. The
fact that RefreshableCredentials exist as at all speaks to the admission of
the validity of that use case. Also I'd argue periodic session refreshing
is more secure than hard coded AKey/SKey values. If the later is leaked
once you have a major security issue for until the keys are changed
manually, depending on your setup that may require redeployment of
software. If a session details created from assumed role are leaked, you
have a major security issue until they expire or you revoke them. The
argument that refreshing them is some how insecure, especially considering
that this is very similar to how the credentials for instance profiles are
rotated, doesn't make a lot of sense to me.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/boto/boto3/issues/443?email_source=notifications&email_token=AAAFJET7DMH74ZJYNOQ472DPWWW3FA5CNFSM4BYZZWHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWAJXAQ#issuecomment-494967682,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAFJES3LZOQU7YVFKSPCD3PWWW3FANCNFSM4BYZZWHA
.

[chrisdlangton]

@kapilt so a comment that only name calls and confirmation bias callouts is called what?

On-topic, the most recent example provide here by @PSiAU has an AWS restriction of 1hr.
If you do not understand the security for that choice, please do not assume that you are correct to circumvent the security controls by promoting boto3 to add your preferred solution to circumvent a security control.

I'm not arguing for imposing any restrictions on anyone, I am arguing to avoid boto3 enabling users who want to circumvent security controls.

case-in-point I have a separate feature request and possible PR to add Perfect Forward Secrecy to boto3.

Respect AWS security controls should not be circumvented. Period.

[kapilt]

@chrisdlangton, recapping

• sts session time out is configurable from 1-12 hours, it is not 1 hour, that is merely a default.
  https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html
• static credentials for iam users are a security anti-pattern, many organizations disable iam users api keys entirely.
• control of whether or not session renewal is possible is controlled via iam permissions, ie the ability of the iam role to invoke AssumeRole on itself, not by the sdks
• attempting to prevent sdks from offering ergonomic interfaces in the name of security, when the underlying capability is available and in common use, is arguing for security through obscurity while misunderstanding the real security controls here are iam based not sdk.

there are many legitimate application use cases for this, when performing any long running operation that needs to cross role or account boundaries. the job of sdks is to enable api usage and application development, if there are common tasks for applications around api usage, rather than having everyone reimplement the same, the better outcome is to have the sdk support that functionality directly.

[olemarkus]

100% agree with @kapilt here.

[ericfrederich]

@chrisdlangton, mentioning the person you're responding to is not being petty. I was merely responding to your rhetorical questions. Without mentioning you it would have less context.

Thanks @kapilt and @jimcarreer ... you put my thoughts into words better than I can.

For now I continue to use get_role_session from boto3_extensions for two different use cases.

• it allows me to assume a role in a Lambda using a profile where ~/.aws/credentials and ~/.aws/config don't exist
• it allows me to have long running scripts use the assumed role

[chrisdlangton]

your use cases (you think they are valid and that's fine) can be implemented using boto3 right now, you don't need boto3 to provide you functionality that circumvents AWS security controls be it 1hr or 12hrs today or 9999999hrs tomorrow because these are security controls whether you like them or not.

If boto3 were to have a built in way to circumvent these, which is an insecure function by nature, boto3 would be encouraging that practice for all users whether or not they have valid uses (you call them valid, they call them valid, who cares if they are valid).

If anyone has valid reason to argue for this feature, speak to your AWS TAM or SA to get AWS to make a change to the security controls. Stop pressuring boto3 to circumvent them for you, you already have the ability to implement such circumvention on your own without compromising anyone other than yourself.

[jimcarreer]

your use cases (you think they are valid and that's fine) can be implemented using boto3 right now, you don't need boto3 to provide you functionality that circumvents AWS security controls be it 1hr or 12hrs today or 9999999hrs tomorrow because these are security controls whether you like them or not.

No one's advocating bypassing these. If you look at @kapilt gist: the maximum amount of time the session lives is still whatever the maximum session time assigned to the role, ie a hard max of 12 hours. We're advocating for a non-hacked way to replace that session using the RefreshableCredentials class that's already provided by botocore. It would probably be better to call this "session rotation" not "session refreshing" because that's what it is: when the session is expired, the gist that @kapilt creates a new one, the old one is still invalid and still expires as it should, the new one seamlessly replaces it within the boto internals. The new assumed role session is created via the instance profile session which you already suggested people should be using. This type of rotation in the boto internals is exactly how regular sessions based on the instance profile are rotated and why when performing long running aws cli operations on an IAM you don't receive a 403 mid completion for the originating session expiring. Literally everyone in this thread wants to extend that behavior to allow an assumed role session to do the same thing.

[chrisdlangton]

@jimcarreer per the OP

boto3 looks like it wires in credential refreshing when its using instance roles by default. The issue is how how to setup the same behavior when doing sts role assumption, to refresh credentials (i'm initially starting with an instance role). Atm the only way to pass the sts assumed role credentials is in via explicitly creating a session/client with key/secret/token, but that results in the creation of a default Credentials class without the refresh behavior. There doesn't seem to be a clean way to provide a RefreshableCredential acquired with sts token assumption to a client.

refreshing enough said?

I was going to point out the whole distinction without a difference, but given you missed the OP and have clearly your own ideas of the feature request - perhaps rather you might have a better day if you start your own feature request for rotate and try to make it sound useful.
Good luck

[jimcarreer]

@chrisdlangton

I was going to point out the whole distinction without a difference

I firmly believed your disagreement came from an initial misunderstanding of the more technical mechanics of the request, I wasn't attempting to describe a difference because, as you point out, there isn't one. I was simply trying to rephrase the request in a way you might better understand. Your statements here show that you obviously don't disagree with the automatic credential refreshing that occurs when using an instance profile in boto, so it is hard to imagine why you'd disagree with this behavior being extended to a session using an assumed role, unless it was unclear to you how this could be done in a secure manner.

I still believe you don't quite understand the request or the details of the suggested implementation, but you also don't seem to be willing to discuss the issue without being patronizing to the people involved. I don't feel engaging with you further would lead to any fruitful outcomes. Apologies that I could not better describe the request here.

@kapilt thanks again for the gist, our security team was extremely happy we came to a solution that did not require a permanent key-pair (these are only granted in very special cases), which was really the only other option for us.

[chrisdlangton]

@jimcarreer I'm glad you found the simple hack provided by @kapilt useful - and I am concerned that you and others feel boto3 should be providing circumvention for users to avoid AWS security controls.

Until now I have stayed on topic, sts assume role refreshing, but it is clear to me now that you feel strongly that precedence of boto3 providing code to work within the instance profile scenario should also give magical inherent approval to sts assume role refreshing also.
This is simply ludicrous, there is no such precedence. For precedence to exist the 2 scenarios must be serving the same purpose and goals. if the goals of sts and instance profiles were identical the security controls would reflect this - the fact AWS do not apply the same security controls to each clearly tells you there is no precedence to reference for this feature request.

The best advice I can give anyone who has a problem with AWS security controls is, speak to AWS about changing the security controls - alternatively you have the option to work-around the controls (using the hacks provided in this thread).

Boto3 is a freamework, it has design goals to provide an SDK to support the AWS API. There is no boto3 design goals that support circumvention of AWS security controls.

[ronaldozark]

@kapilt I opened a PR to allow botocore credentials to be set directly for a session, I had the exact same issue as you did and basically arrived at the same solution. On my travels I found a helper function in botocore:

https://github.com/boto/botocore/blob/develop/botocore/credentials.py#L148

def create_assume_role_refresher(client, params):
    def refresh():
        response = client.assume_role(**params)
        credentials = response['Credentials']
        # We need to normalize the credential names to
        # the values expected by the refresh creds.
        return {
            'access_key': credentials['AccessKeyId'],
            'secret_key': credentials['SecretAccessKey'],
            'token': credentials['SessionToken'],
            'expiry_time': _serialize_if_needed(credentials['Expiration']),
        }
    return refresh

Which looks like it encapsulates at least part of your gist (including the silly serialization of the date).

[chrisdlangton]

@ronaldozark +1 for create_assume_role_refresher not being in boto3
Let us find out how botocore feel about the code lacking support for MFA, at least the error can be handled by user space code.

[ronaldozark]

Let us find out how botocore feel about the code lacking support for MFA

Not sure how MFA is relevant here? Botocore basically supports this assumed role session refreshing you just need to be able to set the credentials object in the session.

[chrisdlangton]

Without intending to derail the on topic conversation, MFA is an important security feature that is commonly used for an IAM Role. There are many times a human is responsible for providing the 6 digit code but in applications that use MFA-enabled roles the 6 digit code is derived from a secret key provided to the application using any form of secrets management tool such as Parameter Store or Hashicorp Vault which I'm familiar with.

@ronaldozark I applaud you making upstream contributions, you have good initiative and I hope you receive support and direction from botocore, however this is boto3 and the feature request itself you have already made clear has nothing to do with your botocore utility function so lets stay on topic here please.

[ronaldozark]

Without intending to derail the on topic conversation, MFA is an important security feature that is commonly used for an IAM Role.

Right, everyone knows what it is, it just odd you mentioned it here as:

1) It never came up in any previous discussion on the issue.
2) It has very little with whats being discussed here.

It seems odd that you would mention it? It's like if everyone was discussing a car's alternator and you brought up the head lights? Just doesn't make a lot of sense.

[chrisdlangton]

I've been consistent;

• I care about AWS security controls
• MFA is a security control
• you provided code in this thread that calls client.assume_role()
• client.assume_role() prompts for MFA
• Your code does not handle MFA prompt, it errors to user space as I pointed out earlier, and the challenge can never be fulfilled in your code.
• again, MFA is a security control, i care about AWS security controls
• as i said, I hope you receive support and direction from botocore

Let's see how boto3 maintainers feel about this feature request, we have certainly exhausted the discussion on the purpose of the feature and i'm comfortable with the message of maintaining AWS security controls. For both sides of the discussion I hope boto3 maintainers can provide guidance to those who want the capability while respecting AWS security controls.

[ronaldozark]

client.assume_role() prompts for MFA

Only if configured to.

[ronaldozark]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts.html#STS.Client.assume_role

(Optional) You can include multi-factor authentication (MFA) information when you call AssumeRole . This is useful for cross-account scenarios to ensure that the user that assumes the role has been authenticated with an AWS MFA device. In that scenario, the trust policy of the role being assumed includes a condition that tests for MFA authentication. If the caller does not include valid MFA information, the request to assume the role is denied. The condition in a trust policy that tests for MFA authentication might look like the following example.

This has to be true:

In that scenario, the trust policy of the role being assumed includes a condition that tests for MFA authentication.

Otherwise there is no MFA handling required. This is not true for my scenario, nor is MFA a mandatory requirement for assuming roles, even across two AWS accounts. Its pretty easy to verify this yourself.

[chrisdlangton]

I agree that MFA is optional, IAM Roles are optional too right? You're clearly missing the point..

Only if configured to

botocore and boto3 are frameworks, code contributions must handle all known configurations, anything less is a defect at best, let's leave the botocore discussion in botocore please.

[ronaldozark]

botocore and boto3 code contributions must handle all possible configuration

Ah but you see it does! If MFA were enabled in the scenario described, it would have to be handled in the custom implementation of the refresh(...) function provided by the user. So it is supported, but if a user is already trying to leverage the RefreshableCredentials object they'd have to handle the MFA prompt in the refresh() call which is implemented by the API user in this case. Otherwise, you'd be denied access.

You're clearly missing the point.

Ah, I think its more we might be have different use cases in mind and are talking past each other! But hey thats how we figure things out.

[ronaldozark]

Taking the OP's gist as an example:

def refresh():
        credentials = session.client('sts').assume_role(
            RoleArn=role_arn,
            RoleSessionName=session_name)['Credentials']
        return dict(
            access_key=credentials['AccessKeyId'],
            secret_key=credentials['SecretAccessKey'],
            token=credentials['SessionToken'],
            # Silly that we basically stringify so it can be parsed again                                                                                                                                                                                                                                                                              
            expiry_time=credentials['Expiration'].isoformat())

Basically if you had MFA enabled for the role specified by role_arn, then it would be up to the implementer to provide that MFA code to the assume_role(...) call. Its handled at the level where the API user is implementing their code, its just the OP's example is for roles with out MFA enabled.

But that is a good point! Thanks for calling it out.

[chrisdlangton]

it would have to be handled in the custom implementation of the refresh(...) function provided by the user

Your PR defines said function, which in turn prevents the user from defining as you suggest

I feel you have grasped the catch22 now, your proposed botocore internal is now performing work intended for botocore user land - and this version is lacking a security control the AWS Well Architected Framework, CIS benchmarks, PCI DSS 3.2, FedRAMP, HIPPA, (list goes on) all require.

Bottom line, both this boto3 thread and botocore PR are lacking some fundamental security considerations, I am happy to connect with you (or really anyone) and provide more details or even guide you on your efforts if you want to apply a degree of security controls to your projects - i do this for the love of it and I just happen to be lucky enough to do what i love for work too.

Let's wrap this up and focus the thread on boto3 issue 443 solely

[ronaldozark]

Your PR defines said function, which in turn prevents the user from defining as you suggest

It does not actually. Please point to the code in my PR that implements a refresh function.

[ronaldozark]

Oh maybe you're confused. If you're talking about this:

def create_assume_role_refresher(client, params):
    def refresh():
        response = client.assume_role(**params)
        credentials = response['Credentials']
        # We need to normalize the credential names to
        # the values expected by the refresh creds.
        return {
            'access_key': credentials['AccessKeyId'],
            'secret_key': credentials['SecretAccessKey'],
            'token': credentials['SessionToken'],
            'expiry_time': _serialize_if_needed(credentials['Expiration']),
        }
    return refresh

^ This is not my code or in my PR. This already exists in botocore today, actually its been in botocore since about 2015: https://github.com/boto/botocore/commit/2dae76f52ae63db3304b5933730ea5efaaaf2bfc

I can see how that might have confused you. My PR simply allows someone to set the Credentials object on a session directly, it leaves the implementation refresh(...) to the user (where it should be).

[lawliet89]

My use case is this:

• I use Vault's AWS Secrets Engine to perform the AssumeRole for me and return the STS Token.
• I would like to allow long-lived jobs to use this STS token and have them contact Vault for new tokens when the tokens expire.

The PR proposed would allow me to implement this bespoke refreshing behaviour myself.

[mrcranky]

Whether this issue should live in boto3 or botocore I can't say as I don't know the internal library structure. What I can say is that there is a clear and valid use case as a boto3 user for being able to configure a session to use credentials that are automatically refreshed as needed by going back to STS for fresh access/secret/tokens. Talking about EC2 instance profiles / roles is missing the point and assuming that a) you're always running in the context of an EC2 instance, and b) you're only ever going to need to assume a single role. For cross-account role assumption (which we use extensively) use of STS is unavoidable, and even within an account there are valid security reasons why you would want to assume a higher privileged role for an extended period without making that role the default for an EC2 instance.

It's worth noting that AWS's own Node.js SDK handles this cleanly by letting you specify a credentials object that abstracts away from the source of the credentials, either they're static or are auto-refreshed as needed. TemporaryCredentials make it clean and easy to use assumed roles securely without having to cache those credentials somewhere in the user code. I expected this functionality to also be available in boto3 and was surprised to find I was going to have to write my own implementation and hook it in somehow. The Java and Ruby SDKs also appear to have the same abstraction, though I haven't used them in anger. The boto3 implementation seems to make many more assumptions about the way credentials are to be provided to the session, and if your use case doesn't fit those common patterns, you're out of luck.

Maybe I missed the suggested implementation above that does this 'cleanly', but all of the samples I looked at seemed to be accessing library internal objects and not using any sort of clean hooks provided for refreshing credentials just-in-time before they're needed. If there are design reasons the library doesn't want to use an abstraction around refreshable credentials, could there at least be a reference implementation provided for the right way to do this somewhere in the docs?

[chrisdlangton]

Note: 
AWS.TemporaryCredentials is deprecated, but remains available for backwards compatibility. AWS.ChainableTemporaryCredentials is the preferred class for temporary credentials.

Now ChainableTemporaryCredentials is actually quite elegant. It preserves all the characteristics of the temporary nature of the role, retrieval of the current session token by default, and an express intent to refresh is needed to give you the appearance of an extending feature but it enforces all initialisation as though it was originally called from the first master credentials.

I support ChainableTemporaryCredentials for boto3, but fear a major refactoring of internals would be required to meet the same level of robust security characteristics

[ericfrederich]

@mrcranky the get_role_session from boto3_extensions is definitely accessing internal objects. It's basically monkey-patching the internal object which represents what is normally read from ~/.aws/credentials and ~/.aws/config.

The functionality we ask for is entirely possible and supported currently by boto3 if the assumed role is configured in ~/.aws/config. We're simply asking that this functionality be provided without the presence of the file.

Hey boto3, can I use my current implicit credentials to assume a role without having to write code which explicitly handles keys, secret keys, tokens, etc... also, can you keep them refreshed just like you do for my implicit credentials? Thanks!

[chrisdlangton]

For the arguments to finally be muted, if (royal) you ever disagreed with me, as a random individual (in your mind), disregarding the AWS intent behind STS and my own 2 decades of professional experience - then i challenge you to try and disagree with the entire industry! Specifically security researchers working on this specific problem, public real world examples, the indisputable facts.

CyberArk and NetSPI are much harder for you to disagree with then me, though all points i made int his thread are being backed up thoroughly by this report, and all reasonable security professionals agree.
The fact is, STS is temporary - and for persistent credentials you should look toward a refactor to use a persistent methodology for your credentials.

The right tool for the job, what any junior developer learns.

Some obvious quotes that relate to this thread;

using scripts that create temporary tokens in a loop -- using the existing (and soon-to-be-expired) temporary token to generate a new one to use going forward

these STS temporary tokens are meant to last between a few minutes to several hours and are usually employed to create temporary AWS infrastructure

these intrusions aren't the fault of a vulnerability in the AWS infrastructure

regular STS activity which many AWS account owners aren't accustomed to investigate for suspicious activity

So it is unquestionable.

AWS market STS as a temporary credential service

Security researchers view STS as a temporary credential service

AWS account owners view STS as a temporary credential service

Why would (royal) you, a random developer, be correct and everyone else (including the vendor) be wrong?

[reversefold]

While the article you link to does talk about abuse of STS token refreshing it doesn't seem to advocate removal of temporary tokens or token refreshing, only monitoring of them to potentially detect abuse by bad actors and limiting permissions to least-privilege, which is best practice. Reading further into the two reports linked shows the same. The potential for abuse is explained and ultimately proper monitoring and use of the least-privilege method is what is suggested, not entirely disabling the feature.

What is unquestionable is that AWS does allow refreshing temporary keys and that this allows for usage that elides the need for static access keys, thus reducing the attack surface exposed by the key leakage and allowing for appropriate monitoring to detect the leakage sooner through the repeated refresh API calls.

In the end, refreshing the credentials is built into the service. If you feel that the service should not allow this then you should petition AWS to change their service. Trying to force consumers of the service to avoid using that feature is, at best, a losing proposition.

[chrisdlangton]

@reversefold while i agree with your observations of the research, you may not be refreshing your memory by reading this thread.
The only reason the debate exists is to get boto3 to perform the STS token refresh for you automatically - rather than having a user make the security decision and implement a few lines-of-code to do this themselves (so many examples were given in this thread, it is rather simple afterall).

for complete clarity, the only counter point in this thread is to keep the decision to implement circumvention of STS temporary token remain in the user-land code, not be a boto3 feature.

You can feel free to implement any relaxed security decisions into your own code, even the incredibly simple code snippet for this feature request, just do not expect all users of boto3 to think like you, you are encouraging a less secure posture for STS then was the AWS intent for the service

[ericfrederich]

Let's remove the ability to use -rf on the rm command. It's a security circumvention. Let end users implement it in their own code if they want!

[chrisdlangton]

@ericfrederich while you joke and still miss the point because only you suggested "removing" something. boto3 already has the -rf in your example - use it. dont ask them to put -rf builtin to rm (what this thread is asking for)

Continue with your pointless joking elsewhere, go troll someone who isn't just pointing out truths about using simple tools correctly as they were designed..

@jimcarreer you may be interested with the security research shared a few posts up.

In the scenario of long lived processes, a key/secret pair exists as the design decision from AWS, and it is actually a better security posture because it provides you a security characteristic called revocation, typical for any incident response, not possible with a compromised STS temporary token. With a compromised long lived process (the security research linked above) using a key/secret pair with least privileges, provides an ability to immediately revoke access, which is an essential security property.

With a compromised long lived process using an STS token a compromise will persist for as long as the token duration. Trust is established when a temporary token is issued. The AWS Account admin has no means for revoking or terminating the STS access immediately.

In the scenario where you have an EC2 instance profile and it rotates role based access, removing the role takes immediate effect to the access, it is not the same experience as STS AssumeRole.

If you go back ant read _my very first comment_ on this thread;

chose the tool for the job.
AWS solve this use case, all you need to do is use an instance profile and assign the role to it, Amazon will keep the credentials alive for you.
If you are not using a host managed by Amazon, and the 12 hour limit on STS Assume Role is too short for your use case, then STS Assume Role is not the tool for the job.
The appropriate solution might be using IAM keyId and secret, or federated through Cognito

Now we have the AWS design of STS and IAM and the security research community backing up my statement.

This feature request has no valid use case. Chose the tool for the job.

If you can't use the right access mechanism, then use the several simple code snippets if you must for meet deadlines for quick hacky wins. All coders should take a step back to consider using the tool for the job at least, it is a fundamental of programming.

And try to remember that not all business requirements require a coded solution, because business processes such as incident response is an incredibly important business requirement that AWS has design STS and IAM with clear intentions to specially address how businesses respond to incidents and breaches.

[chrisdlangton]

@mrcranky wrote;

You keep going back to "instance profiles fix this" and that there is no valid use case, despite the fact that multiple people have pointed out perfectly valid use cases, and that instance profiles do not, in any way, fix this. The overwhelming weight of opinion seems to be against you, and the only justifications you have offered have been repeatedly pointed out to be flawed. Just simply restating the same flawed reasoning over and over is not particularly productive.

Clearly @mrcranky is being cranky and missing most of the conversation entirely.
EDIT: oh look, it's now deleted..

No @mrcranky EC2 instance profiles solves certain problems, Cognito solves others, federation solves others, key/value pairs (heavily discussed) is great for long running process outside AWS, i even promoted constantly that you can keep using STS AssumeRole and refreshing the credential yourself, just don't ask for boto3 to make a change to the framework to do that for you..

[mrcranky]

I retracted it because I didn't want to get into a heated debate this early in the morning; my reply was terse because I didn't have enough time to write a more detailed answer, and even if I did, I would be restating all of the excellent points made by others that you keep avoiding. I am perfectly aware of the whole of the conversation, and the more I read it, the more I find myself agreeing with most everyone else's viewpoint, except yours. You've denounced everyone else's use cases as "invalid" without justification, and it seems to me from your replies, without bothering to actually understand those use cases.

You are appealing to authority by citing your "two decades of industry experience" and papers that, as has been pointed out, don't say what you assert them to say, but the authority you are neatly skipping over is the previously mentioned point that all of the other AWS SDKs, written by AWS, support this functionality, and do not pretend that it becomes some sort of security vulnerability as soon as you make use of it. You're making out like you speak for AWS here, and I really don't think you do; certainly nothing you've said here is convincing me that AWS think it's a mistake to have the SDK framework handle this.

We are asking for boto3 to support the same functionality as all the other AWS SDKs, because in its absence, users are encouraged to reduce security by either a) lengthening the time the STS tokens live to cover their use cases (meaning a leaked token can be abused for longer, b) storing even longer lived non-STS credentials with the same rights. Asking users to to avoid these weaknesses by rolling their own implementation of the exact same logic as all the other AWS SDKs - i.e. automatic renewal of the STS credentials using the local identity credentials - is a recipe for security flaws from bad implementations. As has been pointed out, it's not just "a few lines of code," to do this seamlessly, it is an ugly bolt-on that uses library internals to achieve its effect. If you have a solution that isn't, by all means please share it.

Simply asserting that "Temporary means Temporary" and that somehow means renewing credentials is a bad thing seems to me nonsense. One of the big security wins of STS is that you can make exposed things as short-lived as possible; keeping the long-lived identity credentials that let you get more STS credentials tightly under wraps while you use the STS credentials elsewhere in the software, forcing the system to come back to that secure issuer of tokens frequently, maximising the number of opportunities for quick revocation of compromised credentials, minimising the window of exposure. Allowing seamless renewal of those credentials does nothing to alter that design; the STS credentials that might leak don't suddenly become long-lived just because you can easily get more of them, nor do the identity credentials that need to be kept secure don't get exposed more. You seem to think that making the user's code be the thing that renews credentials somehow adds security, but I can't find anything you've shared here that backs up that claim. Perhaps you think this is a blatantly obvious fact that everyone should know and agree with, but I think that if it were such a slam dunk, it would also be easy for you to explain why it's true.

If you want to keep restating the same unconvincing arguments over and over, declaring this request as meritless and should be closed; you can expect the 30+ other people (many of whom I'd bet also have "two decades of industry experience") who have a need for this feature and who disagree with your assessment of the merits of this to keep replying to say "No."

[jimcarreer]

@chrisdlangton I solved this problem almost a year ago, mostly by ignoring you and consulting the experts at my company's cloud security department, we're using sts roles in the way kapit describes. There's actually a more direct way in boto to do this and it is supported by AWS. I don't particularly need mentions a year later on a dead issue by someone clearly so unprofessional that he's still flaming about this 12 months on, some of us are actually doing work instead of pretending to know what we're talking about to strangers on github issues. I've blocked you.

[chrisdlangton]

@jimcarreer you actually did exactly as i suggested

using sts roles in the way kapit describes

My only point i have been making..

You should do the refresh using the existing boto3 features and boto3 should not do it for you and force that on everyone.

@mrcranky
never have i said that refreshing STS tokens is insecure, that word you use "insecure" is misleading the conversation and changing the meaning to suit your agenda.

I have stated for long running processes that refreshing the STS temporary token, as a built in feature of boto3, is effectively circumventing the intent of the service.
Yes i have said, adding that into boto3 would be a less secure posture for everyone, to save you and a few others here a few lines of code to do the exact thing they are asking boto3 to do for them.

More than this the purpose built functionality of AWS for long lived processes gives you the ability to immediately revoke access, whereas STS doesn't. For this reason alone you should consider, for your own benefit not becuase i said it is right, consider eventually changing from STS refreshing to a long lived key/secret pair so you can actually revoke access in the event of a breach. Go ask your "security team" about this specific question (in context to long running processes) BEFORE you are breached, they will certainly agree revocation is a critical feature.

In fact, AWS have a SOC2 Type II attestation of compliance and in the areas that talk about incident response (CC2.5, CC6.1, CC6.2) there is language used (under NDA watermarked report) directing AWS users to secure themselves to a certain level and this security posture of the customer will be assessed by AWS when they follow internal protocols to deal with security incidents, and the response will be proportionate to the customers intent. If you did not enable yourself the ability to revoke access it may not be something AWS can help you with.
You can also refer to AWS whitepapers yourself to see what they say about use cases for key/secret pairs, STS tokens, session revocations, etc.
Ceck the Well-Architected Framework too, it has only 6 principals in the security pillar, guess what number 1 is?

Implement a strong identity foundation: Implement the principle
of least privilege and enforce separation of duties with appropriate
authorization for each interaction with your AWS resources. Centralize
privilege management and reduce or even eliminate reliance on long-
term credentials.

Not performed a WAR yet? Feel free to contact my employer, we can do a WAR on your AWS accounts if you like, we've achieved the highest level of partnership and passed the assessments of the partner competencies as they are being released.

You say lots of people here disagree with my points?
I keep informing readers of this thread, these are not "my" points, they are AWS design features, and now you have seen them in the links shared from the wider security research community, and as you say, I refer to best practices frequently but don't offer anything from them, go back and check the links and quotes again, there are many but not always present i admit. None-the-less, you can't escape the references, link, and quote above from none other than the WAF itself..
Stop misleading the thread and trying to discredit me personally, I have continuously pointed to the facts of AWS and the security industry, so stop trying to discredit me, you just look uninformed.

[reversefold]

In the official AWS documentation for temporary IAM credentials they say this right near the top of the page:

When (or even before) the temporary security credentials expire, the user can request new credentials, as long as the user requesting them still has permissions to do so.

There you have it. AWS officially supports refreshing temporary tokens as a use-case. This same page also mentions and links to instance role temporary tokens as one of the ways in which IAM's temporary tokens are used. Instances are allowed to assume the role assigned to them and use the temporary tokens provided by STS and IAM to perform actions. Assuming a role explicitly is just another use-case for the same feature and should be treated as such.

[chrisdlangton]

@reversefold this is what I have been saying all along. Thank you for agreeing!

User's should perform the action, this is the use case exactly. Don't ask boto3 to do the action automagically in the framework for all users

[reversefold]

@chrisdlangton "The user" can be a script or library too. Your arguments have all been shown to be nothing but your own opinion, not backed up by fact in any way. The argument for this feature not being implemented in boto3 (or botocore) has been disproven. I urge you to stop wasting everyone's time with your trolling of this ticket and find something more useful to do with your time.

[chrisdlangton]

@reversefold Justin.. strange how your latest post is a response to my support of what you provided, which were additional quotes from AWS, on top of my own, that support the intention of STS i have been sharing all along.

All you do in this last comment is troll me, personally, and try to discredit me which equally discredited your own prior posting!!

Get your thoughts together and be consistent, coherent, and try not to contradict yourself and alienate the person that supported you just to look cool and fit in with the majority of in this thread. Try, if you can be reasonable, realise these so-called majority are only here as a small 0.0001% of boto3 users asking for a change that will affect the realistic majority of boto3 users, as STS is a extremely important security primitive.

For clarity, use STS assume role as you see fit with your userland code, refresh and force the temporary STS credential to simulate an IAM key/secret pair if that is what you believe you need to do. If you don't mind making it extremely difficult for operations and security teams incident response efforts to do their job, misuse STS and avoid the long lived purpose built systems that support critical operations and security requirements such as revoking access with immediate affect - Cognito, Instance profile, and IAM key/secret all support this, STS does not because tokens remain trusted for the entire duration (up to 12hr).

These are facts, supported by provided materials, none of it is opinion I give, the advice I gave originally and stick to consistently is;
You all have the right to use STS how you want, what you ask for in this feature request thread is already a capability boto3 provides where many of you are already running the few lines of code needed, and new observers may find some PoC here too if they chose to use STS in this way.

If anyone truly believes boto3 need to take the current functionality they provide and give you a better experience, try making a credible counterpoint backed by sources instead of the imflamitory denouncement of those that oppose you. It's petty and unhelpful.

I'll repeat the statement echoed on every open source project in existence create a PR to fix the bug or user experience you're having trouble with.

Finally, boto3 maintainers should be cautious not to change the STS service in this framework in such a way that affects the default security intention of the service (optional features are fine, they exist already), the current userbase has expectations on how it operates today, changing the functionality breaks business logic and assumed security posture in this case. Again, optional features are ok, and exist already. If raising a PR or finding a reasonable source to counterpoint these is not something you can do, please act in good faith and restrain yourself from typing out yet another imflamitory nonsensical reply in protest to the facts

[mrcranky]

"The user" can be a script or library too.

Rather than further engage with the circular reasoning and misdirection here, I decided to simply engage with AWS Enterprise Support for clarity on this:

we do not intend to dissuade customers from automatic credential refreshing because we do have that capability built-in to our profiles and at least one SDK.

As far as AWS are concerned, boto3 is the user. Regardless of whether it is implemented via a lazy resolve in something like ChainableTemporaryCredentials, implemented internally using set of "master credentials" stored with the Session, or done via hooks that allow interception and renewal of expired credentials seamlessly mid-operation.

Support have promised that someone on their side will engage with boto3 to progress whatever feature requests are required to support this, with the obvious caveat that there's no guarantee of ETA or priority on this work.

[ericfrederich]

Good to hear @mrcranky. Functionality was always there, just need to make it more user-friendly. I may engage my enterprise support as well. Until then I'll continue to use get_role_session from boto3_extensions (permalink at time of comment).

[prabodhm]

@kapilt dunno if this was already resolved. Your solution helped me get started. I used it and enhanced it to avoid accessing the internal variables of the session. I am posting my solution so that it may help others.

class AWSCredsRefresh:

    def run(self):
        session = get_session()
        cred_provider = session.get_component('credential_provider')

        cred_provider.insert_before('env', CustomCredentialProvider())

        boto3_session = Session(botocore_session=session)
        #Perform AWS operations with boto3_session

class CustomCredentialProvider(CredentialProvider):
    CANONICAL_NAME = "custom-creds"

    def __init__(self):


    def load(self):
        #These creds will be automatically refreshed using the _refreh method if the current creds are going to expire in 15 mins or less
        creds = DeferredRefreshableCredentials(refresh_using=self._refresh, method="sts-assume-role",)
        return creds

    def _refresh(self):
        #Refresh your AWS creds using custom process
        response = self._custom_aws_cred_refresh()
        credentials = {
            "access_key": response.get("AccessKeyId"),
            "secret_key": response.get("SecretAccessKey"),
            "token": response.get("SessionToken"),
            "expiry_time": response.get("Expiration").isoformat(),
        }
        return credentials

    def _custom_aws_cred_refresh(self):
        #Your custom AWS cred refresh code
        return response


if __name__ == '__main__':
    obj = AWSCredsRefresh()
    obj.run()