Gsutil: support for aws_session_token

[nathankw]

Hi, I'm using gsutil v. 4.3.1 and there isn't any apparent support for the AWS_SESSION_TOKEN, which is a required configuration setting when working with AWS pre-signed URLs.

Use case: I'm trying to upload an object from a GCP bucket (in my account) to S3. I have been provided with an AWS pre-signed URL and corresponding credentials that include

1. AWS_ACCESS_KEY_ID
2. AWS_SECRET_ACCESS_KEY
3. AWS_SESSION_TOKEN

I have tested with these variables in the .boto file, and even in different tests as environment variables, and the response I get is "403 AccessDenied" when running my cp command that is of the form

gsutil cp gs://mybucket/obj s3://s3bucket/obj

Things work fine when not using a pre-signed URL.
After lots of testing, I wanted to verify that the AWS_SESSION_TOKEN is not supported, and to request for such support.

Thanks,
-Nathan

[nathankw]

Hi,
Just wanted to check back. Is this something that could make it to the feature list?
Let me know if you need anymore details.

[houglum]

It looks like there are two asks here, neither of which gsutil supports at the moment:

• Uploading to a GCS Signed URL or an S3 Pre-Signed URL
• Adding support for AWS_SESSION_TOKEN (I also mentioned this in the Stackoverflow post at [1], which mentions this is unlikely because it's not supported in the Boto library).

[1] https://stackoverflow.com/questions/51353850/configure-gsutil-boto-file-to-use-aws-sts-assume-role

[iampat]

The last answer is from Aug 2018. Is there any update since then?

[catleeball]

These features would rely on us extending or decoupling from the old deprecated boto library. Currently we have it on our roadmap to explore our relationship with boto and possibly decoupling from it, along with swapping oath2client with google-auth and swapping httplib2 with requests library.

Likely this would be explored after we figure out and implement a path away from boto. These dependency changes are currently goals for us, although the boto change is a slightly lower priority than the oauth2client -> google-auth change.

Likewise, gsutil is transitioning maintainers, which may mean reprioritizing goals depending on the new owner's priorities and roadmap.

In short, we will definitely keep you updated on this! We have not forgotten you. :slightly_smiling_face:

[arinto]

Any updates on this feature?

[ggiill]

+1 - It would be extremely useful to be able to sync from S3 (with an STS Assumed Role) to GCS using gsutil.