Boto: s3.generate_url appends x-amz-security-token to query string even when query_auth=False

[nakedible]

It seems s3.generate_url will produce a url with a query parameter x-amz-security-token if the current auth provider has a security token even when passing query_auth=False. This url is totally non-working as it has the x-amz-security-token parameter, but none of the other authentication parameters.

[teeberg]

I've had the same issue with public static links, and helped myself by subclassing the S3BotoStorage like this:

import urllib
import urlparse

class PublicS3BotoStorage(S3BotoStorage):
    def __init__(self, *a, **k):
        kwargs = dict(location='public', querystring_auth=False)
        # merge in any arguments that were passed
        kwargs.update(k)
        super(PublicS3BotoStorage, self).__init__(*a, **kwargs)

    def url(self, name):
        orig = super(PublicS3BotoStorage, self).url(name)
        scheme, netloc, path, params, query, fragment = urlparse.urlparse(orig)
        params = urlparse.parse_qs(query)
        if 'x-amz-security-token' in params:
            del params['x-amz-security-token']
        query = urllib.urlencode(params)
        return urlparse.urlunparse((scheme, netloc, path, params, query, fragment))

I can't tell in which cases exactly the security token should be stripped, but this suited my needs. Adapt it to your usecase.

[forbode]

This issue still exists. Oddly enough the querystring_auth settings is respected locally, but when I deploy to Elastic Beanstalk it breaks. Other parameters seem to work (AWS_S3_URL_PROTOCOL,AWS_S3_CALLING_FORMAT). I will try teeberg's suggestion.

[forbode]

Teeberg's solution worked for me. It works both in locally and on S3.

[adepue]

I ran into this as well.. worked around it by doing:

conn = S3Connection()
conn.provider.security_token = ""

[LewisMcMahon]

This is still an issue if your using iam roles

[gholms]

This should have been fixed in commit 43217f97, included in boto 2.40.0. Does it work for you now?

[joshma]

@gholms if query_auth=False, won't https://github.com/boto/boto/commit/43217f97d3e7367e5e9b3dbb694381b4ede51125 not affect this particular codepath? It looks like it's still hitting https://github.com/boto/boto/blob/43217f97d3e7367e5e9b3dbb694381b4ede51125/boto/s3/connection.py#L422 - the headers are added to the query part still.

[kylegibson]

Still an issue in boto 2.48.0