Boto3: Any method to get s3 endpoint url for a given region?

[phobologic]

Hi, I'm curious if there is any way in the library to get the endpoint for s3 in a given region? I need to generate the template url for cloudformation [create/update]_stack calls. Thanks!

[dstufft]

Does the get_available_regions() (http://boto3.readthedocs.io/en/latest/reference/core/session.html#boto3.session.Session.get_available_regions) give you the information you're looking for? If not I think the botcore.region module has it, but I don't believe those are public APIs.

[phobologic]

@dstufft unfortunately get_available_regions() doesn't do this. It just returns all the possible regions (despite the docs saying something about endpoints, not sure why).

Basically I'm just looking for a function that, given the value in the Region column, returns the value in the Endpoint column found here: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

[dstufft]

Ok, the answer is there isn't a direct function, but you can load all of that data using:

import botocore.loaders

loader = botocore.loaders.create_loader()
data = loader.load_data("endpoints")

# At this point, you have a big data structure which isn't great
# to work with. However, there is an API you can use to work with
# it, but it's undocumented so it might change in the future.

import botocore.regions

resolver = botocore.regions.EndpointResolver(data)
endpoint_data = resolver.construct_endpoint("s3", "us-east-1")

# At this point, you have endpoint_data which contains something like:
{
    'hostname': 's3.amazonaws.com',
    'signatureVersions': ['s3', 's3v4'],
    'partition': 'aws',
    'endpointName': 'us-east-1',
    'protocols': ['http', 'https'],
    'dnsSuffix': 'amazonaws.com',
}

# At this point, you can do some basic url generation to get the URL.

Besides that, I don't believe there is any other way to get at this programatically.

[phobologic]

Ahh, cool - thanks @dstufft.

[dstufft]

Glad that was helpful!

[aladagemre]

A quick and dirty function for those who need that URL builder.

def get_endpoint_for_s3_bucket(bucket_name, region='eu-west-1'):
    import botocore.loaders
    import botocore.regions
    loader = botocore.loaders.create_loader()
    data = loader.load_data("endpoints")
    resolver = botocore.regions.EndpointResolver(data)
    endpoint_data = resolver.construct_endpoint("s3", region)
    parts = endpoint_data['hostname'].split("-")
    parts.insert(1, "website")
    url_postfix = "-".join(parts)
    url = "{bucket_name}.{url_postfix}".format(bucket_name=bucket_name, url_postfix=url_postfix)
    return url

[kichik]

I'm not sure when this was added, but you can now use client.meta.endpoint_url:

import boto3
import urllib.parse

ep = boto3.client("s3", region_name="eu-west-1").meta.endpoint_url
ep = urllib.parse.urlparse(ep).hostname