Aws-sdk-go: aws/session: Provide option for SDK to automatically get REGION from EC2 metadata.

[antoniomo]

When using session.NewSession() on an EC2 instance, it would by default use the EC2RoleProvider in it's provider chain. If the instance is properly configured, that's all there is to provide credentials and permissions as it would get it from the metadata service. However, we still need to pass an aws.Config with region. Since the metadata service has access to the region, it would be nice to have an elegant way to get that too.

Right now I use something similar to this, that works but feels clumsy (ignoring errors for example reasons):

metaSession, _ := session.NewSession()
metaClient := ec2metadata.New(metaSession)
region, _ := metaClient.Region()

conf := aws.NewConfig().WithRegion(region)
realSession, _ := session.NewSession(conf)
dynDb := dynamodb.New(realSession) // Or whatever AWS service we want to use

Perhaps there are cleaner ways to do this already, if that's the case I appreciate the insight :)

[xibz]

Hello @antoniomo, thank you for reaching out to us. There are multiple ways of providing region, not through code, that may achieve what you are looking for. You can specify either in ~/.aws/credentials or ~/.aws/config your region. If using the shared config,~/.aws/config, you must set AWS_SDK_LOAD_CONFIG to some value to enable it. Another option is to provide it through the AWS_REGION environment variable. Please let me know if that is a better usage for you.

[antoniomo]

Hi @xibz and thanks for the answer!

I'm aware of those possibilities, but we feel that the most secure way to provide credentials to our EC2 instances is through instance IAM roles. If that's the case, we just want to rely on the EC2RoleProvider. As this provider queries the metadata service for credentials anyway, it would be cleaner in code if there was an elegant way to connect it to the region query. And yes, setting a config or environment variable achieves the same result, but makes provisioning a bit harder, and if we are using the metadata service anyway, we could achieve both cleaner code and cleaner deployments with this feature.

Maybe this is more like a feature request than a question if the only way to do that is along the lines of the code sample I put above. Of course if this is a corner case, it's a totally discardable feature request :)

[xibz]

@antoniomo - After some thought, a redesign of the credential package to not need a region, if on an instance, could be a potential. I'll take some time thinking about this and let you know what we come up with. For now I'll mark this as a feature request. Please let us know if you have any questions!

[antoniomo]

Thanks for considering! I'm aware that it's not a perfect fit as it's a credential provider and in the other credential providers so region should be out of it. Perhaps there could be a separate config provider that can read from ec2metadata if running on an instance?

If you come up with a nice way to make it happen that would be great. Thanks for a great SDK!

[jasdel]

I think to easily enable this feature we could add an additional option to the session Options that instructs the session to attempt to retrieve the region from ec2metadata automatically if the region is not already set via aws.Config or ENV var.

Probably should also investigate if this could also be done with a new shared ENV var or shared config field that would instruct the SDKs to get the region from ec2metadata automatically.

[antoniomo]

To me both look very reasonable, depending if the user prefers to set it on code with session options or via provisioning tools with ENV vars.

[mikkeloscar]

+1 for this feature, we have several small services talking to the AWS API where they only care about the region in which they are deployed. If this was automatically discovered from the ec2 metadata we would not have to specify the AWS_REGION on all deployments (or implement the discovery ourselves outside of the SDK).

I would expect the SDK to automatically find everything from the ec2metadata unless it's instructed otherwise by ENV vars or it's specifically hardcoded in the code. I believe this is how the python sdk (boto3) works by default, and it's very convenient.

[Quentin-M]

I was also very surprised to see the following error popping up.. I expect session.NewSession() to get all the required information by itself when running on EC2.

MissingRegion: could not find region configuration

Creating a temporary session simply to find the region through the metadata service, and then creating a real session is indeed counterintuitive.

Thanks.

[BalmungSan]

Hello, any plans to implement this? - it would be great for us.

We are deploying many containers in ecs, all of them use ssm-parent to fetch some secrets from AWS SSM Parameter store in the same region the containers are executing. For now we have to set the AWS_REGION env var in the task definition of each container.

PS: Curiously when running the services with FARGATE _(instead of in a EC2 cluster)_ we don't need to specify the region - any ideas why?

[samuelkarp]

Curiously when running the services with FARGATE (instead of in a EC2 cluster) we don't need to specify the region - any ideas why?

In Fargate, the AWS_REGION and AWS_DEFAULT_REGION environment variables are added automatically since the EC2 instance metadata service is not available.

[BalmungSan]

In Fargate, the AWS_REGION and AWS_DEFAULT_REGION environment variables are added automatically since the EC2 instance metadata service is not available.

Ah, thanks! - at least I know why it worked.

[cjcjameson]

Beware getting a nil-pointer back from ec2metadata.New() in the workaround code!

metaSession, _ := session.NewSession()
metaClient := ec2metadata.New(metaSession)
region, _ := metaClient.Region()