Aws-cli: How do I choose STS endpoint for aws cli calls?

[hysoftwareeng]

For example, when I call aws s3 ls --profile user1, the underlying code calls sts and goes to the STS endpoint sts.amazonaws.com:443. I tried to use the region flag --region us-east-1 to try to route this to the region specific sts endpoint sts.us-east-1.amazonaws.com but it's not working. I have a private link setup for my VPC to this sts endpoint, that's why I need to connect to this one and not the global one. The global sts call is causing all my aws cli commands to timeout. Please advise.

[hysoftwareeng]

command options is an invalid flag. Only these are available right?

assume-role
assume-role-with-saml
assume-role-with-web-identity
decode-authorization-message
get-access-key-info
get-caller-identity
get-federation-token
get-session-token

When I just call a sts command, I could use --endpoint-url https://sts.us-east-1.amazonaws.com. But this doesn't help when I call somethin like aws s3 ls

[justnance]

@hysoftwareeng -

As per our documentation, AWS Security Token Service (STS) is available as a global service, and all AWS STS requests go to a single endpoint at https://sts.amazonaws.com by default which is the US East (N. Virginia) region aka sts.us-east-1.amazonaws.com. There is more information in our docs about using sts and regions here.

The available aws sts commands you listed are correct and also found in our documentation.

How do you have the region configured in the name profile --profile user1? Is it set to use us-east-1.

[hysoftwareeng]

@justnance It looks like the endpoint https://sts.amazonaws.com resolves to a different IP than sts.us-east-1.amazonaws.com even though they are the same region, and yes I do have region configured to us-east-1 in my profile. In the endpoints of my VPC, I can only add a private link for sts.us-east-1.amazonaws.com, but when using the CLI, this still resolves to https://sts.amazonaws.com which is blocked because I do not have a internet gateway setup. It seems like the CLI cannot override the global endpoint for us-east-1.

[hysoftwareeng]

Could someone provide a response on this? aws cli calls are still not working for those that default to us-east-1 due to timeout issue in a private VPC. @justnance

[dee-kryvenko]

I just ran into the same issue. Found this https://github.com/aws/aws-cli/issues/4463 - basically it sounds like regional endpoints are not supported for sts yet. It works if I were to do it like this though (I know it sucks to manually provide an endpoint, but that's at least something):

aws --region us-west-2 sts get-caller-identity --endpoint-url https://sts.us-west-2.amazonaws.com

[timhargreaves]

Ran into the same problem. The workaround from @llibicpep works (thanks) but having profile support would be ideal.

@justnance While the documentation does indicate that STS is by default a global service, the same documentation also recommends using the regional endpoints. I was using a similar setup as @hysoftwareeng where I had a private subnet and an STS PrivateLink endpoint, which _requires_ using the regional domain.

[timhargreaves]

@hysoftwareeng To make your calls to services work, you have to do a few steps:

1. Make an STS assumerole call to get the temporary credentials (using llibicpep's workaround)
2. Stuff those credentials into environment variables
3. Make subsequent calls i.e. to S3, which should reference the env variables
4. Unset the variables when you're done

Ref: https://aws.amazon.com/premiumsupport/knowledge-center/iam-assume-role-cli/

[hysoftwareeng]

@timhargreaves thanks for the response, I can get all other regional endpoints to work with cmd except us-east-1, seems like that's a bug on the AWS side that they haven't addressed.

[Space-Venom]

+1
Not sure why the issue was closed. The OP found a workaround, but the issue remains.
Just tested the same on Amazon Linux 2 latest in us-east-1 and a VPC endpoint attached.
The only way this worked is by specifying the endpoint URL and the region:
aws sts get-caller-identity --endpoint-url https://sts.us-east-1.amazonaws.com --region us-east-1

[nik-willingham]

+1
Yeah this is causing an issue for some things we're trying to do with containers that we don't want to be internet-facing. This should definitely be fixed to take the declared region into account. Please reopen this and fix it.

[rajholla]

Below should work:

export AWS_STS_REGIONAL_ENDPOINTS=regional
aws sts get-caller-identity --region us-east-1 

[DarkMukke]

UPDATE:

• I got it to work, the default aws cli installed on the Amazon Linux, is completely out of date.
• after updating it with yum it now works just fine
• you can configure your region with aws configure and then you don't have to supply it in the command either

export AWS_STS_REGIONAL_ENDPOINTS=regional
aws sts get-caller-identity --debug --region eu-west-2
...
2020-05-30 02:43:24,760 - MainThread - botocore.endpoint - DEBUG - Sending http request: 
<AWSPreparedRequest stream_output=False, method=POST, url=https://sts.amazonaws.com/, 
headers={'Content-Length': '43', 'X-Amz-Date':
...

---

Original Post:

that does not work

nslookup sts.amazonaws.com
Server:     10.226.48.2
Address:    10.226.48.2#53

Non-authoritative answer:
Name:   sts.amazonaws.com
Address: 54.239.29.25

I have a private endpoint for STS though

nslookup sts.eu-west-2.amazonaws.com
Server:     10.226.48.2
Address:    10.226.48.2#53

Non-authoritative answer:
Name:   sts.eu-west-2.amazonaws.com
Address: 10.226.49.16
Name:   sts.eu-west-2.amazonaws.com
Address: 10.226.50.85
Name:   sts.eu-west-2.amazonaws.com
Address: 10.226.48.63

However

aws sts get-caller-identity --endpoint-url https://sts.eu-west-2.amazonaws.com

does work

[hugoprudente]

Can someone re-open this as it's not clear at all and it causes a lot of misleading when using VPCendpoints.

[rajaie-algorithmia]

Ran into the same issue here. Had VPC endpoints created for sts but aws sts get-caller-identity was failing unless I provided the endpoint with the region