Aws-sdk-java-v2: Run IAMClient met exception : Unable to execute HTTP request: iam.us-east-1.amazonaws.com

[xuzhang101]

Current Behavior

Create IAMClient with the code below :
IAMClient.builder()
.overrideConfiguration(clientConfiguration)
.region(region)
.endpointOverride(URI.create("https://iam.amazonaws.com"))
.build();
When I run the IAMClient, I met the exception
software.amazon.awssdk.SdkClientException: Unable to execute HTTP request: iam.us-east-1.amazonaws.com at software.amazon.awssdk.http.pipeline.stages.RetryableStage$RetryExecutor.handleThrownException(RetryableStage.java:212)

I applied a workaround solution
IAMClient.builder()
.overrideConfiguration(clientConfiguration)
.region(region)
.credentialsProvider(new SystemPropertyCredentialsProvider())
.endpointOverride(iamURI)
.build();

Then, I met another exception :
software.amazon.awssdk.services.iam.model.IAMException: Credential should be scoped to a valid region, not 'eu-west-1'. (Service: IAMClient; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: c2ce17d7-ae81-11e7-a24f-d5e0ad542460)
at software.amazon.awssdk.http.pipeline.stages.HandleResponseStage.handleErrorResponse(HandleResponseStage.java:117)
at software.amazon.awssdk.http.pipeline.stages.HandleResponseStage.handleResponse(HandleResponseStage.java:73)
at software.amazon.awssdk.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:58)

The credentials was already set
System.setProperty("aws.accessKeyId", accessKey);
System.setProperty("aws.secretAccessKey", secretKey);

• AWS Java SDK version used:
  2.0.0-preview-4
• JDK version used:
  java 1.8
• Operating System and version:
  Windows 10

[spfink]

What are you passing as the region?

[xuzhnag]

Hi Spfink,
Thank you for your answer. The region I set withRegion.of("us-east-1")

[spfink]

@xuzhnag, IAM requires the use of Region.AWS_GLOBAL currently as it is a global service. Passing in us-east-1 will attempt to call iam.us-east-1.amazonaws.com which does not exist.

If you have an opinion on this we'd love to hear it. We've had some feedback that because AWS_GLOBAL isn't a real AWS region it can be confusing. We want to find a way to make it clear to customers though they are not using a regional service and there requests are not staying within region.

[art-bjss]

It seems odd that IAMClient.builder().build() doesn't set a sensible default region (i.e. AWS_GLOBAL)

I'm using 2.0.0-preview-7

[digitalsanctum]

I've also run into this. In my case, region was set via profile. Since IAM is always global why not set that explicitly?

[tlrdstd]

This is nearly two years old, and still biting folks. I'm writing a ClientFactory class that takes the current region and caches a correctly authenticated client for use across my app. With SDKv1, we could do:

com.amazonaws.regions.RegionUtils.getRegion(region).createClient(
    com.amazonaws.services.identitymanagement.AmazonIdentityManagementClient.class, 
    credentialsProvider,
    PredefinedClientConfigurations.defaultConfig())
);

But in SDKv2, the closest equivalent for the IamClient uses the wrong endpoint:

jshell> software.amazon.awssdk.services.iam.IamClient
    .builder()
    .region(software.amazon.awssdk.regions.Region.of("us-west-2"))
    .credentialsProvider(credentialsProvider)
    .build()
    .listServerCertificates(ListServerCertificatesRequest.builder().build())
|  software.amazon.awssdk.core.exception.SdkClientException thrown: Unable to execute HTTP request: iam.us-west-2.amazonaws.com
|        at SdkClientException$BuilderImpl.build (SdkClientException.java:97)
|        at RetryableStage$RetryExecutor.handleThrownException (RetryableStage.java:136)
|        at RetryableStage$RetryExecutor.execute (RetryableStage.java:94)
|        at RetryableStage.execute (RetryableStage.java:62)
|        at RetryableStage.execute (RetryableStage.java:42)
|        at RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
|        at StreamManagingStage.execute (StreamManagingStage.java:57)
|        at StreamManagingStage.execute (StreamManagingStage.java:37)
|        at ApiCallTimeoutTrackingStage.executeWithTimer (ApiCallTimeoutTrackingStage.java:80)
|        at ApiCallTimeoutTrackingStage.execute (ApiCallTimeoutTrackingStage.java:60)
|        at ApiCallTimeoutTrackingStage.execute (ApiCallTimeoutTrackingStage.java:42)
|        at RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
|        at RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
|        at ExecutionFailureExceptionReportingStage.execute (ExecutionFailureExceptionReportingStage.java:37)
|        at ExecutionFailureExceptionReportingStage.execute (ExecutionFailureExceptionReportingStage.java:26)
|        at AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute (AmazonSyncHttpClient.java:240)
|        at BaseSyncClientHandler.invoke (BaseSyncClientHandler.java:96)
|        at BaseSyncClientHandler.execute (BaseSyncClientHandler.java:120)
|        at BaseSyncClientHandler.execute (BaseSyncClientHandler.java:73)
|        at SdkSyncClientHandler.execute (SdkSyncClientHandler.java:44)
|        at AwsSyncClientHandler.execute (AwsSyncClientHandler.java:55)
|        at DefaultIamClient.listServerCertificates (DefaultIamClient.java:7469)
|        at (#53:1)

Put more simply, the IamClient calculates an endpoint that is literally, objectively wrong:

jshell> software.amazon.awssdk.services.iam.IamClient.serviceMetadata()
    .endpointFor(software.amazon.awssdk.regions.Region.of("us-west-2"))
$45 ==> iam.us-west-2.amazonaws.com

Very strong opinion here: IamClient.serviceMetadata().endpointFor(regionName) should return an endpoint that actually works in that region. Telling me to pass Region.AWS_GLOBAL doesn't help when I'm operating in all AWS partitions and don't want to mess with the magic fake region names that IAM technically supports:

jshell> software.amazon.awssdk.services.iam.IamClient.serviceMetadata().regions()
$47 ==> [aws-global, aws-cn-global, aws-us-gov-global, aws-iso-global, aws-iso-b-global]

[tlrdstd]

For the record, here is a mockup of my current workaround:

public IamClient getIamClient(final String region) {
  // IAM is a global service that does not have regional endpoints,
  // and the V2 SDK can't figure that out
  // https://github.com/aws/aws-sdk-java-v2/issues/206
  final String arnPartition = software.amazon.awssdk.regions.Region.of(region)
    .metadata().partition().id();

  // Have to generate a fake partition/region name that matches:
  // jshell> software.amazon.awssdk.services.iam.IamClient.serviceMetadata().regions()
  // $63 ==> [aws-global, aws-cn-global, aws-us-gov-global, aws-iso-global, aws-iso-b-global]
  final String iamGlobalPartition = String.format("%s-global", arnPartition);

  return software.amazon.awssdk.services.iam.IamClient
    .builder()
    .region(iamGlobalPartition)
    .credentialsProvider(credentialsProvider)
    .build()
}

[debora-ito]

Changing this to a feature request to provide a better experience with IAM endpoint.