Aws-sdk-java-v2: Async response handler creates a new thread for every request on small instances

[kevinkarpenske]

In preview-13 and later, async client request execution uses the default async pool during request execution. For instances with 2 or fewer physical cores, the default async pool is backed by a ThreadPerTaskExecutor which spawns a new thread for every request.

This issue is further exacerbated by the SDK's XmlDomParser, which uses a ThreadLocal to retain a reference to its underlying XML factory and therefore must initialize the factory with every request.

Expected Behavior

The SDK should not spawn a new thread for every request, regardless of instance size.

Current Behavior

The SDK uses the default async pool, which for small instances has undesirable behavior. See: https://bugs.openjdk.java.net/browse/JDK-8213115

Example of a thread and its call stack per jvisualvm:

Possible Solution

The SDK should use same-thread composition (thenCompose) to avoid delegation to the JVM's async pool.

If same-thread composition is deemed undesirable, utilize a new or existing (ideally configurable) executor for async composition. The obvious candidate is the existing configurable FUTURE_COMPLETION_EXECUTOR (defaults to the sdk-async-response pool) given the forthcoming fix. Given that the initial future was likely executed using that pool, this is further argument for simply using thenCompose.

A workaround given the current implementation is to force higher parallelism in the common ForkJoinPool using the java.util.concurrent.ForkJoinPool.common.parallelism system property.

Steps to Reproduce (for bugs)

Execute requests using an async client on an instance with 2 or fewer cores (as reported by Runtime.getRuntime().availableProcessors()).

Context

Encountered while running an application in AWS EC2 using a c4.large instance type, which has 2 cores.

Your Environment

• AWS Java SDK version used: 2.1.1
• JDK version used: 1.8.0_191
• Operating System and version: Ubuntu 14.04

[millems]

That's definitely a bug. It's not intended for the SDK to use the JVM's default async pool.

[millems]

Dug through the code base, and this seems to be the only place we've made the mistake. It should be a simple fix. We have access to an executor service we can use through the execution attributes or, more likely, we can just do it synchronously on the completing thread.

[zoewangg]

The fix has been released in 2.3.0. Closing the issue.

[amigold]

I'm still getting it on 2.3.8... any ideas?

[DamienOReilly]

Hi, @zoewangg I am also observing this issue, is there any further updates to this? Using 2.5.60.

[cwei-bgl]

this is still a issue in 2.9.1. Any further updates to this?

[debora-ito]

Reopening, as it looks like the issue was not resolved.

@amigold @DamienOReilly @cwei-bgl Do you have a minimal reproducible code you can share?

[zoewangg]

Hi @amigold , @DamienOReilly @cwei-bgl

The response completion pool is created to complete the response to prevent the event loop threads from being potentially blocked.

The default executor has 50 core threads and you can override it by providing a custom executor via sdkAdvancedAsyncClientOption.FUTURE_COMPLETION_EXECUTOR like the following:

DynamoDbAsyncClient.builder()
                   .asyncConfiguration(b -> b.advancedOption(SdkAdvancedAsyncClientOption.FUTURE_COMPLETION_EXECUTOR,
                                                             customExecutor))
                   .build();

See more info from #1251 and https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/basics-async.html

We do agree that creating 50 core threads in the response completion pool for the SDK client might not be ideal for every use case and #1436 is created to address that issue.

[zoewangg]

We adjusted the core size and max size for the default future completion executor in #1436 and the change has been released as part of 2.9.9. As mentioned earlier, you can also provide your own executor if the default one does not work for your use case.

Closing the issue, feel free to open new issues if you have further questions.