Aws-sdk-java-v2: NPE in S3AsyncClient.putObject with endpoint override

[aeons]

When attempting to use putObject on an S3AsyncClient with a Path as the second argument, an NPE is thrown because content is null in this line: https://github.com/aws/aws-sdk-java-v2/blob/75440747143a78f19606440d82ebf5cd8837418c/services/s3/src/main/java/software/amazon/awssdk/services/s3/AwsS3V4Signer.java#L177

Expected Behavior

The put object request should succeed or fail with a meaningful error.

Current Behavior

The put object request throws an NPE with this stacktrace:

[error] java.lang.NullPointerException
[error]         at software.amazon.awssdk.services.s3.AwsS3V4Signer.getContentLength(AwsS3V4Signer.java:178)
[error]         at software.amazon.awssdk.services.s3.AwsS3V4Signer.calculateContentHash(AwsS3V4Signer.java:115)
[error]         at software.amazon.awssdk.core.auth.Aws4Signer.doSign(Aws4Signer.java:199)
[error]         at software.amazon.awssdk.core.auth.Aws4Signer.lambda$sign$0(Aws4Signer.java:179)
[error]         at software.amazon.awssdk.utils.builder.SdkBuilder.apply(SdkBuilder.java:58)
[error]         at software.amazon.awssdk.utils.builder.ToCopyableBuilder.copy(ToCopyableBuilder.java:43)
[error]         at software.amazon.awssdk.core.auth.Aws4Signer.sign(Aws4Signer.java:179)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.SigningStage.signRequest(SigningStage.java:60)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.SigningStage.execute(SigningStage.java:48)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.SigningStage.execute(SigningStage.java:35)
[error]         at software.amazon.awssdk.core.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:239)
[error]         at software.amazon.awssdk.core.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:239)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.doExecute(AsyncRetryableStage.java:164)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.execute(AsyncRetryableStage.java:111)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.execute(AsyncRetryableStage.java:105)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.AsyncRetryableStage.execute(AsyncRetryableStage.java:68)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.AsyncRetryableStage.execute(AsyncRetryableStage.java:47)
[error]         at software.amazon.awssdk.core.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:239)
[error]         at software.amazon.awssdk.core.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:239)
[error]         at software.amazon.awssdk.core.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:239)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.execute(AsyncExecutionFailureExceptionReportingStage.java:36)
[error]         at software.amazon.awssdk.core.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.execute(AsyncExecutionFailureExceptionReportingStage.java:25)
[error]         at software.amazon.awssdk.core.http.AmazonAsyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonAsyncHttpClient.java:208)
[error]         at software.amazon.awssdk.core.client.AsyncClientHandlerImpl.doInvoke(AsyncClientHandlerImpl.java:220)
[error]         at software.amazon.awssdk.core.client.AsyncClientHandlerImpl.invoke(AsyncClientHandlerImpl.java:199)
[error]         at software.amazon.awssdk.core.client.AsyncClientHandlerImpl.execute(AsyncClientHandlerImpl.java:142)
[error]         at software.amazon.awssdk.core.client.AsyncClientHandlerImpl.execute(AsyncClientHandlerImpl.java:77)
[error]         at software.amazon.awssdk.core.client.SdkAsyncClientHandler.execute(SdkAsyncClientHandler.java:46)
[error]         at software.amazon.awssdk.services.s3.DefaultS3AsyncClient.putObject(DefaultS3AsyncClient.java:2571)
[error]         at software.amazon.awssdk.services.s3.S3AsyncClient.putObject(S3AsyncClient.java:3562)
[error]         at Repro.main(Repro.java:27)

Possible Solution

Steps to Reproduce (for bugs)

A minimal repro (translated from Scala, so might not be idiomatic):

import software.amazon.awssdk.services.s3.model.*;
import software.amazon.awssdk.core.regions.*;
import software.amazon.awssdk.core.auth.*;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import java.nio.file.*;
import java.net.URI;
class Repro {
    public static void main(String[] args) {
        S3AsyncClient client = S3AsyncClient
                .builder()
                .credentialsProvider(
                        StaticCredentialsProvider.create(AwsCredentials.create("access", "secret")))
                .endpointOverride(URI.create("http://localhost:4572"))
                .region(Region.EU_WEST_1)
                .build();

        PutObjectRequest req = PutObjectRequest
                .builder()
                .bucket("test")
                .key("test.txt")
                .build();

        Path path = Paths.get("/");

        client.putObject(req, path);
    }
}

Context

If the endpoint override is not supplied, then everything works. I've tried stepping through the code, but I couldn't find the place where the content in the request should be set from the AsyncRequestProvider.

Your Environment

• AWS Java SDK version used: 2.0.0-preview-8
• JDK version used: 1.8.0_152-b16
• Operating System and version: macOS 10.13.3

[aeons]

Any update on this?

[by-dam]

Facing the same problem here.

When using non-secure (http) endpoint the AWS SDK always tries to sign the request (AwsS3V4Signer.isPayloadSigningEnabled) but in the async case the content is not available in the request at this stage (content is null as it is provided by an AsyncRequestProvider), hence the NullPointerException.

[telendt]

When using non-secure (http) endpoint the AWS SDK always tries to sign the request (AwsS3V4Signer.isPayloadSigningEnabled) but in the async case the content is not available in the request at this stage (content is null as it is provided by an AsyncRequestProvider), hence the NullPointerException.

Thanks. If you're like me trying to connect S3AsyncClient to your local HTTP mock server then the following config seems to help:

S3AsyncClient.builder()
    .endpointOverride(URI.create("http://localhost:$port"))
    .overrideConfiguration(
        ClientOverrideConfiguration.builder()
            .advancedOption(AdvancedClientOption.SIGNER_PROVIDER, NoOpSignerProvider())
            .build())
    .build()

[zoewangg]

Fixed via #662 and it was released as part of 2.0.0-preview-12