Aws-sdk-java-v2: java.lang.NumberFormatException: For input string: "1.547993531819E9

[oripwk]

When using amazon-kinesis-client, I'm getting

Unexpected exception was thrown. This could probably be an issue or a bug. Please search for the exception/error online to check what is going on. If the issue persists or is a recurring problem, feel free to open an issue on, https://github.com/awslabs/amazon-kinesis-client.

Current Behavior

The ShardRecordProcessor fails when reading records from Kinesis stream with

java.lang.NumberFormatException: For input string: "1.547993531819E9"

Stack trace

java.lang.NumberFormatException: For input string: "1.547993531819E9"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_181]
    at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_181]
    at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_181]
    at software.amazon.awssdk.utils.DateUtils.parseUnixTimestampMillisInstant(DateUtils.java:146) ~[utils-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.core.StringToInstant.lambda$safeParseDate$0(StringToInstant.java:72) ~[protocol-core-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.core.StringToInstant.convert(StringToInstant.java:56) ~[protocol-core-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.core.StringToInstant.convert(StringToInstant.java:32) ~[protocol-core-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller$SimpleTypeJsonUnmarshaller.unmarshall(JsonProtocolUnmarshaller.java:160) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallStructured(JsonProtocolUnmarshaller.java:210) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallStructured(JsonProtocolUnmarshaller.java:114) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.lambda$unmarshallList$2(JsonProtocolUnmarshaller.java:143) ~[aws-json-protocol-2.3.2.jar:?]
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_181]
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_181]
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_181]
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_181]
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_181]
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_181]
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_181]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallList(JsonProtocolUnmarshaller.java:145) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallStructured(JsonProtocolUnmarshaller.java:210) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshall(JsonProtocolUnmarshaller.java:197) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshall(JsonProtocolUnmarshaller.java:168) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:79) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:36) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonResponseHandler.handle(AwsJsonResponseHandler.java:43) ~[aws-json-protocol-2.3.2.jar:?]
    at software.amazon.awssdk.core.client.handler.BaseClientHandler.lambda$interceptorCalling$2(BaseClientHandler.java:133) ~[sdk-core-2.3.2.jar:?]
    at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:40) ~[sdk-core-2.3.2.jar:?]
    at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:28) ~[sdk-core-2.3.2.jar:?]
    at software.amazon.awssdk.core.internal.http.async.SyncResponseHandlerAdapter.lambda$prepare$0(SyncResponseHandlerAdapter.java:85) ~[sdk-core-2.3.2.jar:?]
    at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952) ~[?:1.8.0_181]
    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_181]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_181]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_181]
    at software.amazon.awssdk.core.internal.http.async.SyncResponseHandlerAdapter$BaosSubscriber.onComplete(SyncResponseHandlerAdapter.java:127) ~[sdk-core-2.3.2.jar:?]
    at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.runAndLogError(ResponseHandler.java:158) ~[netty-nio-client-2.3.2.jar:?]
    at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.access$700(ResponseHandler.java:64) ~[netty-nio-client-2.3.2.jar:?]
    at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler$PublisherAdapter$1.onComplete(ResponseHandler.java:268) ~[netty-nio-client-2.3.2.jar:?]
    at com.typesafe.netty.HandlerPublisher.complete(HandlerPublisher.java:408) ~[netty-reactive-streams-2.0.0.jar:?]
    at com.typesafe.netty.HandlerPublisher.handlerRemoved(HandlerPublisher.java:395) ~[netty-reactive-streams-2.0.0.jar:?]
    at io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:670) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:505) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:451) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at com.typesafe.netty.http.HttpStreamsHandler.removeHandlerIfActive(HttpStreamsHandler.java:328) ~[netty-reactive-streams-http-2.0.0.jar:?]
    at com.typesafe.netty.http.HttpStreamsHandler.handleReadHttpContent(HttpStreamsHandler.java:189) ~[netty-reactive-streams-http-2.0.0.jar:?]
    at com.typesafe.netty.http.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:165) ~[netty-reactive-streams-http-2.0.0.jar:?]
    at com.typesafe.netty.http.HttpStreamsClientHandler.channelRead(HttpStreamsClientHandler.java:148) ~[netty-reactive-streams-http-2.0.0.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at software.amazon.awssdk.http.nio.netty.internal.FutureCancelHandler.channelRead0(FutureCancelHandler.java:35) ~[netty-nio-client-2.3.2.jar:?]
    at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:591) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:508) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]

Possible Solution

No possible solution

Steps to Reproduce (for bugs)

Context

This issue may be related: https://github.com/aws/aws-sdk-java-v2/issues/970

Your Environment

• AWS Java SDK version used:
  
  
  
  • amazon-kinesis-client:2.1.0
  
  
  • kinesis:2.3.2
  
  
  • dynamodb:2.3.2
  
  
  • cloudwatch:2.3.2
  
  
• JDK version used: 1.8
• Operating System and version: Alpine

[oripwk]

This is clearly a bug.

According to Kinesis docs, ApproximateArrivalTimestamp can be written in exponential notation, and in this specific case, it doesn't event evaluate to a long, but to double:

1.441215410867E9 == 1441215410.867

The stack trace shows that the parseLong(…) method is used, and there it fails.

This is a very critical bug and we'd appreciate if you can issue a fix.

Thanks

Edit

The root cause is that TimestampFormatTrait.Format evaluates to UNIX_TIMESTAMP_MILLIS instead of UNIX_TIMESTAMP. If it would have been the latter, the parseUnixTimestampInstant() method would be used instead of parseUnixTimestampMillisInstant() and it would have worked.

[aesteve]

Maybe it's related to the bug I reported in #1021 ? What do you think ?

[oripwk]

@aesteve it might be the case, since we also disabled CBOR for localstack.

[aesteve]

I'll add a comment on #1021 to add your EDIT, it's probably gonna help fixing the issue.

[oripwk]

@zoewangg

Sorry to nudge, but this is critical. If someone can look into it we'll appreciate.

[zoewangg]

Thank you for reporting. I'm working on the fix right now.

[zoewangg]

The fix is merged and will be included in the next release.

[zoewangg]

The fix has been released in 2.3.7. Could you try with latest version and verify?

Feel free to reopen if you have further question.

[oripwk]

We've deployed the fix 12 hours ago and there haven't been any NumberFormatExceptions since. It depends on traffic and it always happened sporadically so I'm not 100% sure. But so far looks good.

Thank you!

[aesteve]

Can confirm my integration tests with localstack are all green now.

Thanks for your fix.