Azure-sdk-for-java: CosmosAsyncItemResponse::getRequestCharge() fails on international systems

[tomekapc]

Describe the bug
getRequestCharge() will fail on systems where a "," is used as a decimal separator.

Exception or Stack Trace
java.lang.NumberFormatException: For input string: "3,64"

To Reproduce
Steps to reproduce the behavior:
Set your locale to German and run this example code: SampleCRUDQuickstartAsync.java
The method createFamilies() will fail.

Setup (please complete the following information):

• OS: Windows
• Java SDK v4 preview.2

Additional context
The machine used in this test has German locale defined and uses a "," as a decimal separator.

The source of this issue seems to be in the implementation of populatePartitionedQueryMetrics in DocumentProducer.java
in this line here

The current locale will be used to produce the string and on some systems a "," will be used as a decimal separator. The locale won't be used by Double.parseDouble() (actually, why is the locale ignored by Double..?) when parsing the string which in the end leads to the NumberFormatException mentioned above.

Possible fix

1. set the locale and enforce a "." as a decimal separator:
   String.format(Locale.US, "%f", floatValue)
   or
2. don't use Double.parseDouble() and use NumberFormat etc. instead (?)

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [x] Bug Description Added
• [x] Repro Steps Added
• [x] Setup information Added

[joshfree]

Thanks for reporting this issue with the v4 preview 2 build of Cosmos @tomekapc. Someone from the Cosmos team will follow up shortly.

/cc @kushagraThapar @anuchandy

[kushagraThapar]

@tomekapc I will pick it up in next release. Thanks for reporting this.

[jongio]

@kushagraThapar - Do you have an update on this? It is currently 155 days old. Thanks, Jon

[kushagraThapar]

@jongio - apologies on delayed response, we couldn't pick this.

@tomekapc - I looked at the code you have mentioned above, and I can't see the Double.parseDouble, we are using String.format, which internally uses the formatter with default Locale.

[jongio]

@kushagraThapar - Are you suggesting that they need to add Double.parseDouble?

[kushagraThapar]

@jongio - we are investigating this issue.

@mbhaskar - can you please provide an update with your investigation?

[mbhaskar]

@jongio Thanks for your patience. I am able to reproduce the issue, will prioritize this in the future release and update you once ready.

[pkonyves]

What is the workaround? I am trying with 4.3.2-beta.2 on a hungarian local windows machine.
When I debug QueryMetricUtils#parseDelimitedString I see the input:

totalExecutionTimeInMs=1.20;queryCompileTimeInMs=0.14;queryLogicalPlanBuildTimeInMs=0.05;queryPhysicalPlanBuildTimeInMs=0.10;queryOptimizationTimeInMs=0.01;VMExecutionTimeInMs=0.59;indexLookupTimeInMs=0.43;documentLoadTimeInMs=0.08;systemFunctionExecuteTimeInMs=0.02;userFunctionExecuteTimeInMs=0.00;retrievedDocumentCount=10;retrievedDocumentSize=4955;outputDocumentCount=10;outputDocumentSize=5014;writeOutputTimeInMs=0.02;indexUtilizationRatio=1.00;requestCharge=3,35

I even don't understand how this can be that only the requestCharge is comma delimited.
Is the locale sent to the HTTP Request by the SDK, and the API respond with localization enabled?

edit:
I found _a_ solution:
DocumentProducer#populatePartitionedQueryMetrics

replace String.format(";%s=%.2f", "requestCharge", this.pageResult.getRequestCharge());
with: String.format(Locale.US, ";%s=%.2f", "requestCharge", this.pageResult.getRequestCharge());

Stacktrace:

java.lang.NumberFormatException: For input string: "3,35"
    at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054)
    at java.base/jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
    at java.base/java.lang.Double.parseDouble(Double.java:543)
    at com.azure.cosmos.implementation.QueryMetricsUtils.parseDelimitedString(QueryMetricsUtils.java:36)
    at com.azure.cosmos.implementation.QueryMetrics.createFromDelimitedStringAndClientSideMetrics(QueryMetrics.java:251)
    at com.azure.cosmos.BridgeInternal.createQueryMetricsFromDelimitedStringAndClientSideMetrics(BridgeInternal.java:215)
    at com.azure.cosmos.implementation.query.DocumentProducer$DocumentProducerFeedResponse.populatePartitionedQueryMetrics(DocumentProducer.java:72)
    at com.azure.cosmos.implementation.query.DocumentProducer$DocumentProducerFeedResponse.<init>(DocumentProducer.java:56)
    at com.azure.cosmos.implementation.query.DocumentProducer.lambda$produceAsync$4(DocumentProducer.java:176)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:100)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
    at reactor.core.publisher.FluxMergeSequential$MergeSequentialMain.drain(FluxMergeSequential.java:425)
    at reactor.core.publisher.FluxMergeSequential$MergeSequentialMain.innerNext(FluxMergeSequential.java:297)
    at reactor.core.publisher.FluxMergeSequential$MergeSequentialInner.onNext(FluxMergeSequential.java:563)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
    at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
    at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:162)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1782)
    at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:171)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:144)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1783)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
    at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:173)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1782)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1782)
    at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:171)
    at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onComplete(FluxPeekFuseable.java:270)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1783)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1782)
    at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:171)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.checkTerminated(FluxFlatMap.java:838)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:600)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.innerComplete(FluxFlatMap.java:909)
    at reactor.core.publisher.FluxFlatMap$FlatMapInner.onComplete(FluxFlatMap.java:1013)
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:78)
    at reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:191)
    at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onComplete(MonoIgnoreElements.java:81)
    at reactor.core.publisher.FluxTakeUntil$TakeUntilPredicateSubscriber.onComplete(FluxTakeUntil.java:114)
    at reactor.core.publisher.FluxTakeUntil$TakeUntilPredicateSubscriber.onNext(FluxTakeUntil.java:92)
    at reactor.core.publisher.FluxRepeatPredicate$RepeatPredicateSubscriber.onNext(FluxRepeatPredicate.java:79)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1782)
    at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:121)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.checkTerminated(FluxFlatMap.java:838)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:600)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.innerComplete(FluxFlatMap.java:909)
    at reactor.core.publisher.FluxFlatMap$FlatMapInner.onComplete(FluxFlatMap.java:1013)
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:2016)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.checkTerminated(FluxFlatMap.java:838)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:600)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.drain(FluxFlatMap.java:580)
    at reactor.core.publisher.FluxFlatMap$FlatMapMain.onComplete(FluxFlatMap.java:457)
    at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:138)
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:2016)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1783)
    at reactor.core.publisher.MonoCompletionStage.lambda$subscribe$0(MonoCompletionStage.java:86)
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
    at com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdRequestManager.messageReceived(RntbdRequestManager.java:712)
    at com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdRequestManager.channelRead(RntbdRequestManager.java:174)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1526)
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1275)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1322)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
Cosmos getStarted failed with java.lang.NumberFormatException: For input string: "3,35"

[pkonyves]

Workaround:
Use Locale.setDefault(Locale.ENGLISH); in you application.