Runtime: Formating of double to string using G29 format sometimes returns wrong result

[Petermarcu]

@Eneuman commented on Fri Mar 23 2018

Issue Title

Formating of double to string using G29 format sometimes returns wrong result.

General

Using .Net core 2.0 latest, Windows 10 Pro, Swedish Version

Running this

      var test1 = double.Parse("1,5").ToString("G29");
      var test2 = double.Parse("1,6").ToString("G29");
      var test3 = double.Parse("1,7").ToString("G29");

results in this:

test1 = "1,5"
test2 = "1,6000000000000001"
test3 = "1,7"

I was expecting this:

test1 = "1,5"
test2 = "1,6"
test3 = "1,7"

[Clockwork-Muse]

Assuming this code is accurate, the real values are:

test1 = "1,5"
test2 = "1,600000000000000088817841970012523233890533447265625"
test3 = "1,6999999999999999555910790149937383830547332763671875"

...and those 8s start at index 18. It looks like it's being rounded up, which would be expected (try it with a precision of 1, all of them get 2).

Given the documentation for the G format string says that "17 digits round-trips", my guess is that anything > 29 is being rounded to 17.

[Eneuman]

The usecase I’m working on is the following:
A user wants to calculate the correct dosage of medication that shoud be prescribed. The user enters numbers into text boxes that are later used in calculations. It’s important that the number entered is exactly the same even after the convert. Maby I shouldn’t use double but another type instead?
The scale and precision of the number entered in the text box depends on what type of medication that is prescribed.

[fanoI]

Use decimal to avoid double troubles...

[Eneuman]

Will the same conversion behaviour occur if I do string -> decimal -> double?
Decimal is not a valid type in Swagger Api 2.0 definition so it gets converted to double automatically :(
In Open API 3.0 this has been fixed.

[Clockwork-Muse]

A user wants to calculate the correct dosage of medication that shoud be prescribed. The user enters numbers into text boxes that are later used in calculations. It’s important that the number entered is exactly the same even after the convert. Maby I shouldn’t use double but another type instead?
The scale and precision of the number entered in the text box depends on what type of medication that is prescribed.

.... eh... it's complicated.

On the one hand, having a specific precision and decimal floating point is more accurate from a "record" view. On the other hand, since the numbers here are likely a _measurement_, floating point is probably more correct, in the sense that minor errors are unimportant to the precision.

Calculations complicate things even more, because results may not completely roundtrip, especially with decimal floating point. Simple example: I have 1g of a medicine, enough for 3 doses, and somebody buys them. I package them up as 3 separate doses, each of 333mg; where'd the last 1mg go? You need to figure out how you want to deal with rounding and precision.

Will the same conversion behaviour occur if I do string -> decimal -> double?

Yes. 1.6 (nor 1.7, for that matter) cannot be represented exactly in binary floating point, which is what double/single really are. You'll want to read up on this.

Decimal is not a valid type in Swagger Api 2.0 definition so it gets converted to double automatically :(
In Open API 3.0 this has been fixed.

If you have no format, it'll scan as decimal. But I'm not sure what the behavior is if you mark it as double in swagger and your POCO uses decimal instead - or potentially a custom converter - which may just do the direct string<->decimal conversion, skipping binary floating point entirely. That would avoid funky string output, but would potentially leave calculation-related issues (which aren't gone when using double, they're just different).

[eerhardt]

See also the excellent answer by @tannergooding here: https://github.com/dotnet/corefx/issues/19521#issuecomment-302486502.

This is how IEEE floating points behave, and the behavior is the same as far back as .NET Framework 2.0. (I couldn't test any further back on this machine.)

Closing as this behavior can't be changed and I believe the question has been answered.