Runtime: Respect the precision specifier when formatting floats/doubles using 'E' or 'G'

[tannergooding]

This was broken out of https://github.com/dotnet/coreclr/issues/19802

Currently when formatting a Single/Double using the "G"/"E" specifiers, we clamp the precision specifier to 17. We should remove this limitation and instead return the number of digits the user requested.

[tannergooding]

We do currently respect the precision specifier, up to 99 for F and other supported format specifiers.

[tannergooding]

It may be interesting to consider a mechanism that would allow users to request more than 99 digits (the current format specifier only allows two digit precision specifiers).

The longest roundtrippable string for a double-precision floating-point value is only 17 digits, but the longest exact value is 768 significant digits (1075 digits if you include the leading zeros). There are other programming languages, such as Python, that do allow you to return the exact string. Additionally, our existing formatting algorithms support printing as many digits as requested (provided the destination buffer is large enough).

I think that updating the existing format specifier to support more than two digit precision specifiers may be a breaking change, so adding a new API could be an alternative option if we decided this is something worth enabling (given that it is trivial to support, after we determine the shape, and sometimes useful).

[juliusfriedman]

Yes, I would agree and type that it is much more trivial to support a compat flag than totally disregard for compat altogether! With that being typed I hope you would also agree that the design should allow the underlying implementation to be switched out for whatever reason and also the ability to specify the algorithm in the format... hence the push for the allowable change in algorithm used. :)

[tannergooding]

also the ability to specify the algorithm in the format

Not sure how this would be applicable. The IEEE 754 specification only has one correct answer for any given number of digits. That is, if the user requests 'X' digits, the string returned is deterministic and there is only one correct result that should be returned by any given implementation (if the algorithm returns a different result, it is incorrect).

The only thing a different algorithm should change is how efficiently it does the parsing/formatting, and we should always aim to use the best algorithm possible, overall.

[juliusfriedman]

Does that matter? Yea it complicates the design but that's life...

Who's to say that's the best algorithm and what about users who have other uses besides ieee e.g. we just now are concerned with compliance so I'm not sure what your point is? That you don't understand?

Regardless, specifying the algorithm not only in the format but also at the system level is useful when it is regardless if it can be changed at whim...

Who's also not to say there is not a better algorithm for a different task then you are performing ...

[juliusfriedman]

On a more succinct note, is it really much difference of having a compat flag which is then later enhanced to accept the algorithm and other parameters... only in terms of Implementation... hence where you would want to change this and yes I will concede that you would want the change at the system level in general however general things are... you may encounter a situation where process based configurations are required for cost or other concerns let alone security et al...

[tannergooding]

we just now are concerned with compliance

It is not that we are just now concerned with compliance, it is that we now have the opportunity to fix a backlog of customer bugs that have existed for a number of years that we were not otherwise able to easily resolve (much of this was addressed in the blog post here: https://blogs.msdn.microsoft.com/dotnet/2018/10/04/update-on-net-core-3-0-and-net-framework-4-8/, which discusses how full framework needs to remain compatible but .NET Core, due in part to it's SxS nature, can better take fixes like this).

We have been slowly working through the backlog for float/double in the past few releases and this is just tracking one of the few remaining corner cases we know about.

what about users who have other uses besides ieee

The runtime explicitly specs System.Single and System.Double out to be the IEEE 754 types (and has since its original version 20-odd years ago). For the most part, the core operations have always been IEEE compliant, but we have had some longstanding issues around various edge-cases when dealing with parsing/formatting (as did many other languages at the time; many of which were only fixed after the 2008 version of the IEEE 754:2008 spec was published and/or when the C11 language spec was updated with the new rules).

If users have a use-case outside of this, they should be using a custom data type to ensure they get the behavior they desire. They could also open a proposal suggesting we add a new data-type that better suits their specific needs.

• This is why, for example, we have types like System.Decimal which are more suited to applications dealing with monetary values; and why IEEE defines their own decimal floating-point types (which we don't currently expose) that don't have the same drawbacks as the binary floating-point types (float/double)

Who's also not to say there is not a better algorithm for a different task then you are performing ...

For specific use-cases there may be better algorithms/implementations; but that is true of anything in the framework. For example, System.Collections.Generic.List<T> may not always be the "best" list implementation. But, because it is a "core" type in the framework, we try to ensure it is a good implementation for the most common use-cases.

When users have specific scenarios (with backing hot-spot/profiling data) indicating that a particular thing is the bottleneck; then they can provide their own implementation or consume an existing 3rd party implementation (there are plenty on NuGet, for example) that explicitly suits their needs. If they have an optimization that will benefit a particular code-path (without being a detriment to other commonly used code-paths) in the existing implementation or if they have a better implementation they are free to open an issue suggesting we make the change (and, if we decide its something we would want to take, they are also more than welcome to submit said changes in a PR).

[juliusfriedman]

Thanks for the lengthy explanation. In short why don't you agree that the compat flag support which is already conceded as needed and also the fact that is also conceded that the method would not work with both implementations does not matter?

I'm merely thinking when adding such support to please do so in a isolated and considerate way such that I would also be able to easily switch out the support therein.

[tannergooding]

A compat flag is not automatically added for every breaking change. Instead, we will consider adding a compat flag that allows users to fallback to the old behavior if there are enough users that are dependent on the old behavior. Compat flags are also generally not added to the public surface area of an API. Instead, they are enabled via configuration switches that are consumed in a "well-defined" way (someone else may be able to provide more details on how this works for netcore; on desktop you would often opt-in to the quirks via the app.config file).

For cases like this, I wouldn't expect many users to be dependent on requesting 20 digits but only having 17 returned.

[juliusfriedman]

And then again for 99 or is that in the 7th version :)

[juliusfriedman]

And that's not mention the code or other measures I probably had to put in place to accomadate.

[tannergooding]

And that's not mention the code or other measures I probably had to put in place to accomadate

Could you please elaborate and give examples of how you believe this will break you?

[juliusfriedman]

Specifically I have not and me.. it wont... but would imagine such unsafe code or otherwise to already exist... and to those matters I will not speak. Again not mentioning even the measures put in place to support compat...

In closing. I feel strongly both implementations must reveal the underlying implementation to which they comply regardless of any compatibility flags out in place.

[danmosemsft]

@tannergooding marked Future?

[tannergooding]

@danmosemsft, I think this is one that we should consider fixing for 3.0.

As it is, the UTF8Formatter does not support all the same format specifiers as the UTF16Formatter and there is no way to get an explicit number of digits for the most common formats (such as G17).

The overall fix should be generally trivial, since we are just passing this down to the UTF16 formatter anyways.

[danmosemsft]

Sounds good you're the area owner feel free to change.