Go: proposal: spec: add decimal float types (IEEE 754-2008)

[typeless]

I know there was a proposal about decimal as an std package.
But according to https://en.wikipedia.org/wiki/Decimal_floating_point, there have already been multiple compliant hardware implementations, and supported by several commonly used programming languages. I think having decimal as first-class citizen like the other part of the IEEE-754 standard makes sense.

The upside is we don't have to spend the effort on API design upfront to make it useful like math.big. We can just limit the scope of the operations only within the Go's existing arithmetic operators. Designing a good numerical/arithmetic API beyond the conventional operators is harder and can be deferred later with no harm.

[bradfitz]

Related: #12127, #19770

[bronze1man]

I think this proposal can be implement outside golang runtime, builtin and builtin package.
Does any implement exist?

[typeless]

@bronze1man Yes, there are thirty-party packages and proposals for including a decimal package into std. But this proposal is about adding fixed-size decimal floating point types, along with the existing binary floating point types using the existing arithmetic operators, rather than introducing a new package.

[deanveloper]

I made my own post for this without seeing this one first. Here are the reasons I believe that a decimal float type would be good for Go - #26699

[deanveloper]

Adding onto this - If we had operator overloading, this could be implemented as a 3rd party lib instead

[bradfitz]

Some useful information in:
http://open-std.org/JTC1/SC22/WG21/docs/papers/2014/n3871.html

In particular, this reference was useful:

The need for support of exact decimal computations is recognized in many communities and supported in several systems, although different alternatives for the support are chosen. Below is a list of example programming languages with decimal support:

• The C committee is working on a Decimal TR as TR 24732. The decimal support in C uses built-in types _Decimal32, _Decimal64, and _Decimal128.
• Java provides decimal arithmetic by java.math.BigDecimal, an arbitrary sized integer with an integer scale for the decimal places.
• Python provides decimal.Decimal which is a fixed point decimal representation. The number of decimal digits can be set globally.
• .Net provides System.Decimal which is a 128 bit decimal floating point. The details of this represntation are slightly different from the 128 bit decimal floating point in IEEE 754–2008. System.Decimal is accessible in C# as decimal.
• SQL provides a fixed point decimal representation where the number of digits and the number of fractional digits can be chosen for each context.
• Ruby provides BigDecimal, an arbitrary sized integer with an integer scale for the decimal places.

[ianlancetaylor]

It would help to have a clear idea of who would want to use these new types.

It would also help if the proposal spelled out the new types and how untyped constants would be handled.

[ericlagergren]

It would help to have a clear idea of who would want to use these new types.

From manually polling people who use my (big) decimal package:

• Banks
• CAD
• Numerical analysis
• Databases (see: cockroachdb/apd)
• E-commerce sites
• Bitcoin sites

[ianlancetaylor]

Thanks! But we shouldn't confuse big decimals with this proposal, which is for fixed size decimals. It's not obvious to me that people who use the former will want to use the latter. For example, the fact that people use the math/big Rat type does not suggest that we should add a rational number type to the language.

[ericlagergren]

I think there's more overlap than it would seem. A 128-bit decimal gives you 34 significant digits which is plenty for most regular usage, like wanting to perform accurate monetary arithmetic.

And, as Brad's link mentioned, four of the six listed languages have _only_ big decimals. So, I think it's somewhat germane :)

A point to consider (though I'm definitely in support of adding decimal type(s)): if decimals are added there will likely need to be support in the math/ package, like math and math/cmplx.

[djadala]

In this proposal, i expect only fixed size decimals, and i need them for financial computations.

[ianlancetaylor]

For people like me who know nothing, can you expand on why decimal floats are better for financial computations that 1) ordinary floats; 2) the int64 type? Just pointing to a paper would be great. Thanks.

[deanveloper]

Section 1.1 discusses a few reasons here - http://speleotrove.com/decimal/IEEE-cowlishaw-arith16.pdf

[ericlagergren]

That PDF is good. His FAQ sections is good as well: http://speleotrove.com/decimal/decifaq1.html#inexact

TL;DR: floats are inaccurate because they can't exactly store base 10 numbers.

Integers (representing the smallest currency amount, e.g. 1 cent USD) work well, but can be annoying to use for anything other than basic arithmetic.

And for non-monetary calculations integers don't work. E.g., http://speleotrove.com/decimal/decifaq4.html#order

[djadala]

In my company, we used all 3 alternatives:
decimal in c/c++
int64 in go, c/c++,
float in c/c++, perl

floats are not exact numbers, making some elementary calculations can give surprising ( for financial people) results.
just storing and loading float64 from database (oracle use some sort of decimals for numbers) can lead to inexact value, and some financial applications are 90% database io.
int64s mostly works, but are cumbersome (IMO), in all IO to external systems (including databases) you must convert them to "right" form.

hope this is useful

[alanfo]

As I can personally testify, the 128 bit decimal type is great for writing financial applications in C# which need to be accurate to the cent, penny etc. as you don't have to worry about the rounding errors which can accumulate if you use their float or double types. The results are always exact even for the largest amounts likely to be encountered in practice.

The alternative is to always work in cents using their 64 bit long type and then div/rem by 100 when you need to display or store the final result. However, this is both tedious and error prone and may even not be large enough when you're dealing with some currencies.

The twin drawbacks are the extra memory required and performance - decimal is much slower than double which in turn is slower than long. However, for most financial programmers this is a price worth paying for the convenience they gain from using decimal.

Back in pre-.NET days, VB6 etc. had a 64 bit Currency type which (IIRC) was scaled to 4 d.p. This was more than adequate for applications which didn't need to process large amounts of money and I always thought it was a mistake not to support it in .NET as well.

So, yes, if you'd like Go to appeal to financial programmers, I think that adding language support for fixed point decimal types would certainly be worthwhile and I'd suggest that 64 bit and 128 bit types (perhaps named dec64 and dec128 respectively) would be adequate as I can't see much point nowadays in having a 32 bit type as well.

As far as untyped constants are concerned, I'd suggest that they should work in a similar fashion to what they do now for the other basic numeric types. So literals would accommodate themselves to whatever type was required, there'd be conversions to/from the other numeric types and so on. However, no untyped numeric constant would have a default type of either dec64 or dec128. So for example, we'd have:

    a := 123.45              // a is float64
    b := dec128(123.45)      // b is dec128
    var c = dec64(a)         // c is dec64
    d := b + 6               // d is dec128
    var e dec64 = 678.9      // e is dec64

[typeless]

I believe most people can agree that there is a need for decimals. The question is whether the standard library should provide it or whether it should be a primitive type.
The shortcoming of making decimal as a third-party package is that libraries working with it are hard to cooperate. For example, a package provides finance calculations based on a particular decimal type would not be naturally composable with another decimal type belonging to a time series data store. Then, it would create a situation where a financial application might have to include multiple decimal packages, and programmers have to write conversion code between them, which is clumsy and inefficient.

[alanfo]

As @deanveloper indicated earlier, the lack of operator overloading in Go would also be a problem if decimal support were provided as a library rather than built into the language.

In fact, I suspect most folks would prefer to stick with what they currently do rather than having to deal with named methods for the basic arithmetic operations. Whilst it's true that you have to do this already for 'big' number arithmetic, the difference is that you have no choice if you need to deal with such numbers.

[vitesse-ftian]

Just some background from data/database people's point of view. Decimal64/128 is really a big help. As said in previous posts, dec32 is probably not interesting.

Unfortunately IEEE allow two different formats for decimal, BID and DPD -- I believe one is from Intel and the other from IBM -- therefore, it is necessary to decide on one format and maybe code conversion routines between these two. Both formats have high quality open source implementations.

Decimal64/128 is slower than double, but not too slow, we measured 2 to 3 x compared to double, which is quite impressive. Databases, for example, PostgreSQL, have to implement their own decimal/numerical type. In case of PostgreSQL, it is a variable length data type, supporting greater precision (but, decimal128 is enough for almost everybody), but much slower.

We need a native decimal64/128 type instead of cgo binding to intel/IBM library because cgo cost may just killed it.

[vitesse-ftian]

https://github.com/gcc-mirror/gcc/tree/master/libdecnumber

What GCC has.

[tc-hib]

Here, we're working with a proprietary language that is made for quickly developing business applications.

Today, I've helped a colleague fix a bug that was due to using binary floats instead of a builtin monetary type. The latter is an 80 bits fixed precision number with 6 decimal places, and it has always been sufficient for us.
My colleague only had to replace the type name everywhere and the bug was fixed.
It wouldn't have been that easy if the monetary type didn't have support for arithmetic operators.

I plan to present Go to my team, but I am a bit afraid of their reaction when I tell them how to write calculations with an external lib.

I think any serious business application should use decimal types. Maybe some developers don't realize that until an accountant asks them why their report doesn't show exactly same numbers as his Excel sheet.

A builtin dec128 would completely fulfill our need.
Of course, the ideal solution would be operator methods.

[tc-hib]

I have just stumbled upon this striking example:
https://play.golang.org/p/w-e7BDgo734

Trying to round 35.855 to 2 decimal places gives unexpected result.
Can be embarrassing in a business app.

For people like me who know nothing, can you expand on why decimal floats are better for financial computations that 1) ordinary floats; 2) the int64 type? Just pointing to a paper would be great. Thanks.

[ianlancetaylor]

@tc-hib Thanks. I understand that floating point numbers behave in ways that people find surprising (https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html). That's not the question I was trying to ask.

[tc-hib]

Oh, sorry I didn't get it.

Decimal type is a true silver bullet for all of us who don't need to save every cpu cycle, yet have to follow strict business rules. We simply use it everywhere. Once we've done that, it is really hard to go back, so we would always prefer translating our formulas to method calls rather than using ints or floats.

There are many legacy business apps (in COBOL for example, quite old), Excel sheets, etc. using decimal arithmetic as fluently as integers. I think Go would be a good fit for modernizing all those commerce specific apps thanks to its simplicity, but it would require "translating" every calculation carefully.

I believe using integers and floats is trickier than it seems, because we often have varying number of decimal places (3 for purchases, 2 for sales, and some intermediate calculations not rounded, such as taxes and commercial agreements). If we're only adding and multiplying numbers, or dividing by multiples of 5 and 2, users expect exact results. In fact, they expect same results as Excel sheet or pen and paper.

I must admit i don't have much experience in workarounds, because I've always been working with decimal types, and we nearly never use binary floats, as strange as it may seem.

[dunglas]

The upcoming "Structured Field Values for HTTP" RFC will have a decimal type. To support structured headers and trailers properly in the stdlib, support for decimal floating point will be mandatory.