Julia: Maybe deprecate BigFloat(f::FloatingPoint) constructor

[ivarne]

There has been a long of discussion in the julia-users group that started with a user that used the BigFloat(2.1) constructor and expected to get the same result as BigFloat("2.1"). The problem arrises because most floating point types in computers are stored as base2 fractions, but a programmer works in base10. A lot of base10 fractions can not be represented as a finite binary fraction, just like one third = 0.1(base3) = 0.33333333333333...(base10). When programmers forget this property of computers they get unexpected results.

As the current solution is confusing (for newcomers) and leads to very hard to find bugs, I want to propose some solutions and have it open for discussion on github (where it might be easier to find in the future than the High traffic julia-users list).

1. Change BigFloat(f::FloatingPoint) so that it gives the shortest base10 fraction that would convert back to the same f value. Easy implemented as BigFloat(f::Union(Float64,Float32,Float16)) = BigFloat(string(f)). This approach might be unexpected by experienced MPFR users and also lets programmers write BigFloat(2.1^256) which obviously will not give the desired result. See: https://github.com/JuliaLang/julia/pull/1015
2. Deprecate BigFloat(f::FloatingPoint) constructor so that it gives an error explaining floating point precision and suggest initialization by string. For those who needs the floating point constructor, and understand, it could be renamed to something somewhat scary so that it has a better chance of not being used wrong. (eg. BigFloatFromFloat64(), or convert(BigFloat, 2.1))
3. Somehow change how Julia treats numeric constants so that the BigFloat constructor can access the raw base10 representation.
4. Add syntax for construction of BigFloats and other special numbers. big"2.1" and or 2.1b0
5. Use the existing api and let those who won't read the documentation have programs that contains errors they tried hard to avoid by using BigFloat.
6. Add a (mandatory?) second parameter, ´BigFloat(f::Float64, signif::Int)´, where signif <= 0 gives the current behavior. Deprecation can be done by giving signif a magical negative value that triggers a deprecation warning. This approach could also be used for big.

[JeffBezanson]

(2) is the best option, allowing conversion from Float64 only via convert. (4) is also good. If there could be some syntax for bigfloat literals, it would be perfectly fine. Something like big"2.1" is possible.

The fact that floats are binary fractions is something you just have to get over. The fact is that float64(2.1) is mathematically equal to some number. The idea that when this number arises, the machine should somehow pretend it is a different number that the user perhaps intended is insane.

[StefanKarpinski]

I completely agree that it's insane to pretend that a number is a different number than it actually is. Hallucinating extra digits that aren't there is crazy town. I also really don't think that 2 is a reasonable option. Are we seriously going to make people write convert(BigFloat,1.2) instead of big(1.2) or BigFloat(1.2)? That's ridiculous and awful from a usability perspective. It's especially awful if you consider cases where the argument to be converted isn't a literal value.

I think that having big"1.2" is a good option. It allows a literal form for BigInts and BigFloats without making any language changes – all it requires is a @big_str macro that parses numbers and turns them into the corresponding BigInt or BigFloat at parse time.

There's also a fifth option that wasn't listed: (5) Document that this is an issue and move on.

[karatedog]

What about the Ruby BigDecimal way?

When initializing a BigDecimal with a Float, Ruby expects a precision argument or it throws an error:
⇒ irb 2.0.0p247 :001 > require 'bigdecimal' => true 2.0.0p247 :002 > BigDecimal(2.1) ArgumentError: can't omit precision for a Float. from (irb):2:in``BigDecimal' from (irb):2 from /home/karatedog/.rvm/rubies/ruby-2.0.0-p247/bin/irb:13:in '<main>' 2.0.0p247 :003 > BigDecimal(2.1,5) => #<BigDecimal:8a2d87c,'0.21E1',18(36)>

[JeffBezanson]

I would say that makes more sense for BigDecimal than for BigFloat.

[BobPortmann]

In addition to big"1.2" (or instead of) why not have a notation like 1.2b0 be a BigFloat (similar to how 1.2f0 is Float32).

[ivarne]

I have updated the Issue with your suggestions. I hope that is okay with you.

@StefanKarpinski What do you argue is insane in the two first sentences? The problem with all of convert(BigFloat,1.2), big(1.2) and BigFloat(1.2) is that they first "round off" 2.1 to the nearest Float64 value and then expand to 256 bit precision. I would say that it is the current behaviour that is "Hallucinating extra digits".

julia> BigFloat(0.1)
1.000000000000000055511151231257827021181583404541015625e-01 with 256 bits of precision
# When what I want is
julia> BigFloat("0.1")
1.000000000000000000000000000000000000000000000000000000000000000000000000000002e-01 with 256 bits of precision 

@karatedog I agree with Jeff that the Ruby BigDecimal approach makes more sense for Decimal types than Float. The number of decimal digits is not really relevant when you are going to store it as a binary fraction anyway. (see the 2 at the end of BigFloat("0.1"))

@BobPortmann That is a good suggestion, if they want to add more syntax. It will also be usable for BigInt if the decimal point is missing. Should we use a captial B?

I realized that the big() function also has the same problem. I am not sure what I think about that. It is a multiple dispatch function that have different return types for different arguments so I tend to think that it is less likely to be misused with Float64 literals.

[JeffBezanson]

The syntax big(1.2) is not a special case. It is just big(x) where x is a Float64.

[mschauer]

Rather than introducing a new notation for BigFloat which basically reproduces the problem on smaller scale (2.1 still not representable) one could think of introducing a floating point notation for (big) rationals. In the end, one wants the BigFloat representation of 21//10 = 2.1 (= "2.1d0"?), or?

[StefanKarpinski]

I'd really like to stop eating away at the space of valid numeric juxtaposition syntaxes. The more we do that, the less people who don't know all of the language by heart are going to feel comfortable with using juxtaposition and are just going to avoid it because it is "brittle" and sometimes just doesn't mean what you wanted it to. We already have 1e0, 1E0 and 1f0. If we're also going to have 1.2b0, etc. then we might as well just give up on juxtaposition syntax for coefficients. But I'd rather not do that. Another option would be doing something like 1.2_b or 1.2_f which is less likely to conflict. Then only the traditional 1e0 syntax would need to be grandfathered in.

[StefanKarpinski]

@ivarne – @JeffBezanson's statement that big(1.2) is not a special case is why this is insane in Julia. The semantics of the language mean that big(1.2) is no different than x = 1.2; big(x). If you want big(1.2) == big("1.2") you _are_ hallucinating binary digits – which are the real digits – it's only in decimal that it looks the other way around.

[andrioni]

I strongly support @StefanKarpinski's fifth alternative, document it and move on. I really see no point in trying to hide floating point arithmetic, because this is exactly the kind of behavior that creates these issues, not to mention that it makes reasoning about what your code is doing _much_ more difficult, and makes debugging rounding issues _much_ worse. Case in point:

julia> BigFloat(1.1 + 0.1)
1.20000000000000017763568394002504646778106689453125e+00 with 256 bits of precision

julia> BigFloat(string(1.1 + 0.1))
1.2000000000000002e+00 with 256 bits of precision

julia> BigFloat(string(with_rounding(RoundDown) do
       1.1 + 0.1
       end))
1.200000000000000000000000000000000000000000000000000000000000000000000000000007e+00 with 256 bits of precision

Also, please notice that doing BigFloat(string(1.1 + 0.1)) is actually less accurate than BigFloat(1.1+0.1).

[ivarne]

@mschauer - rationals have problems too, especially when combined with floating point representations of fractions that can't be represented accurately. There was a discussion somewhere if (1//13)+0.1 should be a float or a rational that exactly represent the floating point approximation. A Decimal type would be useful, or maybe a rational type where the denumerator is a type parameter.

@StefanKarpinski - So in short you were arguing against suggestion 1. (Sorry for putting it first, it is not my first choice). Making a special case for the BigFloat constructor (what I intended in 3), or other forms of syntax(4), is also a bigger issue for the design and consistency of the language.

@andrioni - you are just using the BigFloat(::Float64) to print Float64 numbers with more than the significant digits. The problem I want to avoid is that 1.1 + 0.1 is exactly the same calculation as BigFloat(1.1) + BigFloat(0.1), when you want to use BigFloat calculation to reduce the error. Printing insignificant digits for floats for debuging can be done in other ways (feature request?).

[ggggggggg]

Chiming in as someone likely to make this error I vote 2 followed by 5.

[timholy]

Documenting the problem is not antithetical to doing something more drastic, and the current paralysis does not serve any useful purpose, so I went ahead and pushed a documentation change in 9a691d054ff01dbc51947abbfd960ae7e2615fdb.

I'm not convinced that option 5 2 will really solve anything; what if users don't discover the string syntax and just use convert for everything? If we really want to solve this problem, perhaps we need a "mode-changing macro" (similar to how @inbounds works), in this case turning on a different mode for parsing numeric literals and wrapping pre-existing variables. For example,

@bigfloat y = exp(x) - 1

to perform numerically-delicate calculations in BigFloat precision. This is not a great example for several reasons, but I chose it for its familiarity. And in case anyone gets excited about this, let me caution that there are non-obvious decisions to make about what type y should have on output. (For this calculation, I personally would want y to have the same type as x, but someone else would surely want their carefully-computed pi to keep its BigFloat precision.)

But I presume this would be a huge amount of work for something that seems like a pretty small problem.

[Ned-Nowotny]

I support deprecating the BigFloat(f::FloatingPoint) constructor. It is rare that a BigFloat is desired once a number has already been converted via translation or computation into FloatingPoint number. And for naive programmers, the result will often be "surprising."

However, it can be very convenient to provide a decimal notation which can be checked and translated by the compiler into a rational or arbitrary precision decimal number. Using strings means parsing literals at runtime with errors only detected when the code containing a decimal number encoded as a string is finally executed -- hopefully no later than unit testing...

In fact, creating initialized arrays and matrices for unit tests themselves can benefit from the notational convenience of rational or arbitrary precision literal notation. For one not entirely artificial example, consider an array of market share values which must arithmetically sum to 1.0 as input to, say, a markov chain analysis. I would prefer:

mktShare = [ r0.8, r0.1, r0.1 ]

...to either an array of string processed into an array of rational number or an array of expressions which construct rational numbers form string parameters.

Though, to be honest, I would prefer even more to just write:

mktShare = [ 0.8, 0.1, 0.1 ]

...using unadorned decimal notation with some other means used to tell the compiler that numeric literals in "this block," however identified, are to be parsed into rational numbers or arbitrary precision decimal numbers instead of IEEE floating point representation.

[pao]

mktShare = [ r0.8, r0.1, r0.1 ]

For whatever it's worth, we do have a Rational type:

mktShare = [ 8//10, 1//10, 1//10 ]

[Ned-Nowotny]

@pao I know. That is one of the many things I like about Julia and your code example is what I would use absent some means to express rational numbers using a decimal notation.

However, a decimal notation may be a clearer expression of some values even when floating-point operations are not appropriate. Granted, it is a minor nit in a world where I have to use far more verbose syntax and cumbersome run-time solutions in other languages. Still, all numeric literals in computing can be represented internally as rational numbers and absent a literal syntax for a single rational number expressed as a ratio such as that provided by Julia, all numeric literals are finite rational numbers. Therefore, it does not seem unreasonable to want to use a common decimal notation for all numeric literals, if possible -- obviously, the decimal point and fractional digits are not appropriate for integers. But I do get that "reasonable to want" is not the same as "reasonable to implement" in either syntax or a lexical scanner...

[nalimilan]

The BigFloat constructor could require a second argument giving the number of significant digits to use.

[ivarne]

Thanks @nalimilan. I added (a hopefully improved version of) that suggestion as #6. I wonder what the others think about this.

[pao]

using unadorned decimal notation with some other means used to tell the compiler that numeric literals in "this block," however identified, are to be parsed into rational numbers or arbitrary precision decimal numbers

This is possible too without parser changes by using a macro which delimits a block and transforms numeric literals into the desired forms.

[pao]

This is possible too without parser changes...

Scratch that, the AST will have the interpreted literal in it. Never mind.

[Ned-Nowotny]

From Overloading Haskell numbers, part 3, Fixed Precision, it appears that Haskell provides for treating decimal numbers with a fractional part as rational numbers rather than floating-point literals in some cases. At least, that is what I gather from a quick reading of Haskell documentation and that blog post in which the author states:

...notice that what looks like a floating point literal is actually a rational number; one of the very clever decisions in the original Haskell design.

I realize that is Haskell -- the sometimes impenetrable -- and here we are discussing Julia -- the generally comprehensible -- but it seems to offer an example in the wild of what I am looking for.

Note, my interest is to be able to provide a literal number in familiar decimal notation, including a fractional part, and have it converted to a rational number without having already been converted to an IEEE floating-point representation in which precision may have been lost. Admittedly, not a high priority kind of request, but it does seem like something the compiler could manage, perhaps by delaying the conversion of the literal token into a machine or arbitrary precision rational number until the expression types have been determined by the parser.

[JeffBezanson]

I agree that the ideal behavior would be to keep numbers exactly as written, until they are forced to some other particular type. Unfortunately this is hard to do in julia, since unlike Haskell our expression contexts do not have types. A literal needs to have a specific type; if a function only accepts Float64, and the literal type is not Float64, you will get a no-method error.

[Ned-Nowotny]

@JeffBezanson I understand, but just a thought: What if numeric literals were not Float64 by default, but were instead a rational number until converted to a floating-point representation if required? All number literals are rational numbers -- though some may have a numerator or denominator which require an arbitrary precision number to correctly represent it. Integers, trivially so, but any floating-point literal representation which only allows an integer exponent -- every example of which I am aware -- is also rational. In that case, no precision is potentially lost until a conversion occurs and that could still be performed by the compiler under some circumstances. This might even be a transparent change to the language...or perhaps not...dunno...

[StefanKarpinski]

That's the trouble. This cannot be done transparently in Julia. As Jeff said, if a floating-point literal is not a Float64 then there will have to be methods for that type that know to convert the FloatLiteral type to Float64. That means this change would necessitate new methods for the vast majority of functions in Base. And it's not just the ones in base – it's even worse that this affects user-defined functions. Suppose you have this definition:

f(x::Float64, y::Float64) = singnificand(x)*2.0^exponent(y)

Under the scheme you're talking about, you cannot call this function as f(1.2, 3.4) – because 1.2 and 3.4 are not of type Float64, they're of type FloatLiteral. Rather, you'd have to do f(float(1.2),float(3.4)) or equivalently, add a method for this:

f(x::FloatLiteral, y::FloatLiteral) = f(float(x),float(y))

Now you can argue that you could just allow the original definition to apply to Union(Float64,FloatLiteral), but now you're forced to use these awkward union types everywhere.

[Ned-Nowotny]

@StefanKarpinski I see. Then no, it does not seem remotely practical to treat float literals as rational numbers. Thank you for explaining.

[filmackay]

+1 for the objective of leaving literals "exactly as written". 3.4 should be a decimal/rational, with 3.4e0 being the way of avoiding float(3.4)'s everywhere?

I found this thread when researching a Decimal type. Is there one proposed for Base?

[nalimilan]

@StefanKarpinski Julia would have to keep it internally as a rational until it is used somewhere by the code, in which case it would be converted silently depending on whether a Float or a Rational is expected. I realize this probably is not super practical implementation-wise.

[andrioni]

Not to mention that decimal fixed-point implementations tend to be even more problematic than floating-point, and rational arithmetic can get pretty expensive pretty fast.

[rominf]

I vote for deprecation too. I think that using strings (BigFloat("0.1") or big"0.1") in math code is ugly. IMHO, @BigFloat 0.1 is a better alternative (but it's still ugly).

macro BigFloat(x)
    :(BigFloat($(string(x))))
end

[ivarne]

@rominf Your macro only works for floats with less than ~16 significant decimal digits. The parser will truncate the remaining digits.

[rominf]

@ivarne Hmm, I didn't know that.

[jiridanek]

@nalimilan I believe it could be made practical. Go has arbitrary precision compile time constants that get converted to Int or Float if they are used in a non-compile-constant context. I want to propose doing something similar.

Constant expressions are always evaluated exactly; intermediate values and the constants themselves may require precision significantly larger than supported by any predeclared type in the language. The following are legal declarations […] —https://golang.org/ref/spec#Constant_expressions

If a function is called with a FloatLiteral argument at position where it explicitly declares a FloatLiteral argument, it will get FloatLiteral, if it does not declare any type or declares a Float64 type, it will get Float64.

In line with this, I suggest making FloatLiteral a second-class citizen. It cannot be assigned into a variable (that would convert it into Float64), it is converted to Float64 before being used in a non-literal expression (0.5 * 0.6 is still FloatLiteral, 0.5 * a for some variable a first converts 0.5 to Float64). It can pretty much be only passed as an argument to a function (BigFloat constructor). One trick pony.

[nalimilan]

Sounds interesting. The problem is that in Julia the distinction compile time vs. run time isn't as clear as in Go, so you'll necessarily expose that FloatLiteral type outside of the compiler, and currently there's no mechanism to make objects of this type "disappear" as soon as you touch it. Maybe something like that ("automatic conversion"?) would also be useful for MathConst.

[StefanKarpinski]

I don't think that introducing second-class, compile-time-only types is a satisfactory solution – it just complicated matters further. In Go, for example, you get completely different behavior if you operate on a literal than if you assign that literal to a variable and do the same operation on it, which, of course, confuses people. Instead of just explaining to people that computers binary instead of decimal – which is something they're going to need to know about to do numerical computing effectively – you _also_ have to explain that there's this subtle difference between literal numbers runtime numbers. So now people have two confusing things to learn about instead of just one.

[jiridanek]

I am using Julia only as "faster Octave", so I surely don't see all the consequences for the language. I just realized my example with 0.5 * 0.6 staying a FloatLiteral is very problematic because the programmer might want to redefine operators (something that Go does not allow, and for good reasons, while Julia wants that, and for good reasons too).

If a FloatLiteral would always collapse into a Float64 every time it is touched except when it is passed as a function parameter or treated as a string, the whole thing provides only syntactic convenience over passing in strings. Then it is probably not worth having. On the other hand, it does not create the semantic difficulties as in Go (because then it has no semantics).

What about attaching a string property to every Float64 that would contain the token in the source code that led to creation of that number? Possibly keeping the property around only "for a short time" (the same lifetime as my version of FloatLiteral was supposed to have)? Or storing it only if the string cannot be "losslessly" recovered from the binary representation of the number, otherwise computing it when needed? (suggestion # 3)

I like suggestion # 6 the most, it would IMO work well and it does not require wild changes like # 3.

[simonbyrne]

I originally thought a float literal type could be a good idea, but I have come to change my mind: it would be really confusing to have

x = 0.1; y = BigFloat(x)
y = BigFloat(0.1)

do different things.

I think this problem could be alleviated somewhat by having a separate syntax for nonstandard numeric literals: I think postfix underscores could be nice:

y = 0.1_BigFloat

[simonbyrne]

On a related note, one thing I would like to have is a print_full which displays the full decimal expansion of a floating point number, e.g.

julia> print_full(0.1)
0.1000000000000000055511151231257827021181583404541015625

At the very least, it would be great for teaching floating point....

[StefanKarpinski]

I'm starting to feel like we should ditch numeric literal juxtaposition and use <number><identifier> more generally for different kinds of number inputs. If this invoked a macro in the general case, then we could continue to have im and unit input syntax work with the appropriate macro definitions. What we would give up generally is using that syntax for random local variables, but we would retain that kind of syntax for globally defined syntaxes. I also think it would be nice to allow spaces: <number> <identifier>.

[simonbyrne]

I'm starting to feel like we should ditch numeric literal juxtaposition and use more generally for different kinds of number inputs.

I, for one, would be willing to sacrifice implicit multiplication for this.

[johnmyleswhite]

+1 to that. I never use implicit multiplication in practice.

[mikewl]

On implicit multiplication, isn't number.variable close enough? Though it currently only works with integers. Not sure how difficult it would be to extend that to other numerical types.

[jiahao]

@Mike43110 "4.x" is parsed as "4.0 * x" and does not always preserve the semantics of integer operations. A simple counterexample:

julia> x=1;

julia> 10_000_000_000_000_001x == 10_000_000_000_000_000x #integer arithmetic
false

julia> 10_000_000_000_000_001.x == 10_000_000_000_000_000.x #floating point arithmetic
true

[oscardssmith]

Bump with new idea: What would happen if we just deprecated creating BigFloats from floats? If we forced them to be made from int's, rationals, or other more exact types, we would get rid of all cases with unexpected behavior. This does seem fairly radical, but I'm not sure it's a bad idea.

[simonbyrne]

I frequently create bigfloats from floats, for checking the precision loss of computations. In fact that's the main thing I use them for.

[oscardssmith]

Could there at least be a warning? It seems like a lot of hard to catch errors could result form expressions like BigFloat(1/3), which really look like construction of a value rather than a conversion.

[stevengj]

It seems like this issue could be closed? I think the clear consensus of core devs here is that the current behavior of BigFloat(::AbstractFloat) is correct and desirable, as well as being locked in for Julia 1.x.

As I commented on discourse, my @changeprecision BigFloat macro already basically does what people want here — it allows you to write ordinary floating-point literals (including floating-point rationals like 1/3) and change them en masse in a large code block to the desired BigFloat value, thanks in part to the grisu algorithm which allows us to recover the "exact" form in which the literal was entered.

In principle, we could use the grisu trick in the BigFloat(::AbstractFloat) constructor as well:

julia> mybig(x::AbstractFloat) = parse(BigFloat, string(x))
mybig (generic function with 1 method)

julia> mybig(2.1) == big"2.1"
true

though this is somewhat expensive and I think big"2.1" is a clearer way of writing BigFloat literals anyway.

[StefanKarpinski]

Even if people find it confusing, I don't think having BigFloat(x::Float64) do the equivalent of parse(BigFloat, string(x)) is really appropriate since converting a float value to BigFloat has a correct and precise meaning which is what we're doing now.