Julia: allow the user to check overflow in all aritmethic operations

[RicardoHS]

I started playing with Julia just two days ago and I love in what this language could become. Nevertheless it shocks me the fact that overflow is silently occurring. I have check #10802, read the FAQ and even the Dan Luu post.

In the manifesto(?) it is stated that a simple language for general purpose (like python) is wanted. Well, this is impossible without safe arithmetic operations. If we take python as the inspiring language for general purpose, we have to embrace his zen, or at least part of it. Concretely:

Errors should never pass silently.

I understand that checking if an overflow has occur in every arithmetic operation slow down significantly the language, but if it stated that Julia wants to be a general purpose language this is needed to be implemented.

Maybe I'm used to get an exception or a warning if an overflow occurs, because normally I programing in a general way. Precisely because I programing in that way (and IMHO I think the vast majority of people code with this philosophy in mind too) it shocks me when something like this happens.

I was playing with markov chains when...

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]
3×3 Array{Float64,2}:
 0.6  0.4  0.0
 0.0  0.2  0.8
 0.5  0.0  0.5

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000
3×3 Array{Float64,2}:
 0.434783  0.217391  0.347826
 0.434783  0.217391  0.347826
 0.434783  0.217391  0.347826

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000000000
3×3 Array{Float64,2}:
 0.434784  0.217392  0.347827
 0.434784  0.217392  0.347827
 0.434784  0.217392  0.347827

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^1000000000000
3×3 Array{Float64,2}:
 0.434796  0.217398  0.347837
 0.434796  0.217398  0.347837
 0.434796  0.217398  0.347837

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^10000000000000
3×3 Array{Float64,2}:
 0.43492  0.21746  0.347936
 0.43492  0.21746  0.347936
 0.43492  0.21746  0.347936

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000000000000
3×3 Array{Float64,2}:
 0.436156  0.218078  0.348925
 0.436156  0.218078  0.348925
 0.436156  0.218078  0.348925

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^1000000000000000
3×3 Array{Float64,2}:
 0.448716  0.224358  0.358973
 0.448716  0.224358  0.358973
 0.448716  0.224358  0.358973

julia> [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^10000000000000000000000000000
3×3 Array{Float64,2}:
 Inf  Inf  Inf
 Inf  Inf  Inf
 Inf  Inf  Inf

julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

As some of you know, the long run power of a markov chain matrix cannot be infinite. I'm aware of this, and because of that, I could notice that the inf matrix is wrong. Someone that was simply playing with maths could not notice this. Or even worst, some algorithms could lead (and will lead) to unexpected behavior without even notice it.

I know that Julia wants to be a lot of things, as the manifesto(?) said, but this is impossible without a trade off. So here is my proposal:

Implement the safe arithmetic operations and allow the user to enable of disable it like many other options (like setting the random seed). The decision if this option needs to be enable by default or not would depend in which kind of language wants to be julia first. A general one or a fast one.

[KristofferC]

I believe the current thinking is to use a custom numeric type (like https://github.com/JeffreySarnoff/SaferIntegers.jl) which has these checks.

[RicardoHS]

I just notice it right now, and there is also a conversation of this in #855

But I dont think this has to be treated in an external library. It looks like, unsafe...

[JeffBezanson]

but if it stated that Julia wants to be a general purpose language this is needed to be implemented.

I guess C, C++, and Java are not "general purpose".

Maybe I'm used to get an exception or a warning if an overflow occurs, because normally I programing in a general way

In what language? Python, ruby, and lisps automatically switch to bigints instead of giving an exception.

Your example uses floating point. With floating point, overflowing to Inf is the behavior defined by the IEEE spec. Not doing that would be wildly surprising to anybody writing numerical code, and would make it unnecessarily difficult to port floating point code between languages. This is simply not a serious suggestion.

Catching integer overflows could be nice in some cases. It would make sense to have a compiler switch to turn on overflow checking, but for that we would first need to somehow rewrite calls that are intended to overflow, e.g. the famous (a + b) >>> 2 expression.

[JeffBezanson]

Also, python isn't general purpose either:

In [4]: x=np.array([[0.6,0.4,0],[0,0.2,0.8],[0.5,0,0.5]])

In [6]: np.linalg.matrix_power(x, 10000000000000000000000000000)
Out[6]: 
array([[ inf,  inf,  inf],
       [ inf,  inf,  inf],
       [ inf,  inf,  inf]])

[RicardoHS]

I guess C, C++, and Java are not "general purpose".

Despite the fact that those languages were initially designed as general purpose, nowadays C/C++ are used to write programs with strong computational efficiency requirements. One does not simply wake up one day and say "Hey, I want to do X, let's do it in C", one simply wakes up one day and say "Hey, I want to do X incredibly fast, let's do it in C". Regarding Java, well, I dont know who wake up one day and want to program anything in Java nowadays.

So, yes, C/C++ are general purpose languages but not, they are not used for general purpose.

In what language? Python, ruby, and lisps automatically switch to bigints instead of giving an exception.

Exactly, those languages checks for overflow and automatically cast to higher types. No exception is needed. The thing I was trying to say is that in general purpose you are not worrying about overflows, and in case it occurs, instead of unexpected behavior, one would expect at least a warning.

Your example uses floating point. With floating point, overflowing to Inf is the behavior defined by the IEEE spec. Not doing that would be wildly surprising to anybody writing numerical code, and would make it unnecessarily difficult to port floating point code between languages. This is simply not a serious suggestion.

My example is not the best one, because the concept I want to discuss was not float overflow. The thing I want to discuss is that unexpected behavior occurs silently. Also, I think that float overflow is easier to warn because (and I'm not sure about this) micros has an automatically flag for float overflow, but this is not the discussion.

Catching integer overflows could be nice in some cases. It would make sense to have a compiler switch to turn on overflow checking, but for that we would first need to somehow rewrite calls that are intended to overflow, e.g. the famous (a + b) >>> 2 expression.

So it is planned as a possible feature or it is not yet?

Finally, this discussion arise due the fact that when I see that behavior I was intrigued by what is Julia supposed to be. In the post "why we created Julia" it's not said what Julia exactly is. I mean, it's unclear what comes first; a math language, a general purpose a scripting one... Of course everyone wants a language to rule them all, but at least, say what comes first, and as you are the first name in the post, probably you are the indicated person to ask.

In any case, read this as a fresh suggestion coming from someone totally new with julia. Maybe some of this impressions are more common over the people outside the language.

PS:

Also, python isn't general purpose either:

That was numpy, a mathematical external library, not python itself. Anyway, in numpy,

>>> np.uint8(0) - np.uint8(1)
__main__:1: RuntimeWarning: overflow encountered in ubyte_scalars
255

The issue in numpy were this is somehow discussed. Also in R

> .Machine$integer.max + .Machine$integer.max
[1] NA
Warning message:
In .Machine$integer.max + .Machine$integer.max :
  NAs produced by integer overflow

[JeffBezanson]

I mean, it's unclear what comes first; a math language, a general purpose a scripting one...

This is not a productive kind of discussion; applying a certain label to ourselves is not an interesting goal, and none of those terms have agreed-on meanings. In fact this may be the first time I've heard the claim that checking overflow is a key component of being "general purpose". Rather, everything is tradeoffs. We like reliability and ease of use, but so far the consensus among julia developers is that the performance cost of overflow checking is too high.

That was numpy, a mathematical external library, not python itself. Anyway, in numpy,

Python has floating point built in:

In [1]: 1.7976931348623157e308 + 1.7976931348623157e308
Out[1]: inf

From the issue you linked:

I suspect this is still true, and it's too expensive to check on arrays. I suggest improving the numpy.seterr docs to make it clear that number errors can only be detected on scalar types, and not array types.

So, there you go. People like their performance tradeoffs. And you can't really dismiss numpy as some random external library; it's a huge part of the reason many people use python at all.

[RicardoHS]

You can perform cherry picking over the numpy discussion if you want but the fact is that there are opposing opinions inside. This is a comment of another numpy member right behind the one you quote

Eh, you can do it, it just has some penalty in speed. If we added it for arrays then I think we'd also want to have a dedicated option in seterr (no need to add a separate setintwrap function), and make sure that if errors are disabled then we take a fast-path that disables the checking entirely, so that it actually works as a speed/safety tradeoff knob. All of which seems doable and reasonable to me; integer overflow causes a lot of silent miscalculations.

Anyway, it looks like you already answer to me

but so far the consensus among julia developers is that the performance cost of overflow checking is too high.

If the consensus is that, it makes no sense to continue the conversation. I just want to point out that in other languages or libraries this issue is not so clear

If it is not necessary to add anything new, I think this issue can be closed

[elextr]

It seems to me the question was answered by the first response, there are safer integer packages available for those who wish to use them.

Its possible (as shown by the existence of the packages) to build safer but likely slower integers on top of unsafe fast ones, but there is no way of producing a fast integer from a slow one. So the default needs to be fast (and therefore unsafe) to give users the choice of speed with the default, or safety with an alternate package.

[dkarrasch]

When there are different options, and you need to choose one and can't have the other at the same time, then documentation is key, I guess. @RicardoHS Maybe you could give a hint as to where to improve the documentation so that users will know how it works, and can make a choice as to whether have manual consistency checks (in your Markov chain case you know what you need to check), or use additional packages.

Just as a side remark

[0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000000000000000
3×3 Array{Float64,2}:
 10.1917  5.09587  8.15339
 10.1917  5.09587  8.15339
 10.1917  5.09587  8.15339

100000000000000000 < typemax(Int)
true

So, even before integer overflow occurs, errors accumulate and lead to nonsense results. But

[0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000000000000 * [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000000000000
3×3 Array{Float64,2}:
 0.437534  0.218767  0.350027
 0.437534  0.218767  0.350027
 0.437534  0.218767  0.350027

which corresponds to your overflow example. So one could handle it.

[RicardoHS]

A clarification in the FAQ (here) that safe integer operations can be achieve through external libraries could be a plus. Also a indication that the use of those libraries lead to slower time executions.

I'm not familiarized with the full structure of the documentation yet, so idk if that kind of clarification would also fit in other parts.

[dkarrasch]

Sounds like you have a plan for a PR. 😉 Click on your FAQ link, scroll to the top, click "Edit on GitHub", and welcome among the Julia contributors!

[RicardoHS]

created #33871

[abraunst]

[0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000000000000 * [0.6 0.4 0; 0 0.2 0.8; 0.5 0 0.5]^100000000000000
3×3 Array{Float64,2}:
 0.437534  0.218767  0.350027
 0.437534  0.218767  0.350027
 0.437534  0.218767  0.350027

which corresponds to your overflow example. So one could handle it.

not to nitpick, but a^n * a^n == a^2n, not a^(n^2). Using (a^n)^n instead would be correct but still overflows...

[dkarrasch]

True... 😔

[RicardoHS]

Closed in #33871