Julia: bit shift operator precedence

[JeffBezanson]

Scrutinizing the operator precedence table, it struck me as a bit odd that bit shift operators have their own precedence level, strictly _lower_ than multiplication. a*b>>c*d is an ugly expression to be sure, but it's parsed as (a*b)>>(c*d), which doesn't make a whole lot of sense.

Since a<<b means a*2^b, << should perhaps have the same precedence as *, or maybe even the same precedence as ^. Or maybe it should be moved to one notch higher than *, instead of one notch lower (putting it between * and //).

[sbromberger]

Reference table here: https://en.wikipedia.org/wiki/Order_of_operations#Programming_languages

Bitwise shift left and right is level 5, above ("lower than", technically) + / -.

See also http://www.knosof.co.uk/cbook/accu06a.pdf

[JeffBezanson]

Giving << lower precedence than + is widely regarded as a mistake.

The linked study looks interesting but I didn't manage to extract a concrete recommendation at a glance. Do you have a strong take-away from it?

[sbromberger]

Do you have a strong take-away from it?

None at all. I knew Ritchie never liked the precedence of bitwise operators in C; I wasn't suggesting that we emulate them. I was just providing a couple of reference points for the decision :)

[StefanKarpinski]

I wonder if we could do better with precedence in general by borrowing the idea from Fortress of making precedence a partial ordering and having it be an error not to parenthesize ambiguous cases.

[jakebolewski]

Wouldn't you still have to define a relative ordering of operators? I don't see how that proposal is that much better than having fixed rules with a couple ambiguous cases in terms of precedence level.

[mikewl]

I do feel while erroring or at least warning on potentially ambiguous statements would be nice. It seems though like something a linter could deal with instead.

[JeffreySarnoff]

I prefer bit ops binding more tightly than arith ops, and, subordinately, unary ops binding more tightly than binary ops.

[JeffreySarnoff]

Clarification:

   unary ops  before binary ops of the same nature
   bitwise logical ops { &, |, $, ↑  }  before arith ops { +, -, *, /, \, %, ^}
   arith ops before wordwise ops { >>, <<. >>>, ror, rol }
   wordwise ops before  comparatives { < .. == .. > } 
   comparatives before truth functional connectives { &&, ||, $$, ↑↑ }

[JeffBezanson]

Proposal: move bit shifts one notch higher than //, so x//1<<n is parsed as x//(1<<n). Triage feels it's better for shifts to be either strictly higher or strictly lower than arithmetic operators, instead of in the middle. It also seems weird to shift the result of a division; more useful to divide the result of a shift.

It's also possible shifts should be merged into the ^ level, since it behaves similarly.

[JeffBezanson]

One issue: ^ is parsed right to left, but << is parsed left to right. So we can also decide whether we want to change that.

[JeffreySarnoff]

I concur: arith < shift < power

There does not appear any explicit advantage to
arith < shift, power. And while thier purposes intersect, the do mot coincide: Shifting does mach powers of 2 ... power does not help with bit field masking.

[JeffBezanson]

Done.

[PallHaraldsson]

This seems important to document (and not just in NEWS).

I'm not sure if Compat can handle this or is wanted there.

[JeffBezanson]

The precedence has not actually been changed yet; I just required parentheses. Adding parens is easier than typing @compat.

[saolof]

This is a pretty big change.

While I can see that this actually does work substantially better for most arithmetic expressions where you want to use bitshifts, it is a different convention from what every other language uses and could break the principle of least surprise. It may especially surprise a lot of people coming from C++ where the bitshift operator is a notoriously common target for overloading because of its convenient place in the precedence table and the fact that the >> notation suggests chaining.

Julia Libraries that overloaded >> to denote monadic bind or andthen semantics will be shafted by this change if it goes live. This is a common enough expression that it could cause quite a lot of code breaking.

Then again, Go also decided to give >> higher precedence than + (it is a multiplicative operator), so there is at least one other language that made the same decision. Pascal also had a similar precedence, it's mainly a case of most languages after C copying C's operator precedence table. So there is a case for adopting this as the more logical precedence for bitshift.

[StefanKarpinski]

Julia Libraries that overloaded >> to denote monadic bind or andthen semantics will be shafted by this change if it goes live. This is a common enough expression that it could cause quite a lot of code breaking.

Julia libraries should absolutely not be doing this. Overloading random operators like << and >> to do completely unrelated things is exactly why C++ gave operator overloading such a terrible reputation. The only way for operator overloading and multiple dispatch in general to work sanely is if programmers are disciplined about only overloading implementations and not meanings of functions.

[StefanKarpinski]

The point about principle of least surprise is valid, however – it's concerning to have different precedence from other mainstream languages, evn if that precedence is a bit unfortunate.

[JeffreySarnoff]

afaik the principle of least surprise was developed to encourage cognitive consonance rather than conform to alternative practice (translation: we have the opportunity to do it better, and we should -- it takes little effort to learn this specific distinction, and overcoming Julia promulgating a worse practice takes enormous effort)