Julia: Infix operator definition syntax needs documentation

[tkelman]

I had no idea we support the below syntax for defining infix operators. AFAICT it's not in the manual, and before https://github.com/JuliaLang/julia/commit/f07485f89461047531a2c9e10b51dc1301b9c699 I don't think it was in any tests either. Am I the only one who was unaware of this?

julia> import Base.+
julia> immutable wrapper
           val
       end
julia> a::wrapper + b::wrapper = wrapper(a.val + b.val)
+ (generic function with 172 methods)
julia> wrapper(1) + wrapper(2)
wrapper(3)

[KristofferC]

Didn't know about that! Cool though.

[toivoh]

It comes as a natural consequence of the fact that

a::wrapper + b::wrapper = wrapper(a.val + b.val)

is parsed the same way as

+(a::wrapper, b::wrapper) = wrapper(a.val + b.val)

but I actually seem to recall a discussion about explicitly disallowing it a long time ago, since it can cause inadvertent method definitions at unexpected occasions (almost thought that we had).

[StefanKarpinski]

IIRC, Alan posted a bug email about this since he'd done something like a + b = 2 or something and was surprised when there was no error since this just defines a fallback method for +.

[ararslan]

I actually seem to recall a discussion about explicitly disallowing it a long time ago, since it can cause inadvertent method definitions at unexpected occasions (almost thought that we had).

A hearty +1 to disallowing assignment using infix notation. If you accidentally use a + b = 2 instead of a + b == 2 you're in for an odd surprise, which is what I assume happened to Alan. I doubt this syntax was being used much in any serious manner so it would likely be a pretty undistruptive change.

[JeffBezanson]

I like that this syntax works. And I generally dislike the design strategy of "looking around for things to ban".

[ararslan]

I don't think it's about looking for things to ban so much as the principle of least surprise. For me, anyway.

[tkelman]

It's more error-prone than it is useful.

In my head it feels like LHS and RHS of assignments should have slightly different rules, it runs against my intuition sometimes when they don't.

[JeffBezanson]

The argument that somebody might be confused or make a mistake is another one I usually find unconvincing. If you're confused, the right response is to look at the reasons actually in play and see if they make sense. I'm also concerned about piling up dozens of special cases in order to fix each instance where somebody was confused once. It seems unlikely to lead to a coherent result.

[tkelman]

"the reasons actually in play" are that the parsing happens the same on both sides of an assignment. That simplifies the parser but doesn't result in clearer semantics for users of the language IMO. This is also terribly underdocumented, undertested, and virtually unused such that people who've been using the language for years don't even realize it works because they subconsciously think it shouldn't.

[JeffBezanson]

a + b parses the same as +(a,b) everywhere. But this is not _just_ a quirk of the parser; it's pretty fundamental that a + b is a function call like any other.

[vtjnash]

Is it OK to agree with both sides?

Coming from the perspective of the parser, I agree with Jeff that a + b is the same function call as +(a, b). This simplifies syntax rules, since there's no need to make a distinction between the two parsed forms. Thus, fewer parser bugs, macro bugs, edge-cases, yada yada yada

Coming from the perspective of the user, I'm on the side of disallowing this on the LHS as unnecessary and too feature-y for something the user probably will never encounter "in the wild" other than in typos. Of course, simply encountering +(a, b) = a-function-definition is probably surprising to many users (coming from languages where even +(a, b) isn't a valid), but in counter-point, this form is common to see in Julia code, thereby making it idiomatic.

Fwiw, as Jeff said, it's fundamental that a + b and +(a, b) both lower to Expr(:call, :(+), :a, :b). But it doesn't follow that needed to parse to the same syntax. For instance, the infix form could parse to Expr(:+, ...) or Expr(:infixcall, :(+), ...) or variations on this theme. It's then completely trivial to disallow method definitions that came from an infix call syntax.

Since I believe the sole job of the parser is to interpret user intent, I don't believe that there's anything "fundamental" about the rules about how it should be coded (sorry lisp, but it seems mexprs won over the ease of writing a parser for sepxrs. http://c2.com/cgi/wiki?EmExpressions)

If you're confused, the right response is to look at the reasons actually in play and see if they make sense.

Huh? This advice seems to miss the point. I interpret Tony's OP as essentially: "hey, this unexpectedly worked instead of giving a syntax error seemingly because it was easier to parse that way instead of trying to infer user intent, but I'm not sure it is making sense" (which is what this statement says for him to do). Also, isn't our job as language designer developers to make it so the user doesn't need to stop and say "I'm confused, what does the compiler think is going on here?"

[tkelman]

f(x, y) = body is new if you're coming from other languages, but it usually clicks right away with people because it's intuitively familiar from standard mathematical notation for defining functions. And the correspondence between infix and functions makes definition of operator methods with prefix notation feel natural, even when calling them with prefix notation usually doesn't.

The infix definition form x op y = body might appear in papers if you're defining some nonstandard operator, but there aren't as many papers that need to define addition so seeing x + y = body in math without any context could have several intended meanings.

[JeffBezanson]

This isn't about the parser. If we wanted to add an exception for this, of
course we could. The question is more why, conceptually, an exception is
warranted, whether it involves the parser or whatever other component.

(As an aside, I do think it would be very annoying to have to handle both
call and infixcall forms everywhere that handles surface exprs.)

[toivoh]

I agree with @tkelman and @vtjnash that occurrences of a + b = ... are probably almost always mistakes, so it would be nice to catch them. Especially since the effects are so unexpected.

On the other hand, I'm involved with the development of a DSL that allows you to write equations inside a macro, such as x + y = z. So from that point of view, I really hope that that syntax will continue to parse.

[JeffBezanson]

For prior art, Haskell also supports this syntax.

As for this being "undertested" --- the fact that +(a,b) and a+b are treated so similarly means they use almost all the same code paths, so you naturally get better test coverage and there is less bug surface. If we start calling out exceptions here and there, then we'll really start feeling a test shortage.

[nalimilan]

More evidence that this confuses people: https://discourse.julialang.org/t/unexpected-behavior-of/2891/9

[StefanKarpinski]

I think there's a lot more that's confusing about that example than just the a*b = c syntax.

[ChrisRackauckas]

More evidence: https://discourse.julialang.org/t/for-loops-giving-weird-results/8316/5

[simonbyrne]

And https://discourse.julialang.org/t/redefining-x-y-0/17015

[EricForgy]

Hi guys 👋

At one point a while back, I asked for a list of available infix operators and was directed here:

https://github.com/JuliaLang/julia/blob/master/src/julia-parser.scm#L9-L29

That was helpful. Then months went by, and I wanted to know again, but couldn't remember, so was again directed to

https://github.com/JuliaLang/julia/blob/master/src/julia-parser.scm#L9-L29

This is where my "Rule of 3" kicks in. Before I forget this again and get directed to julia-parser.scm again, it would be good if the docs said something about infix operators and I'm happy to give it a shot.

It seems like this:

https://docs.julialang.org/en/v1/manual/mathematical-operations/#Operator-Precedence-and-Associativity-1

is a good place to put something. Even just a word "infix" somewhere would allow me to find it next time.

It was suggested on Slack:

Or maybe a short subsection after the "precedence and associativity" part explaining that all those can be called as infix. This should then probably also mention the sub/superscript thing

Any thoughts? Should I just go for it and try to submit a PR? Someone more qualified want to do it? I am certainly no expert on infix operators 😅

[DhruvaSambrani]

I for one love this syntax. It just needs to be documented well.

== is for conditional, = is for assignment, and if someone messes that up, isn't it their fault? Maybe throw a warning?

But I'm in favour of keeping this syntax because it's very similar to what you'd do in Math, and allows for greater expressivity, something that Julia is known for.