Julia: remove `:=` operator?

[JeffBezanson]

We have long parsed := as an assignment-like operator, but have not yet used it in the language. I used to think it was harmless, but if we removed it then we'd be able to write Base.:== instead of Base.:(==). Worth it?

[ararslan]

Absolutely worth it, IMO. The fact that == and === have to be written as symbols with parentheses while the rest of the commonly overloaded symbols don't is confusing. When someone goes to overload basic operators from Base, it's not immediately clear why Base.:+ works but Base.:== gives ERROR: syntax: invalid identifier name ":=" and instead requires Base.:(==).

[KristofferC]

How about macro based APIs? https://github.com/JuliaPlots/RecipesBase.jl#a-real-example

[JeffBezanson]

Yep, macros can use it now and this would be breaking. Another consideration is that colon has an awful lot of meanings, and := is yet another (though rarely used).

Another change to consider is adding a <- operator, meaning a space would be required in x < -y. That would provide another assignment-like operator.

[rfourquet]

I think I prefer not this change. It's nice having an assignment-like operator usable in macros (FWIW, I've used it privately for what is proposed in #22460, cf. this comment, although this is not necessarily the best syntax for this functionality...)

[tshort]

I like leaving := for DSLs. Modia uses it, I think. Pinging @MartinOtter and @toivoh.

[ChrisRackauckas]

I do like this potential use for :=: https://discourse.julialang.org/t/idea-for-setting-both-mutables-and-immutables-without-allocation/1318

[Jutho]

It's also used (within a macro) in TensorOperations.jl. However, I could consider switching to .= and = for the two different meanings. However, the change closest in spirit to Julia's base use of assignment operators would then be to make the non-trivial and breaking modification to replace = by .= and := by =.

[ararslan]

It actually seems like =, ==, and === are the _only_ operators that require parentheses when referring to them as symbols, and that's solely due to :=. The consistency of being able to write Base.:== feels really nice. Plus that means there are, as far as I can tell, no operators which require parentheses to write as a symbol after this change.

In keeping with that, it might also be nice to make := equal to Symbol("="), which #23039 does not do; that still requires parentheses.

[KristofferC]

I don't think killing (injuring) all the packages that uses := for macros is worth it in order to sometimes not have to write ().

[StefanKarpinski]

In any case, the current form of := is super broken and crazy, since a := b means a = a:b. That should be changed and if we're going to allow := as an operator, it should be parsed like =>.

It seems to me that the reason that Base.:== isn't a valid syntax for the == operator is a combination of the fact that := is an operator and that it can be dotted so that the parser doesn't know if you're writing Base .:= = or Base. :(==). If the := operator was a normal assignment-type operator that didn't support dotting, then Base.:== could work.

[vtjnash]

since a := b means a = a:b

I don't think it means that any more than a .= b means a = a.b. Also:

julia> expand(Main, :( a := b ))
:($(Expr(:error, "invalid syntax a := b")))

[JeffBezanson]

:= does not support dotting. In Base.:= it thinks you're trying to look up := in Base.

[StefanKarpinski]

:= does not support dotting. In Base.:= it thinks you're trying to look up := in Base.

That was the case that I suggested could be addressed by greedy parsing of operators – which is already what we do for identifiers, so seems like the least surprising behavior. Then making := undotted would, I believe, resolve the Base.:== issue without completely removing :=.

[JeffBezanson]

I don't understand. We already parse operators greedily --- as soon as we see an operator character, we keep collecting characters as long as the result remains a valid operator. Here, first we see .. Then the next character is :, and .: is not an operator or a prefix of an operator, so we get the token .. The next character is :, followed by =, and := is an operator. The next character is =, but :== isn't an operator so we yield :=, etc.

[StefanKarpinski]

julia> Base.:==
ERROR: syntax: invalid identifier name ":="

1. The name := is not a valid identifier – i.e. this is like += not like => even if there's no actual lowering to a = a : b. If we have := it should just be a normal infix operator like =>.

2. The parser stops after Base.:= even though consuming the next character would make the :== a valid identifier. That doesn't seem greedy (enough) to me. Yes, :== is not a valid operator, but == is and we want :== to be a way to write the symbol for ==.

[StefanKarpinski]

Once upon a time, we could writeBase.op for most operators, which is why the parser seems to reach Base.:= and decide that it has parsed a qualified operator. The fact that := is not a currently valid identifier is a side issue – I'm arguing that it should be, so that would change. However, now that Base.op does not work for most operators – because .op is an operator for most of them, the precedence of the Base.(:=) interpretation is causing problems here. Perhaps when the parser sees Base.:<something> it should give precedence to the interpretation of Base.:op instead of what it's doing now, which is preferring the interpretation of : being part of the operator name? I think with that change we'd be able to write all operators as Base.:op as long as no operator that begins with : is dottable, which seems like a modest restriction.

[JeffBezanson]

The only reason we give the invalid identifier name ":=" error is so := can't be used as a variable name. I assumed that allowing (:=) = 3 would be considered a bug. The error also reserved := for all non-macro uses, e.g. making it easier for us to remove it now if we want. Being an invalid identifier has nothing to do with being an update operator. It just means it's an invalid identifier. ... gives the same message.

Quoted symbols aren't parsed as single tokens; they consist of a : token followed by a name. We don't have any way to tokenize :==, then decide that it can instead be interpreted as : followed by ==. That seems more like backtracking than greediness to me.

You're right that this is due to a switch from having very few dot operators, to having dots on almost every operator. Your solution of preferring Base.: seems like it could be implemented by (somewhat ironically) making .: an operator that basically does the same thing as .. Thankfully nobody seems to want a broadcasted version of : for making arrays of ranges...?

[StefanKarpinski]

Thankfully nobody seems to want a broadcasted version of : for making arrays of ranges...?

Some sick bastard probably wants it :rofl:. But no, I think that could work. Of course, there's always the Base. op option as well, which happens to already work.

[JeffBezanson]

I tried making .: an operator and it seems to work like magic so far. The only downside is that Base.:== works but :== doesn't, which is a bit suboptimal.

[StefanKarpinski]

From triage: we want to keep := as a parseable operator, the main motivation here is actually to be able to write Base.:== for the qualified == function. However, if we figure out a way to do that in the future without getting rid of the := operator (e.g. change the way the parser parses operators), that won't be a breaking change since Base.:(==) will still work, the parens just won't be necessary anymore.