Julia: make numeric literal juxtaposition less brittle

[StefanKarpinski]

The most obvious two changes would be:

• disallow 0 as a juxtaposed numeric literal coefficient: 0n would be a syntax error.
• disallow juxtaposition of floating-point numeric literals with trailing .: 1.x would be a syntax error.

There may be others but these two strike me as clearly a good idea. One nice thing about disallowing 0 as a juxtaposed numeric literal coefficient is that it opens up as many 0x1234 syntaxes as one might ever want, so it kind of future-proofs us for that. Another nice thing is that instead of getting a surprise when they try 0x after 0y just working, people will get a warning as soon as they try 0y and they can immediately learn that 0 doesn't work as a juxtaposed numeric literal coefficient.

[stevengj]

See also #5246, #11529, #10920, ... for related problems with literals and operators etc.

[StefanKarpinski]

I think I would generally be in favor of deprecating float literals with trailing . but that's another matter.

[StefanKarpinski]

This is a pretty simple change and we would benefit from doing it sooner rather than later.

[StefanKarpinski]

Bump – @JeffBezanson, could you do this?

[JeffBezanson]

Number parsing changes are never simple :)

Working on this. First interesting issue I hit is 0im, which seems useful to allow.

[StefanKarpinski]

Ah, that's an interesting one.

[omus]

Maybe a better option to disallowing 0n is to require 0x1234 syntaxes to use upper case letters. Then we could also support many 0x style syntaxes via r"0[a-z][A-Z0-9]+". This would mean that 0xa is 0*xa, 0xA is 0x0A, and 0im would still work.

[StefanKarpinski]

How about allowing 0x to mean 0*x and disallowing 0 juxtaposed with any starting with a single non-digit followed by a digit (and anything after that). That way 0x and 0im are allowed, and 0x1 would be a hex value while 0y1 would be illegal. It's slightly more brittle than I would like, but at least the rule doesn't care about a specific list of leading characters.

[StefanKarpinski]

@omus: Unfortunately, using the lowercase form is by far more common and also looks better.

[omus]

@StefanKarpinski I agree that forcing uppercase letters looks worse.

I like your suggestion but unfortunately the non-digit following a digit heuristic means that 0x1234 is a hexidecimal while 0xabcd would not be.

Could we not just try to parse r"0x\S+" as a hexidecimal and allow for 0x as 0*x? Where I see this breaking down is if we decide to make 0i a special prefix then 0im would be parsed as a special value.

[StefanKarpinski]

I like your suggestion but unfortunately the non-digit following a digit heuristic means that 0x1234 is a hexidecimal while 0xabcd would not be.

smacks forehead

[stevengj]

Honestly, disallowing float literals with a trailing . seems like a more pressing issue. (Disallowing it in Julia source code, that is; functions to parse CSV files etcetera should continue to allow trailing ..)

[JeffBezanson]

I agree. Shall we merge #16339?

[tkelman]

Was #16339 the biggest part of this that we wanted to do prior to 0.5? For the rest should we remove the milestone?

[JeffBezanson]

It doesn't seem worthwhile to me to disallow juxtaposing 0, so I'll close this.