Julia: fully switch static parameter syntax to `where`

[MikeInnes]

julia> type Foo{T}
           bar
       end

julia> Foo{T}() = Foo{T}(1)
Warning: static parameter T does not occur in signature for call at none:1.
The method will not be callable.
Foo{T}

julia> call{T}(::Type{Foo{T}}) = Foo{T}(1)
call (generic function with 941 methods)

julia> Foo{:bar}()
Foo{:bar}(1)

i.e. the method definition form fails but the call form works. It might be nice if the former was lowered to the latter – having them mean different things is counter-intuitive to me.

Edit: Looks like the issue is the ambiguity of Foo{T}() = ... with call{T}(::Type{Foo{T}}) = ... and call{T, S}(::Type{Foo{S}}) = .... If there's no way around this perhaps it should raise an ambiguity error with the call methods as suggestions.

[yuyichao]

It is actually not limited to when the parameter is not used. The following script prints

A{Int64}(1)
Int64

type A{T}
    a
end

A{T}(::T) = T

type B{T}
    a
end
call{T}(::Type{B{T}}, ::T) = T

println(A{Int}(1))
println(B{Int}(1))

[yuyichao]

And I guess the issue is that (in the above case), whether A{T}(::T) should be treated as call{T}(::Type{A}, ::T) or call{T}(::Type{A{T}}, ::T).

IMO, the current behavior is better. It might be possible to make the non ambigious case working but it will be easily broken if you add other parameters/arguments to the definition.

[yuyichao]

Or maybe we can allow sth like A{T}{}() although I'm not sure if anyone will appreciate this syntax....

[MikeInnes]

Ok, so I guess the original example is effectively lowered to call{T,S}(::Type{Foo{S}}) = .... Which is probably right if you want to parameterise on the type of the arguments.

It's a shame, because Julia almost always achieves an amazing level of _Just Makes Sense™_, but I always find myself banging my head against parametric type constructors.

[IainNZ]

(I also find parametric type constructors one of the more confusing parts of the language, especially if there is a mix of inner and outer constructors)

[yuyichao]

I just end up only using overloading of call for all outer constructors....

[JeffBezanson]

This is the best idea I currently have for dealing with this:
https://github.com/JuliaLang/julia/issues/10146#issuecomment-74924844

[mauro3]

In @one-more-minute's original example, it is a bit easier to see what is going on when using different symbols for the parameters:

julia> type Foo{TT}
                  bar
              end

julia> Foo{A}() = Foo{A}(1)
WARNING: static parameter A does not occur in signature for call at none:1.
The method will not be callable.
Foo{TT}

julia> methods(call, (Type{Foo},))
2-element Array{Any,1}:
 call{A}(::Type{Foo{TT}}) at none:1             
 call{T}(::Type{T}, args...) at essentials.jl:57

(Aside: resolving #10794 would make the display clearer in the original example.)

Instead the expectation was that the following call method would be generated:

julia> call{A}(::Type{Foo{A}}) = Foo{A}(1)

(I'm sure this is clear to most but wasn't to me.)

[JeffBezanson]

I was thinking some more about this syntax problem today. We need to fix this and obtain syntax for "unionall" types at the same time, since underneath they're the same issue. I thought of a few principles I think a solution should obey:

• Method definitions and constructors must contain, with nothing interposed, <thing>(args...) where <thing> simply evaluates to a callable singleton object, or (::FT)(args...) where FT evaluates to a type of callable object. Definitions always look like uses.
• Unionall type syntax and method static parameter syntax should be as close as possible.
• Type variables should be introduced at the beginning (left) to be consistent with x->2x.
• { } should always refer to type application and not be punned.
• Ideally no new keywords (but could live with it if necessary).

One syntax that obeys these is to use some infix operator, here .:

function T<:Real . f(x::Array{T})
    ...
end

T<:Real . f(x::Array{T}) = x[1]

T . Complex{T}(re, im) = ...

# a unionall type
const Vector = T . Array{T, 1}

Using dot is marginal, but might be possible since dot surrounded by spaces is currently deprecated. Some other options are infix !, infix _, or even |_| which looks like a big U (ugly but makes sense).
Or we could use a keyword. I think for almost works:

function for T<:Number +(x::T, y::T)
end

It's as if you're adding definitions for every T value in a loop, which is a pretty accurate mental model.
Does any of this look promising?

cc @StefanKarpinski @ViralBShah @vtjnash @Keno @timholy

[StefanKarpinski]

I kind of like the for version although it's a fairly large change and for T Array{T,3} seems like a kind of weird way to write the type for general 3-tensors. Infix @ is also available and infix ~ could be made available fairly easily.

[eschnett]

In the line

function for T<:Number +(x::T, y::T)

the text for T<:Number belongs together. This isn't visually clear here. Maybe adding parentheses around the T<:Number term would help? Or adding parentheses if there are multiple type variables?

Alternatively, using a "bigger" visual separator before the function name might help. What about ->? =>? ==>? ::>? You mention similarity with x->2x above, so I was naturally thinking of an operator reminiscing of ->, but for types.

Maybe the for can then also be omitted:

function (T<:Number) => +(x::T, y::T)

[JeffBezanson]

Ideally this will be new syntax, so we can use it for free-standing unionall types as well without ambiguity. So ==> is possible. I'll sleep on it.

for does seem to work much better for method definitions than for unionall types by themselves.

[mauro3]

I think for will be confusing to new users because of for-loops. I like ..

[nalimilan]

One could also imagine using forall and its Unicode equivalent ∀. Looks like it would read very naturally.

[mindbound]

I second that, forall has precedents in other languages and IMO reads very easily.

[iamed2]

I would love forall and having ∀ as a synonym, which wouldn't extend long function definition lines much further.

[JeffBezanson]

I'm not sure forall is strictly correct, compared to what it means in languages with universal polymorphism. I think exists is closer, but doesn't read very well.

[iamed2]

where? I believe there was some talk of adding it as a keyword for this before.

[StefanKarpinski]

Where only reads right if it comes after.

On Monday, January 11, 2016, Eric Davies [email protected] wrote:

where? I believe there was some talk of adding it as a keyword for this
before.

—
Reply to this email directly or view it on GitHub
https://github.com/JuliaLang/julia/issues/11310#issuecomment-170634045.

[pabloferz]

Just throwing another idea. Maybe

function with T<:Number +(x::T, y::T) #= function body =# end

with T<:Number +(x::T, y::T) = #= function body =#

[jtravs]

+1 for with.

[tkelman]

with is the first suggestion that makes what's going on here more clear to me rather than less. We have a few with***() do forms, but maybe with{T<:Number} could also work for delimiting this?

[pabloferz]

Maybe, besides of using with, also renaming all with_*() do forms to given_*() do would help.

[mauro3]

Having read through this again, I think the qualifier should come after (as was discussed before over in https://github.com/JuliaLang/julia/pull/13412#issuecomment-152254762):

function +(x::T, y::T)  with T<:Number
    ....
end
+(x::T, y::T) with T<:Number = ...
const IntVector = Array{T, 1} with T<:Integer

versus

function with T<:Number +(x::T, y::T)
   ...
end
with T<:Number +(x::T, y::T) = ...
const IntVector = with T<:Integer Array{T, 1} 

(you can replace with with your favorite keyword)

The reason being that I'm interested most in the function name and not some type-qualifiers, the same goes for the "uinonall" types. Thus it should come first (and I think this is more important than consistency with x->2x, which was Jeff's argument for preceding qualifiers). Imaging scanning a file with many one-line functions some of which will be cluttered preceding type qualifiers. Example from Base:

## Current syntax (abstractarraymath.jl)
conj{T<:Real}(x::AbstractArray{T}) = x
conj!{T<:Real}(x::AbstractArray{T}) = x

real{T<:Real}(x::AbstractArray{T}) = x
imag{T<:Real}(x::AbstractArray{T}) = zero(x)

+{T<:Number}(x::AbstractArray{T}) = x
*{T<:Number}(x::AbstractArray{T,2}) = x


## Syntax with following qualifier
conj(x::AbstractArray{T}) with T<:Real = x
conj!(x::AbstractArray{T}) with T<:Real = x

real(x::AbstractArray{T}) with T<:Real= x
imag(x::AbstractArray{T}) with T<:Real = zero(x)

+(x::AbstractArray{T}) with T<:Number= x
*(x::AbstractArray{T,2}) with T<:Number = x


## Syntax with preceding qualifier
with T<:Real conj(x::AbstractArray{T}) = x
with T<:Real conj!(x::AbstractArray{T}) = x

with T<:Real real(x::AbstractArray{T}) = x
with T<:Real imag(x::AbstractArray{T}) = zero(x)

with T<:Number +(x::AbstractArray{T}) = x
with T<:Number *(x::AbstractArray{T,2}) = x

[pabloferz]

+1

Ok, I agree with the "one-line definitions" argument. with, where or any similar keyword following the identifier seems to me now the best option.

[JeffBezanson]

@mauro3 That's pretty convincing. It also lets us use where, which is the closest thing to a standard syntax for this. with has other stronger associations e.g. from its use in python. We would also eventually have

function foo(x::Array{Array{T} where T})

which doesn't look too bad.

[vtjnash]

+1 for the postfix

would given be a closer choice of standard term? unfortunately parsing the common symbols for that (| and :) unambiguously seems like it may be hard. the mathematica equivalent of /. (aka replacement rule) might be possible?

for clarity, i'm hoping we can have newlines:

+(x::AbstractArray{T,1}) = x
    given T<:Number
+(x::AbstractArray{T,2}) = y
    given T
function +(x::AbstractArray{T,N})
given T<:Number, N
    return y
end

[JeffBezanson]

I think that's some pretty difficult lookahead for the parser. I believe there is no situation in the present syntax where a complete expression followed by a newline can be augmented by something on the next line. Seems more doable if you let the keyword dangle:

+(x::AbstractArray{T,1}, y::S) = x where
    T<:Number, S<:SomeReallyLongType

[StefanKarpinski]

I'm not a fan of having the where/with clause so late – right after the method signature seems clearer.

[pabloferz]

I think that the clause right after the signature would also allow to parse symbols such as : or | without too much trouble.

[mschauer]

Just a possibility which was not mentioned,

where T<:Real, N<:Integer
   conj(x::AbstractArray{T}) = x
   conj!(x::AbstractArray{T}) = x

   real(x::AbstractArray{T}) = x
   imag(x::AbstractArray{T}) = zero(x)
   convert(::Type{N}, x::Ptr) = convert(N,convert(UInt, x))
end

[JeffBezanson]

@pabloferz That would be nice but leaves open how to write unionall types, since we couldn't use Array{T} | T.

[mauro3]

@mschauer: that could be some good extra syntax sugar.

[JeffBezanson]

That would be especially easy to implement as

begin where T<:Real
    ...
end

[vtjnash]

this doesn't seem like a v0.5 priority to me. it seems like it might be better not to introduce a new syntax for this right now. can we remove from the milestone?

[StefanKarpinski]

Major syntax changes at this point seem like they're not in the cards.

[JeffBezanson]

Agreed, I don't think we have time for this. Although technically separable, this change goes along nicely with the 0.6 type system rewrite since it provides syntax for the "new" UnionAll types.

[ararslan]

@mschauer I really like your proposed block syntax, though isn't that how we use with currently? That is,

with(something) do
    stuff
end

For consistency it might make more sense to do something like

with(T<:Real, N<:Integer) do
    conj(x::AbstractArray{T}) = x
    conj!(x::AbstractArray{T}) = x

    real(x::AbstractArray{T}) = x
    imag(x::AbstractArray{T}) = zero(x)
    convert(::Type{N}, x::Ptr) = convert(N,convert(UInt, x))
end

in that case.

[mschauer]

@ararslan Sorry, I don't get what you mean with "currently". In any case do currently indicates an action (a function call) and I think it should not be used to start a block of declarations.

[ararslan]

@mschauer Sorry, I wasn't clear. I guess I was thinking about how Tom Breloff's Plots.jl package uses with() do: it specifies arguments that are common to all subsequent plot commands. From that perspective it makes sense that one could use that syntax to specify types that are common to all subsequent definitions as I showed in my comment above, but it didn't occur to me that Plots.jl could be using the syntax in a nonstandard way.

[ararslan]

My opinion doesn't really matter but for what it's worth I _greatly_ prefer with to where in this scenario. I really like @pabloferz's proposed syntax for it as well; I think it makes it much clearer what's happening.

[StefanKarpinski]

It would be nice to have some thoughts on where this is going to go now that we have where syntax. @JeffBezanson, could you write a few words on what might go into 0.6/1.0?

[JeffBezanson]

The new syntax we need is all set, and inner constructors (the most confusing and urgent part) are fixed. I decided not to fully deprecate the static parameter syntax (f{T}(x::T) = x), since there are simply too many uses. We can gradually transition to the new syntax during 0.6, and fully deprecate it in 1.0beta.

[lyrachord]

tardy sytax

f[T:constraint of T; V](T param1, V param2, ...)

[logankilpatrick]

Hello, Does anyone know how to resolve the following issue: Warning: Deprecated syntaxparametric method syntax Base.show{S}(io::IO, m::Base.MIME("text/plain"), scvec::Vector{StatesContainer{S}})around /Users/logankilpatrick/.julia/packages/SHERPA/A8APz/src/utils/states_containers.jl:74. │ UseBase.show(io::IO, m::Base.MIME("text/plain"), scvec::Vector{StatesContainer{S}}) where Sinstead.

I did what it suggested and instead, I now get : ERROR: LoadError: LoadError: ArgumentError: invalid type for argument m in method definition for show at /Users/logankilpatrick/.julia/packages/SHERPA/A8APz/src/utils/states_containers.jl:74

Thanks!

[nalimilan]

Please use the Discourse forum for questions, and keep GitHub for bug reports. Thanks!