Mypy: Allow type aliases such X = List[T] if T is a type variable

[JukkaL]

This code should be okay:

from typing import TypeVar, List
T = TypeVar('T')
X = List[T]
def f(x: X) -> None: ...

It should be equivalent to this:

from typing import TypeVar, List
T = TypeVar('T')
def f(x: List[T]) -> None: ...

This was suggested by Guido, "based on the general equivalence in Python of

X = <stuff>
f(X)

to

f(<stuff>)

".

[JukkaL]

This is not included in PEP 484, so closing this.

[gvanrossum]

Whoa, maybe this should be added to PEP 484? I still am in favor of this.

[gvanrossum]

In particular, PEP 484 (https://github.com/ambv/typehinting/blob/master/pep-0484.txt) contains this example:

Vector = Iterable[Tuple[T, T]]

[JukkaL]

What about something like this:

Table = Dict[str, T]

def f(t: Table[int]) -> ...: ...

[gvanrossum]

The implementation in typing.py requires that you write Table[str, int] in the argument attribute. I agree this is a bit arbitrary, but the alternative felt pretty confusing. In an earlier version I was trying to support this to the extreme, allowing e.g.

# T, U, V are TypeVars
A = Tuple[T, U, V]
X = Dict[int, Union[Tuple[U, U], A]]

and then X would have 3 parameters (U, T, V). That felt too complicated, so I retreated on the rule that you can't change the number of parameters at all, and substitutions must be either completely concrete or single type variables, and the resulting type still has the original number of parameters. Maybe we need to find a middle ground, where your example is allowed and does in fact reduce the number of parameters. But you could still have only substitutions that are either completely concrete or a single type variable. An edge case would be X = Dict[T, T] which gives X a single parameter, T.

[berdario]

Just stumbled upon this, the confusing thing is that the error is reported not at the line of the type alias, but at the line at which the TypeVar (T in this case) has been defined:

 error: Invalid type "module.T"

[ze-phyr-us]

With mypy 0.4.4 I'm getting the error at the line of the type alias:

from typing import TypeVar, Tuple

T = TypeVar('T')
U = TypeVar('U')

X = Tuple[T, U]

$ mypy test.py
test.py:6: error: Invalid type "test.T"
test.py:6: error: Invalid type "test.U"

[roganov]

+1 for this

I'm developing a dependency injection library where types are first-class citizens (users can request instance provided a type), that's why I need either generic aliases or ability to define generic types at runtime.

[ddfisher]

I think we shouldn't allow this, but should instead allow definition of parameterized type aliases. Having the type variable be explicit makes a big difference in understandability and power.
Here's an example. Let's say we have Vec defined in some module like so:

# in module vector
T = TypeVar('T')
Vec = List[Tuple[T, T]]

Let's say we want to define append and map. Here's how we'd do that with this proposal:

# in module __main__
from vector import Vec, T

def append(vec: Vec, elem: T) -> Vec: ...

U = TypeVar('U')
def map(vec: Vec, f: Callable[[T], U]) -> ???: ...

Note that we _have_ to use vector.T to properly define append. If we define our own T locally, that will cause a confusing error (because it will be considered a different type variable). (Of course, for all expressions _not_ involving a Vec, T will behave just like any other type variable, so I think it's likely to be seen in other contexts too, which would be additionally confusing.) Also note that the fact that elem is the right element type for vec is implicit rather than explicit (as is the fact that vec and the return type have the same type). Finally, note that we _cannot_ define map using the type alias -- uses of the type alias are confined to those which do not change the parameterized type.

Here's how it would look with parameterized type aliases:

# in module __main__ with parameterized type aliases
from vector import Vec

T = TypeVar('T')
U = TypeVar('U')
def append(vec: Vec[T], elem: T) -> Vec[T]: ...

def map(vec: Vec[T], f: Callable[[T], U]) -> Vec[U]: ...

With parameterized type aliases, all the relationships between parameters have been made explicit, and you can express concepts like map using the type alias.

Another problem with the proposal as it stands is that type variables only really make sense in context. For example, you can't have a top-level variable with type T. T has to be captured in either class or function scope. The way this happens is already a bit more implicit than I'd like, but this proposal would make it very non-obvious where type variables are hiding.

To sum up, parameterized type aliases would be both more powerful and more understandable than non-parameterized type aliases, and I think we should implement them instead.

[gvanrossum]

Just note that this would require a corresponding PEP change. TBH I'm not
excited about parameterizable type aliases, maybe we should just disallow
them in the PEP (as mypy already disallows them).

[rwbarton]

I agree with @ddfisher that the original X = List[T] with the result that X is "linked" to the variable T is confusing and insufficiently expressive. Of course, in this particular case you could simply write X = List instead.

The problem with Vector = Iterable[Tuple[T, T]] with the idea that Vector[int] then means Iterable[Tuple[int, int]] is that you need a way to disambiguate the order of multiple type parameters. Plus it just looks wrong and doesn't fit in with the idea of replacing things with their definitions: (Iterable[Tuple[T, T]])[int] isn't legal. The syntax should be more like

Vector[T] = Iterable[Tuple[T, T]]

though that exact syntax isn't possible without a previous definition of Vector.

[JukkaL]

I also agree that the original formulation is not a good idea.

Making generic types indexable if they have type variables could likely be made to work with some heuristics and I believe that it would be at least be marginally useful, but I don't have a strong opinion. I'd suggest finding several real-world use cases where this could be used before making a decision so that we can validate whether this actually looks worth implementing. If the benefit is minor, it might not justify the extra rules and implementation complexity.

If we want to support it, we'd perhaps want to make List[T][int] valid syntactically, for consistency, even though it seems questionable stylistically. However, somebody might want to write something like Tuple[T, T][Dict[int, List[int]] as a shorter form for Tuple[Dict[int, List[int]], Dict[int, List[int]]].

For cases like X[T, T, S][int] where the substitution is ambiguous we can order the type variables based on the order they first appear in the type textually. This would mean that X[T, T, S][int] is equivalent to X[int, int, S], and X[T, T, S][int, str] is equivalent to X[int, int, str].

[gvanrossum]

Then we need at least an issue in the typing repo reminding us that we
probably have to change the PEP.

[ilevkivskyi]

@rwbarton Maybe I misunderstand you, but it looks like you a talking about an outdated version of typing.py. Namely, (Iterable[Tuple[T, T]])[int] is perfectly legal since "Revamped Generics" (https://github.com/python/typing/pull/195). The problem of disambiguating the order of multiple type parameters was also solved in that PR by Guido. Few simple examples (assuming that https://github.com/python/typing/pull/296 for nicer repr is merged soon):

>>> Vector = Iterable[Tuple[T, T]]
>>> Vector[int]
typing.Iterable[typing.Tuple[int, int]]
>>> Iterable[Tuple[T, U, T]][int]
Traceback (most recent call last):
...
Too few parameters for typing.Iterable[typing.Tuple[~T, ~U, ~T]]; actual 1, expected 2
>>> Iterable[Tuple[T, U, T]][int, str]
typing.Iterable[typing.Tuple[int, str, int]]

[JukkaL]

I had forgotten about this change as well... Did it go in without much discussion?

[ilevkivskyi]

@JukkaL Most of discussion about this happened in https://github.com/python/typing/issues/115
I thought the "specification" was well discussed there. The actual implementation maybe was not well discussed, although I like it and it seems quite robust.

[JukkaL]

Oops, I think that I wasn't paying much attention to the original discussion then!

Yeah, the implementation sounds reasonable -- my biggest reservation is how useful this will be in practice, and it's unclear when we are going to have the bandwidth to add support for this to mypy.

[ilevkivskyi]

@JukkaL
I am interested in this not directly, but mostly because of https://github.com/python/typing/issues/299 (allow subclassing Tuple and Callable). The point is that both of these trigger similar code in typing.py:

C = List[Tuple[T, T]]
C[int]
# and
class C(List[Tuple[T, T]]): ...
C[int]

Probably you are right that the first form might be of less priority for mypy. Both above forms are already allowed in typing.py, but the following forms are _not_ supported:

class C(Tuple[T, T]): ...
class C(Tuple[T, ...]): ...
class C(Callable[[T], T]): ...

I would like to submit a PR to typing.py to allow these forms and using C[int] for them. As I understand your response on the mentioned issue, you are in favor of this. My idea is just to extend the GenericMeta infrastructure (__origin__, __args__, etc) on Tuple and Callable (these two could be seen as special variadic generics). As a side effect of this Tuple[T, T][int] will also work.

I think this is rather positive side effect. If people will ask for this and we decide to support this in mypy, then we would not need to change typing.py. Also IMHO generic aliases improve readability:

VecT = Iterable[Tuple[T, T]]
def dilate(vec: VecT, scale: T) -> VecT: ...
def rotate(vec: VecT, angle: T) -> VecT: ...
v: VecT[float] = []

[JukkaL]

@ilevkivskyi Yeah, I'm not opposed to this.

By the way, shouldn't your example be written like this:

Vec = Iterable[Tuple[T, T]]
def dilate(vec: Vec[T], scale: T) -> Vec[T]: ...
def rotate(vec: Vec[T], angle: T) -> Vec[T]: ...
v: Vec[float] = []

(Based on what @ddfisher wrote above.)

[ilevkivskyi]

@JukkaL

By the way, shouldn't your example be written like this:

Good point! Indeed, it looks much cleaner and Vec[T] is also supported in typing.py already.

[gvanrossum]

Re: Vec[T] we should update the PEP example.

Re: subclassing Tuple, let's implement it first and add to the PEP once
implemented. Seeing it as variadic type var is great.

Re: subclassing Callable, I foresee some issues related to the argument
list.

--Guido (mobile)