Mypy: Type variable return value from class methods

[orenbenkiki]

mypy gets confused about the type of a type variable return value from a class method:

from typing import TypeVar
from typing import Generic

P=TypeVar('P')

class G(Generic[P]):

    @classmethod
    def c(cls) -> 'P':
        assert False

    def s(self) -> 'P':
        assert False

class C:
    pass

class GC(G[C]):
    pass

c = C()
c = GC().s() # OK
c = GC.c() # error: Incompatible types in assignment (expression has type "P", variable has type "C")

This is causing a huge PITA in my code...

Actually, in my code I would really want to make c a @classproperty, that is, be able to write GC.c instead of GC.c(). I can implement a @classproperty decorator by myself, of course, but that would mean giving up on any hope whatsoever of mypy being able to type-check my code. Of course, in an ideal world, one would just be able to say both @property and @classmethod, but they don't compose... Sigh.

I expected the simple @classmethod to work, though. I _assume_ this is a bug? Is there a workaround?

BTW - I am using mypy 0.501 and Python 3.6.0, if that matters...

[ilevkivskyi]

It is not so clear this is actually a bug. Normally, type arguments are only applied on instances, _not_ on classes (we have even special errors for things like C[int].x). It is less clear in this case, but why can't you use c = GC().c() (it is allowed to call classmethods on instances)?

(Also you don't need quotes around P in return annotations of s and c.)

[orenbenkiki]

Using GC().c():

• This requires creating an actual instance of GC, which I then just throw away, which is inefficient.

• This implies that c belongs to the instance, when it actually does belong to the class as a whole. This would be confusing to the reader and seems un-Pythonic.

In my specific case, c is actually called special_instance, that is, there is some special instance of the class that I want to obtain. Creating an instance just to destroy it to get another instances is, well, "yuck".

As for "type arguments are only applied on instances, not on classes", that is a nice principle, but, classes are instances (of meta-classes). If I could write:

class M(type, Generic[P]):
    @property
    def c(cls) -> P:
        ...

class G(Generic[P], metaclass=M[P]):
    ...

class GC(G[C]):
    ...

GC.c  # Has type `C`

Then I would be a very happy camper - and I would only be applying type arguments to instances here :-)

However this doesn't work, as meta-classes are not allowed to be generic.

BTW, I could work around my problem in another (less convenient) way, if I could write:

def foo(t: type) -> t:
    ...

That is, I could live with writing x = special_instance_of(GC) instead of GC.special_instance; but mypy doesn't allow for this, either.

I'm pretty much out of workaround ideas at this point...

[gnprice]

That is, I could live with writing x = special_instance_of(GC) instead of GC.special_instance

If I understand you right, Type[C] will give you what you want:
https://www.python.org/dev/peps/pep-0484/#the-type-of-class-objects

Your example would look like

def foo(t: Type[C]) -> C:
    ...

where C is a type variable.

[gvanrossum]

@orenbenkiki You might want to tone down your language a bit.

[JukkaL]

Mypy should perhaps reject using a class type variable in a class method (or a static method). Type variables are basically meant to be used in an instance context. If you'd need to use a type variable in a class method, you'd have to define another type variable. However, this wouldn't help with the original example.

To support the original example properly, it seems to me that we'd need something like a 'class-level' type variable (vs. the currently supported 'instance-level' type variables). The value of the variable would be fixed for all instances of a particular class. However I'm not sure whether something like this would actually be a reasonable (or useful) thing to have.

Before we even start seriously thinking about a new type system feature we need better justification and real-world examples, though.

[gvanrossum]

Yeah, we should start by rejecting such class/static methods, since the type variable is only specified by instantiation. Agree it's too early to consider a new feature (IMO this has come up before but usually due to some confusion on the user's part -- better error messages could help them without new type system features).

[orenbenkiki]

@gnprice : Somehow I missed Type[...] - my bad. I'll try it out. Thanks!

[orenbenkiki]

@gvanrossum : I can see why associating types with classes, as opposed to instances, would be a significant feature. Allowing meta-classes to be generic might side step this (as classes are instances of their meta-class), but I'm not familiar enough with the code to tell - perhaps both features are basically the same, implementation-wise.

The justification is that a natural and correct programming idiom is currently being rejected.

Specifically, the following are currently rejected:

class G(Generic[P]):

    cp: P  # Class property

    @staticmethod
    def sm() -> P:  # Static method
        ...

    @classmethod
    def cm(cls) -> P:  # Class method
        ...

Having generic class properties/methods that use the type parameters seems like a natural thing to expect, when coming from languages with templates and/or generic types. And, of course, the code runs just fine.

That said... whether this idiom is common enough to make this high-priority compared to other features is a separate issue. If you are saying that more justification is needed to push this to the top of the list, that's perfectly reasonable.

I suggest keeping this issue open for a while and seeing whether people upvote it.

[gvanrossum]

Seriously? You "thumbs up" your own comment?

[orenbenkiki]

AFAIK github doesn't allow reactions to an issue, just to a specific comment. Placing the 1st one makes it easier for people to add their own up-vote if they so choose (a button appears right next to it). At least that's the way I saw this being done in other projects. If a different system is used in mypy, I'll be happy to follow it.

[elazarg]

I think this is duplicate of #1337

[ilevkivskyi]

Hm, indeed looks like a duplicate of #1337