Mypy: Mypy doesn't identify subclasses within List responses?

[ABitMoreDepth]

I believe this may be a bug.

Reproduction of this error can be seen with the following:

from typing import List, Union

class C:
    pass

class D(C):
    pass

class A:
    def method(self) -> C:
        return C()

class Al:
    def method(self) -> List[C]:
        return [C()]

class B(A):
    def method(self) -> C:
        return D()

class Bl(A):
    def method(self) -> List[C]:
        return [D()]

• What is the actual behavior/output?
  Mypy reports the following error when analysing the above code:

$ mypy temp.py
temp.py:28: error: Return type of "method" incompatible with supertype "A"

• What is the behavior/output you expect?
  I would have expected mypy to correctly understand that return [D()] satisfies the subtype of List[C] given that D() satisfies the type C in the above source.

• What are the versions of mypy and Python you are using?
  Tested this with Python 3.7.2, with mypy version 0.670.

• What are the mypy flags you are using? (For example --strict-optional)
  None, the stock options when calling mypy with no arguments.

From what I can tell, this doesn't appear to be a violation of Liskov's Substitution Principle, but I am happy to be educated. ;)

Thanks for all the work you've done on mypy.

[JelleZijlstra]

Did you mean to write class Bl(Al): in your code? List[C] is definitely incompatible with C.

[ilevkivskyi]

Yes, this has nothing to do with returning [D()], List[C] is not a subtype of C.

[ABitMoreDepth]

Oh dear, this looks so much less coherent now than when I wrote it, this is a simpler variant that should elicit the issue I was hitting, as @JelleZijlstra pointed out:

from typing import List

class BaseReturn:
    pass

class SubclassedReturn(BaseReturn):
    pass

class A:
    def method(self) -> List[BaseReturn]:
        return [BaseReturn()]

class B(A):
    def method(self) -> List[BaseReturn]:
        return [SubclassedReturn()]

When I have code similar to this, I get a mypy error about the return of B.method(), but when you remove the List part of the above equation I see no error's, so it seemed to me to be something to do with the List type that was causing the confusion. Note that I also tried using def method(self) -> List[SubclassedReturn] within B, but that exhibited another error where B.method had a conflicting return type to its parent class.

Hopefully that makes a little more sense, apologies for the confusion above.

[JelleZijlstra]

The code from your second example passes mypy without errors for me.

[ABitMoreDepth]

Thats interesting, nor do I, but looking back at my original source code, I definitely see an error, oddly whilst fiddling about attempting to reproduce, modifying method of B to look like this:

class B(A):
    def method(self) -> List[SubclassedReturn]:
        results = [SubclassedReturn()]  # type: List[SubclassedReturn]
        return  results

I then get the following error:
temp.py:33: error: Return type of "method" incompatible with supertype "A"

Meanwhile, if I modify B's `method to look like this:

class B(A):
    def method(self) -> List[BaseReturn]:
        results = [SubclassedReturn()]  # type: List[SubclassedReturn]
        return  results

Then I get this error from mypy:
temp.py:35: error: Incompatible return value type (got "List[SubclassedReturn]", expected "List[BaseReturn]"),

Which is a bit odd, given the lack of any lint errors from the previous sample code?

Note that I get the same errors from mypy when using 3.6+'s newer type annotation style as well.

What would be the recommended approach here?

[ABitMoreDepth]

@JelleZijlstra Would you mind having a look to see if you see the same errors as my most recent comment? Sorry to pester,

[JelleZijlstra]

@ABitMoreDepth there are two different issues you have to understand.

First, the correct annotation for the method. As you observed, mypy is OK with -> List[BaseReturn], but not with -> List[SubclassedReturn]. See https://mypy.readthedocs.io/en/latest/common_issues.html#invariance-vs-covariance for more on why that is an error. Therefore, the subclass method needs to have -> List[BaseReturn] as an annotation.

Next, what should the implementation of the method look like? You've already observed that return [SubclassedReturn()] works, but putting it in a variable first doesn't. What's going on here is that mypy uses context to figure out what type to infer for the expression [SubclassedReturn()]. If you return it directly, it's being used in a context where it expects a List[BaseReturn], so that's what mypy infers. But if you put it in a variable first, mypy doesn't have that context, so it simply infers List[BaseReturn]. Therefore, one way to get your code working would be to add an explicit annotation: results = [SubclassedReturn()] # type: List[BaseReturn].

[ABitMoreDepth]

Ah... I think I understand, thank you for your patience in helping me to understand whats going on.

Its not a bug at all, its just a feature that requires some more in depth understanding. :)