Mypy: Idiomatic isinstance(x, List[T])

[llchan]

Is there an idiomatic way to check for isinstance(x, List[T]) such that mypy infers the right type downstream?

Example code looks something like this:

from typing import List, Union


class A(object):
    def foo(self, xs: Union[List[int], List[str]]) -> List[float]:
        if isinstance(xs, list):
            if all(isinstance(x, int) for x in xs):
                return self.bar(xs)  # type error here
            elif all(isinstance(x, str) for x in xs):
                return self.baz(xs)  # type error here
        raise TypeError('unhandled type')

    def bar(self, xs: List[int]) -> List[float]:
        return [0.0 for _ in xs]

    def baz(self, xs: List[str]) -> List[float]:
        return [0.0 for _ in xs]

If not, can we teach mypy this ability?

EDIT: It appears that if we change the signature to def foo(self, xs: Any) -> List[float]: ..., it's able to infer the right type for xs, so I think it already knows how to do this in some scenarios.

[gvanrossum]

No, typing xs: Any just makes the intermediate types also Any.

I don't think this is a reasonable feature to request -- we don't want an isinstance() call to take O(N) time in the size of the list.

[llchan]

I think maybe I wasn't too clear on what I'm asking for. The code I'm annotating already has the instance(xs, list) and the O(N) all(isinstance(x, int) for x in xs) checks, so I'm already paying for that (this is fine because N is small). What I'm wondering is whether mypy can use the all(isinstance(x, T) for x in xs) idiom to tighten the List[Any] into a List[T], or if there's a different way to coax it into recognizing that type (e.g. with an assert).

In theory a manual cast call could work, but this code needs to work with PY2 in an environment without the typing module, so I think it'd introduce a lot of noise that adds no information.

if isinstance(xs, list):
    if all(isinstance(x, int) for x in xs):
        if MYPY:
            typed_xs = cast(List[int], xs)
        else:
            typed_xs = xs
        return self.bar(typed_xs)

If there's a comment-based way to express the cast, I can work with that, but given that the information is already there, I think it'd be cleaner to have mypy infer the type.

Btw, you can ignore my edit. Didn't realize the Any type bypassed nearly all checks even in strict mode, I thought it was just basically a Union[<everything>], but turns out it's more like a # type: ignore with auto-propagation. I see there are other tickets regarding Any so I'll save discussion for those threads.

[ilevkivskyi]

@llchan

What I'm wondering is whether mypy can use the all(isinstance(x, T) for x in xs) idiom to tighten the List[Any] into a List[T].

It is not super-difficult to teach mypy understand this, but still this will require some effort. This situation is not that frequent, and one can use a cast, so I am not sure we should do this (of course we will consider a PR if someone decides to do this).

[intgr]

I don't think this is a reasonable feature to request -- we don't want an isinstance() call to take O(N) time in the size of the list.

I understand it's a different feature request. But I, for one, would very much like a helper function that's capable of checking more complex nested types, such as List[T], at runtime. It would be OK if it's a new function distinct from isinstance. And it would make implementing this request quite easy in Mypy.

Compare:

check_type(val, List[str])
# vs
isinstance(val, list) and all(isinstance(x, str) for x in val)

I doubt anyone would say they prefer the 2nd form?

But where one would put that function is far from clear. I guess it can be a PyPI package just as well. Or maybe such a thing already exists and I'm not aware of it?

[JukkaL]

I agree that this is an interesting use case, for example for validating JSON data.

Non-trivial runtime type checking functionality probably belongs to another PyPI package. Mypy might include a plugin to integrate with it. I remember having seen some PyPI packages that seemed relevant but I can't remember their names.

[intgr]

I found the pytypes package whose is_of_type() function does this. All that's needed then is a Mypy plugin.

[rocknamx8]

I found the pytypes package whose is_of_type() function does this. All that's needed then is a Mypy plugin.

This function will lead to bad performance when I try to check the type of a large object.
e.g. an object of the type of List[Tuple[str, int]] with large length.