Mypy: type refinement in function call

[dfroger]

This is ok for mypy:

from typing import Union

def foo(x: Union[int, str]):
    if isinstance(x, int):
        return x + 10

but how can I make it work if the isinstance test is in a function call?

from typing import Union

def check(x: Union[int, str]) -> bool:
    return isinstance(x, int)

def foo(x: Union[int, str]):
    if check(x):
        return x + 10

I get:

foo.py:8: error: Unsupported operand types for + ("Union[int, str]" and "int")

[JelleZijlstra]

This could perhaps be done with literal types in the future, something like this:

@overload
def check(x: int) -> Literal[True]: ...
@overload
def check(x: str) -> Literal[False]: ...

[gvanrossum]

That's pretty verbose though..

On Tue, Jun 12, 2018, 08:08 Jelle Zijlstra [e-mail] wrote:

This could perhaps be done with literal types in the future, something
like this:

@overload
def check(x: int) -> Literal[True]: ...
@overload
def check(x: str) -> Literal[False]: ...

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/python/mypy/issues/5206#issuecomment-396624167, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACwrMvKHVxoXmkkO8mpf8aeuMzX0ZZmTks5t79lfgaJpZM4UkjNs
.

[gwerbin]

I've had this issue recently too. I definitely think Mypy would benefit from some way to refine types by following conditional paths. Unfortunately implementing that kind of logic is well beyond my expertise.

AFAIK you would still have the problem if you wrote if True instead of if check(x), so having literal return types probably won't fix this specific problem.

[elazarg]

Related: #1203, #2357 #4063, #3062

[ilevkivskyi]

Here I copy the relevant link from one of the above issues I just closed https://github.com/Microsoft/TypeScript/issues/1007

[dbrgn]

I've found the TypeScript type guards to be a very useful concept, especially when dealing with structured data coming in over the network (e.g. a JSON API).

This is the TS syntax:

function isNumber(x: any): x is number {
    return typeof x === "number";
}
function isFish(pet: Fish | Bird): pet is Fish {
    return (pet as Fish).swim !== undefined;
}

That syntax is probably not feasible in Python (I don't think pet is Fish can be used as return type annotation, but can't find the definition for TYPE_COMMENT in the grammar).

Maybe a placeholder type (def is_fish(pet: Union[Fish, Bird]): TypeAssertion) could be used, which would tell the type checker to "inline" the type hint function when determining the type refinement in an if statement (or another similar control structure)?

Maybe TypeAssertion[pet=Fish] could work as well? Depends on the grammar of course.

[JelleZijlstra]

Grammatically type annotations are just expressions, so something like this:

def is_number(x: object) -> x is int:
    pass

is grammatically valid. However, it has some possible problems:

• The name x is obviously not necessarily defined at this point, so if you try this right now you'll likely get NameError: x is not defined. This is not a problem if from __future__ import annotations is on, because then Python will not attempt to evaluate the annotation.
• The is operator cannot be overloaded at runtime, so it's going to be hard to provide runtime introspection of this sort of annotation, although again this is mitigated by from __future__ import annotations.

In contrast, your TypeAssertion[x=int] syntax is invalid at runtime, so it would be much harder to add than -> x is int.

In any case, the bigger difficulty for getting something like this into mypy is going to be first convincing people that it's a good idea and second getting an implementation written and merged into mypy. Syntax is easy. :)

[dbrgn]

first convincing people that it's a good idea

What would be a reason against this (besides implementation complexity)? It considerably helps when dealing with external data, because it combines runtime validation and static type checks.

[DustinWehr]

@JelleZijlstra would implementation be easy too? Could you not typecheck a call is_number(e) to

def is_number(x) -> x is int:  # or whatever syntax makes the cut
    pass

by just virtually (or literally) replacing it with (is_number(e) and isinstance(e,int))?

[DustinWehr]

How about a bundled do-nothing decorator for the syntax?

@mypypred(t=int)
def is_number(x) -> bool:
    pass

[lubieowoce]

by just virtually (or literally) replacing it with (is_number(e) and isinstance(e,int))

@DustinWehr a literal isinstance() wouldn't work bc it can't be used with e.g. generic types (as they're erased at runtime). but i guess the machinery used for cast() could be reused here

[DustinWehr]

@lubieowoce I would've guessed the static analyzer's use of isinstance isn't encumbered by that runtime constraint. I haven't looked at the code to see the approach taken though. It seems to understand at least simple boolean combinations of isinstance expressions (e.g. isinstance(x,T) and not isinstance(x,T2)). Maybe they're doing a brute force enumeration of the possible type narrowings along each path (approach I would implement first), or something more complex like symbolic execution.

[DustinWehr]

I see that this was already suggested in #7870 by @gantsevdenis. Good idea imo.

How about a bundled do-nothing decorator for the syntax?

@mypypred(t=int)
def is_number(x) -> bool:
    pass

[lubieowoce]

@DustinWehr i guess i was being a bit pedantic :) btw if you want to take a look, the logic for isinstance checks seems to start in mypy.checker.find_isinstance_check: https://github.com/python/mypy/blob/9101707bd0c96624d09cb31fe573d7e25c89a35c/mypy/checker.py#L3798-L3814

Maybe they're doing a brute force enumeration of the possible type narrowings along each path

i think that's indeed the case if i'm understanding you correctly. in general

if isinstance (x, A):
  # yes
  ...
else:
  # no
  ...

seems to be handled with two different typings of variables, one for the yes branch and one for the no branch. if the condition is isinstance(x, A) or isinstance(x, B), the two "type narrowings" given by the isinstance checks are combined, giving (more or less) x: Union[A, B] etc. see: https://github.com/python/mypy/blob/9101707bd0c96624d09cb31fe573d7e25c89a35c/mypy/checker.py#L3953-L3971

i imagine that a hacky proof of concept impl of type guards could simply add an elif is_typeguard_expr(node): .... i might try adding it later

[vbraun]

Repeating the name x of the argument of the type guard isn't really necessary, the syntax could just be
~
def is_integer(x: Any) -> IsInstance[int] # or TypeAssertion[int] or TypeGuard[int]
pass
~
On a related note, a still-open typescript feature request for is to extend type guards to multiple arguments. It would be nice to cover that use case as well, e.g.
~
def is_integer_and_string(x: Any, y: Any) -> IsInstance[int, str]
pass
~