Pyright: TypeVar does not work as expected

[mmicu]

Hello. I don't understand why random_in_list called in the main doesn't return an error:

import random
from typing import List, TypeVar


T = TypeVar('T')
def random_in_list(l: List[T]) -> T:
    if len(l) == 0:
        raise ValueError('List cannot be empty.')

    return random.choice(l)

if __name__ == '__main__':
    random_in_list([1, 2, 'Hello'])

pyright version: pyright 1.0.27

[erictraut]

There is a bug here, but it's not the bug you've reported :)

The code above should generate no type checking errors. The method random_in_list indicates that it accepts a List[T] where T is an unconstrained and unbound type variable. That means it can accept any type. In this case, you are passing it a variable with type List[Union[int, str]], which matches List[T].

If your intent is to constrain random_in_list to accept only lists of int or float values, you can redefine your type variable as follows:

T = TypeVar('T', int, float)

Now the type checker should complain if you attempt to pass it a value of [1, 2, 'Hello']. However, it currently doesn't. There are two reasons:

1. There's a bug in the type checker's logic that I just fixed.
2. There's a more subtle issue that I haven't yet determined how to work around.

Let me explain 2 in more detail. Consider the following code:

a = [1, 2, 'Hello']    # inferred to be type List[Any]
b = [1, 2, 3]   # inferred to be type List[int]

a += 3.4   # no type check error
b += 3.4   # type check error reported

What type should the type checker infer for a and b? Currently, pyright assumes that if all elements in a list literal statement are the same type, then the list is homogeneous, but if it contains different types, its type element is considered "unknown". So it infers that a is a List[Any] and b is a List[int]. Because of these assumptions, it will flag b += 3.4 as an error because it's attempting to add a float to a List[int]. But it won't flag a += 3.4 as an error because it's OK to add a float to a List[Any].

In your code example, if you constrain T to be an int, it would make sense that the type checker should generate an error for random_in_list([1, 2, 'Hello']), but it won't because the argument type is inferred to be List[Any], which matches against List[int]. I am still thinking about how to solve this without making assumptions that will break lots of other code that makes use of homogenous lists.

[mmicu]

Thank you for the explanation. :)

I thought that TypeVar was used like C++ templates, but I was wrong.

Regarding your example, I think that the assumption that you're doing is right. However, if you can determine all the types for a certain list, int and str for list b, you can assume that you're dealing with a list of unions U, where U = (int, str). In this way, you should be able to report the error when you do b += 3.4 and maybe you'll not break all the code that makes assumptions about homogenous lists.

[erictraut]

The bug should now be fixed in version 1.0.28, which I just published.

I decided that there's no "correct" assumption. A conservative assumption will fail to catch some type-checking errors. A strict assumption will generate false positive errors in some legitimate (although ill-advised) usage of lists and dictionaries. I decided to add two new configuration switches: strictListInference and strictDictionaryInference. You can read about them in the documentation.

Thanks again for the bug report!