Mypy: Mypy doesn't recognize a class as iterable when it implements only `__getitem__`

[akaptur]

The following code runs fine, but mypy produce an error:

from typing import List

class Foo(object):
    def __init__(self):
        self.version = [1, 2, 3]

    def __getitem__(self, k):
        return self.version[k]


def list_foo(f):
    # type: (Foo) -> List[int]
    return list(f)

print list_foo(Foo())

example.py: note: In function "list_foo":
example.py:13: error: No overload variant of "list" matches argument types [example.Foo]
Fail

[gvanrossum]

We should support this. It is part of the Python spec for iter(): https://docs.python.org/3/library/functions.html#iter

Without a second argument, object must be a collection object which supports the iteration protocol (the iter() method), _or it must support the sequence protocol (the_ __getitem__() _method with integer arguments starting at 0)_. [emphasis mine]

[JukkaL]

We currently require an explicit Iterable base class, as mypy doesn't have structural subtyping. A potential quick fix would be to not complain about unimplemented __iter__ in a subclass of Iterable if there is a suitable __getitem__.

A better fix would require structural subtyping (or protocols), but that's going to be harder. And we'd need another special case for __getitem__.

[chadrik]

I ran into this problem with sre_parse.SubPattern using mypy 0.580. Is there an adjustment that still needs to be made to structural subtyping to recognize this, or should the stubs be "fixed" for objects like these?

[ilevkivskyi]

This is not really a mypy problem. If you ask Python at runtime

class A:
    def __iter__(self):
        ...
class B:
    def __getitem__(self):
        ...
isinstance(A(), collections.abc.Iterable)  # True
isinstance(B(), collections.abc.Iterable)  # False

There were long discussions about this but there is still no decision about this. If you control the expected type you can define your own protocol:

class OtherIterable(Protocol[T]):
    def __getitem__(self, index: int) -> T:
        ...
WideIterable = Union[Iterable[T], OtherIterable[T]]

def fun(arg: WideIterable[str]) -> None:
    ...

class One:
    def __iter__(self) -> Iterator[str]:
        ...
class Other:
    def __getitem__(self, index: int) -> str:
        ...

fun(One())  # OK
fun(Other())  # OK

or whatever else you want.

[chadrik]

So what's the solution for sre_parse.parse? To be clear, the problem is that I get false errors in mypy when running the following code:

for pattern in sre_parse.parse('(\w+)(\d+)'):
    print(pattern)

Mypy prints:

error: Iterable expected
error: "SubPattern" has no attribute "__iter__

[JelleZijlstra]

We should probably just lie in the stubs for sre_parse and claim that SubPattern is Iterable.

[pmhahn]

Same problem for Exceptions in Python2:

d = {0: 0}
try:
  d[1]
except KeyError as ex:
  (key,) = ex
# error: 'builtins.KeyError' object is not iterable

I can work-around that by explicitly accessing ex.args, but I still get that error for existing code.
But as exceptions in Python3 no longer work like this, I will have to update that code anyway in the future.

[Recursing]

Has there been any progress on this issue?