Mypy: Master (0.530-dev) crash on tuple unpacking

[ambv]

Repro:

from typing import Tuple, TypeVar

_T1 = TypeVar('_T1')

def make_tuple(elem: _T1) -> Tuple[_T1]:
    return (elem,)

def main() -> None:
    ((a, b),) = make_tuple((1, 2))

produces:

Traceback (most recent call last):
  File "/Users/ambv/.venvs/mypy/bin/mypy", line 11, in <module>
    load_entry_point('mypy', 'console_scripts', 'mypy')()
  File "mypy/__main__.py", line 7, in console_entry
    main(None)
  File "mypy/main.py", line 50, in main
    res = type_check_only(sources, bin_dir, options)
  File "mypy/main.py", line 103, in type_check_only
    options=options)
  File "mypy/build.py", line 198, in build
    graph = dispatch(sources, manager)
  File "mypy/build.py", line 1841, in dispatch
    process_graph(graph, manager)
  File "mypy/build.py", line 2091, in process_graph
    process_stale_scc(graph, scc, manager)
  File "mypy/build.py", line 2194, in process_stale_scc
    graph[id].type_check_first_pass()
  File "mypy/build.py", line 1753, in type_check_first_pass
    self.type_checker.check_first_pass()
  File "mypy/checker.py", line 194, in check_first_pass
    self.accept(d)
  File "mypy/checker.py", line 282, in accept
    stmt.accept(self)
  File "mypy/nodes.py", line 519, in accept
    return visitor.visit_func_def(self)
  File "mypy/checker.py", line 530, in visit_func_def
    self.check_func_item(defn, name=defn.name())
  File "mypy/checker.py", line 590, in check_func_item
    self.check_func_def(defn, typ, name)
  File "mypy/checker.py", line 750, in check_func_def
    self.accept(item.body)
  File "mypy/checker.py", line 282, in accept
    stmt.accept(self)
  File "mypy/nodes.py", line 773, in accept
    return visitor.visit_block(self)
  File "mypy/checker.py", line 1304, in visit_block
    self.accept(s)
  File "mypy/checker.py", line 282, in accept
    stmt.accept(self)
  File "mypy/nodes.py", line 817, in accept
    return visitor.visit_assignment_stmt(self)
  File "mypy/checker.py", line 1311, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "mypy/checker.py", line 1339, in check_assignment
    infer_lvalue_type)
  File "mypy/checker.py", line 1589, in check_assignment_to_multiple_lvalues
    self.check_multi_assignment(lvalues, rvalue, context, infer_lvalue_type)
  File "mypy/checker.py", line 1630, in check_multi_assignment
    context, undefined_rvalue, infer_lvalue_type)
  File "mypy/checker.py", line 1650, in check_multi_assignment_from_tuple
    reinferred_rvalue_type = self.expr_checker.accept(rvalue, lvalue_type)
  File "mypy/checkexpr.py", line 2372, in accept
    typ = node.accept(self)
  File "mypy/nodes.py", line 1256, in accept
    return visitor.visit_call_expr(self)
  File "mypy/checkexpr.py", line 266, in visit_call_expr
    ret_type = self.check_call_expr_with_callee_type(callee_type, e, fullname, object_type)
  File "mypy/checkexpr.py", line 503, in check_call_expr_with_callee_type
    object_type=object_type)[0]
  File "mypy/checkexpr.py", line 563, in check_call
    callee, context)
  File "mypy/checkexpr.py", line 752, in infer_function_type_arguments_using_context
    erased_ctx = replace_meta_vars(ctx, ErasedType())
  File "mypy/erasetype.py", line 94, in replace_meta_vars
    return t.accept(TypeVarEraser(lambda id: id.is_meta_var(), target_type))
  File "mypy/types.py", line 982, in accept
    return visitor.visit_tuple_type(self)
  File "mypy/types.py", line 1554, in visit_tuple_type
    return TupleType(self.translate_types(t.items),
  File "mypy/types.py", line 1574, in translate_types
    return [t.accept(self) for t in types]
  File "mypy/types.py", line 1574, in <listcomp>
    return [t.accept(self) for t in types]
  File "mypy/types.py", line 982, in accept
    return visitor.visit_tuple_type(self)
  File "mypy/types.py", line 1554, in visit_tuple_type
    return TupleType(self.translate_types(t.items),
  File "mypy/types.py", line 1574, in translate_types
    return [t.accept(self) for t in types]
  File "mypy/types.py", line 1574, in <listcomp>
    return [t.accept(self) for t in types]
AttributeError: 'NoneType' object has no attribute 'accept'
/tmp/test.py:13: : note: use --pdb to drop into pdb

[carljm]

This was introduced sometime since Aug 11; we've been using a master revision of mypy from that date up until now without this error, and saw this error when attempting to update to eec674584316 today.

[ethanhs]

Confirmed as a regression on master.

[ambv]

I modified the example to make it way shorter.

[gvanrossum]

Can the OP bisect this to pinpoint to a precise commit/PR?

[gvanrossum]

I don't know where you get the Aug 11 date from. The same crash happens with v0.4.6, released Nov last year. So I don't think this counts as a regression and that makes the fix less urgent.

[carljm]

I'm seeing the same thing in attempting to bisect. I think this changed in @ambv 's simplification of the repro case. IOW I believe the root cause is an old bug in mypy, and a typeshed change since Aug 11 triggered the bug on our actual code using asyncio.gather.

[carljm]

The original longer repro case:

import asyncio
from typing import Tuple

async def get_location(arg: str) -> Tuple[int, int]:
    return 1, 2

async def main() -> None:
    (
        (a_x, a_y),
    ) = await asyncio.gather(
        get_location('start'),
    )
    print(a_x, a_y)

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

bisects to #3967. Before that it typechecks clean.

Working on bisecting it within typeshed.

[carljm]

Not surprisingly, the recent regression bisects to https://github.com/python/typeshed/pull/1550

[gvanrossum]

Thanks for confirming my hunch that the simplified repro was worth a separate message rather than editing the OP. :-)

So it seems that the actual bug is old and the improved gather() stub is correct. What do you suggest we do now? I'd like to see it fixed, of course, but even with the simplified repro that may not be so easy. Is it worth cherry-picking into the release branch for 0.530? Maybe the fix will be simple and obviously correct, but for a bug that hid this long I kind of doubt it, and I'm out of energy tonight to double down on this.

Maybe @JukkaL or @ilevkivskyi have a fresh insight when they get up.

[ethanhs]

I noticed that this does not happen for list types, which led me to believe it is part of the special tuple checking code. Stepping through the code, we end up here. Essentially, we usually give the rvalue type to definitions, which is the correct thing to do. However, when checking nested lvalue tuple types, we check recursively, so the new definition types don't have an rvalue to inherit, causing their type to be None, and things breaking.

[ilevkivskyi]

Yes, this is related to https://github.com/python/mypy/issues/3966, I actually already have a fix on my last strict optional branch. I will add a test and make a PR now.

[ilevkivskyi]

Hm, although the crashes are fixed on my branch, the revealed types of a and b in the example are Any. I think we can give a more precise type here.

[gvanrossum]

@ambv, @carljm -- do you need this cherry-picked into the 0.530 release or can you work around it? (I imagine it's easy to work around using a temporary variable.)

[carljm]

@gvanrossum We don't need it cherry-picked into the release because we don't use releases. Thanks for the quick fix and merge to master!