Mypy: Module X has no attribute Y when using star imports

[nicktimko]

mypy seems to emit a spurious warning that it can't find an attribute of a module that should be imported via a star import and where another import refers to it.

Input

$ head mypackage/*

• ==> mypackage/__init__.py <==

from mypackage.submod1 import *     # * -> HELLO and error goes away
from mypackage.submod2 import greet # comment out and error goes away
__all__ = ['HELLO', 'greet'] # does nothing

• ==> mypackage/submod1.py <==

__all__ = ['HELLO']
HELLO = 'hello there'

• ==> mypackage/submod2.py <==

from mypackage import HELLO
__all__ = ['greet']
def greet():
    print(HELLO)

Current Output

$ mypy mypackage
mypackage/submod2.py:1: error: Module 'mypackage' has no attribute 'HELLO'
$ python -c "import mypackage;mypackage.greet()" # Python has no problems
hello there
$ python --version; mypy --version
Python 3.6.4
mypy 0.590

(haven't checked with master, but I couldn't find any recently closed issues that seemed to talk about something like this)

Expected Output

No error from mypy.

As per the comments in the __init__.py, if some of the imports are tweaked, then mypy is happy. It doesn't seem to be sated by an __all__ to help the star import. I don't quite understand why not importing mypackage.submod2 causes the error to go away as well; it's not like there's a circular import between 1 and 2 where the names in the module aren't defined before it imports (i.e. if you moved the ...import greet line above the other, Python would choke)

[jacebrowning]

I can still reproduce this with mypy 0.600.

[JelleZijlstra]

I looked into this for a bit. Here is a test that reproduces the problem: https://github.com/JelleZijlstra/mypy/commit/6b124e2578e73cad80444833ce0b43ab481a40c8. The problem goes away when one uses from b import HELLO instead of from b import *.

I think what's going on here is as follows:

• semantic analyzer pass 1 adds names from import_from, but not import_alll (because at that point we don't know anything about other modules yet)
• semantic analyzer pass 2 adds names from import_all, relying on what was added in pass 1. But if we're trying to import from a module that uses import_all, and pass 2 hasn't run on that module yet, we can't see the names from the import_all, and we get an error.

This could be fixable with an additional semanal pass or something like a callback in pass 1 that runs when the module we're import_alling from is processed, and adds names to the module that is using import_all.

[JelleZijlstra]

My branch https://github.com/python/mypy/compare/master...JelleZijlstra:circularstar has a fix for the original bug now, but it introduces various new bugs. I don't understand for now what's going on in some of those tests. I'll take another look at this later.

[JelleZijlstra]

The reason why my code is buggy is that if there is a circular dependency involving import * (as in this test: https://github.com/python/mypy/commit/cb20eaf2d29b7aeb8560d06bc441d7f97997db70), I end up adding an ImportedName in both modules, so mypy believes the name exists in either both files or neither file.

This doesn't seem easy to solve. Strictly speaking, mypy should follow the same sequence as Python does at import time: start with one file, then if there is an import, switch to the imported file, then return to the original file. But that would require a full rewrite of pass 1 of semantic analysis and probably a broader change in mypy's build pipeline.

Another solution could be to declare that mypy does not support import * in an import cycle and throw an error that makes this explicit.

[gvanrossum]

declare that mypy does not support import * in an import cycle

I like that.

[sebastian-philipp]

not supporting import * is a bit unfortunate. :disappointed:. I guess we can live with it. Needs proper documentation, though.

[gvanrossum]

Clarification: we'd document that we don't support import * in an import
cycle. It seems expendable, since you can add an explicit import where
necessary.

[sebastian-philipp]

ok, the effect I'm seeing here is then caused by something else. thanks!

Click to expand

    #  cat <<EOF > ./mypy.ini
    [mypy]
    strict_optional = True
    no_implicit_optional = True
    ignore_missing_imports = True
    warn_incomplete_stub = True
    check_untyped_defs = True
    show_error_context = True
    EOF
    #  cat test_foo.py 
    from nose.tools import *
    def test_foo():
        eq_(1,1)
    #  nosetests test_foo.py 
    .
    ----------------------------------------------------------------------
    Ran 1 test in 0.000s
    OK
    #  mypy --config-file=mypy.ini test_foo.py
    test_foo.py: note: In function "test_foo":
    test_foo.py:3: error: Name 'eq_' is not defined

[JukkaL]

This might be fixed by the new semantic analyzer (experimental version available on GitHub mypy master by using --new-semantic-analyzer). It generally improves the processing of imports within import cycles.

[chadrik]

I have the same issue. I tried with 0.710+dev.f0491839adf2db2da74768b1d01f8e244e769f59 using --new-semantic-analyzer and it did not fix the problem.

[chadrik]

If you do decide to drop support for import * in an import cycle, it would be great to at least inform the user about the source of the cycle. In my case it's not obvious, and it could be (???) caused by imports inside a TYPE_CHECKING block, which I can't remove without invalidating my annotations.

I would really rather mypy did not drop support for import * in a cycle.

[JukkaL]

We are not planning to drop support for import * in import cycles, but there may be specific use cases that are too hard to support.

[JukkaL]

I'm dropping the new-semantic-analyzer label since this doesn't seem specific to it.

[nicktimko]

Would it be any easier to constrain support for import * in a cycle where the * imported module/package defines a (static) __all__?