Mypy: crash when same file appears with different module names

[msullivan]

This bug found via a report from @eddieschoute

mypy crashes on the following cmdline test:

[case testDuplicateModules]
# cmd: mypy src
[file mypy.ini]
[[mypy]
mypy_path = src
[file src/__init__.py]
[file src/a.py]
import foo.bar
[file src/foo/__init__.py]
[file src/foo/bar.py]
1+'x'
[out]
-- The below would be what we produced if it didn't crash, but really we should
-- produce a useful error.
src/foo/bar.py:1: error: Unsupported operand types for + ("int" and "str")
src/foo/bar.py:1: error: Unsupported operand types for + ("int" and "str")

This crashes with

File "/home/msullivan/src/mypy/mypy/errors.py", line 264, in _add_error_info
assert file not in self.flushed_files

Here, the "canonical" module name for src/foo/bar.py is src.foo.bar. But if src is on the search path, both python and mypy will also happily let it have the name foo.bar.
In the test above, the inclusion on the command line results in bar.py being processed as src.foo.bar while the import for foo.bar in a.py, combined with src being on the mypy_path, also leads to it being processed as foo.bar. An error is generated in both, which trips an assert in Errors as we try to add a new error to a file that has already had its error messages flushed to the user.

One simple fix would be to track flushed_files by module id instead of file name, but it would probably be better to generate an error when a file appears multiple times with different module names, since it is unlikely to be intended?

[gvanrossum]

Yeah, I think when we detect this we should give an error. As you said this is also an issue in Python itself and there are some ideas for detecting and rejecting it there as well. (Though sadly I cannot find the thread about it.)

[ilevkivskyi]

Raising priority to high, since this is a crash, and it appeared several times.

[canassa]

Hello,

I created a small project that replicates this error:

https://github.com/canassa/mypy-error

What I don't understand is that I saw on some other comments that "this errors only happens when doing something nonsensical" like adding a package to mypy_path. But the way that I have configured my project, I have no other option besides having the src directory on the mypy_path.

Is there any workaround for this? This issue is blocking me from using mypy

[JukkaL]

Even if we fix the bug, it's not clear that mypy would support this use case. It would probably produce an error instead of crashing.

Why have you configured your project like this? Why can't you remove the __init__.py file from your src directory?

[canassa]

You are correct, that __ini__.py shouldn't be there, I guess that's the nonsensical part 😁

I wasn't aware that's what was causing the crash. Thanks! that solves my problem