Mypy: Infer attributes from __new__

[JukkaL]

In this example (from #982) we define an attribute via assignment in __new__:

class C(object):
    def __new__(cls, foo=None):
        obj = object.__new__(cls)
        obj.foo = foo
        return obj

x = C(foo=12)
print(x.foo)

Currently mypy doesn't recognize attribute foo and complains about x.foo. To implement this, we could do these things:

• Infer the fact that obj is something similar to self because of the way is created via object.__new__. This should happen during semantic analysis so that this will work in unannotated method as well.
• Infer attributes from assignments via obj because it is classified as similar to self.
• Also recognize object.__new__ calls via super(...).

[JukkaL]

This is follow-up to #982.

[JukkaL]

Decreased priority since this doesn't seem like users often struggle with this -- it's easy enough to work around by inserting type annotations.

[toolforger]

One use case is NamedTuple. It's turning mypy into a noise generator for me.
Not sure how this data point affects priorization.

[JelleZijlstra]

@toolforger how are you using NamedTuple? I don't think the example from the original post in this issue is applicable to NamedTuple.

[toolforger]

Using a NamedTuple subclass, subclass it with a __new__ override to provide standard parameters.
Details in #5194, which is a duplicate of #1279.

[ilevkivskyi]

Using a NamedTuple subclass, subclass it with a __new__ override to provide standard parameters.

Use instead:

class Foo(NamedTuple):
    x: int
    y: int = 0

Foo(1)  # OK, same as Foo(1, 0)
Foo(1, 2)  # OK

works on Python 3.6+. If you are on Python 2, then wait for this issue to be solved.

[toolforger]

@ilevkivskyi your advice works for direct subclasses. My use case is for an indirect subclass. (I already had to move all fields up into the root class because NamedTuples don't properly support subclassing, but I cannot use frozen dataclasses until 3.7 comes out for my distro. I'm on the verge of ripping out mypy again, or switching from Python to Kotlin.)

[gvanrossum]

@toolforger I hear your frustration, but please don't take it out on us. We have a lot on our plate and we're only a small team (half of us are volunteers). This will eventually get fixed. (If you want to help by submitting a PR that might speed it up of course.)

[toolforger]

Heh. Getting advice that's inapplicable because the person didn't really understand the situation, after spending weeks of my free time on a dozen or so approaches - well what can I say, it was just the last straw.

Won't work on Python, sorry:

• I'd be just another volunteer;
• given my tribute to the ecosystem already (two years on SymPy I believe);
• deep philosophical differences about what a language should be, I'd have to bite my tongue constantly;
• already other projects on my desk anyway (I'm using Python to get something off my desk to I can work on the tool for the project I'm really interested in - I guess this rings with many people here ;-) ).

[gvanrossum]

Can we raise the priority here? I encountered several cases of this in S (links provided at request). What's worse, I also get an error on the super __new__ call:

class C(str):
    def __new__(cls) -> C:
        self: C = str.__new__(cls, 'foo')  # E: Too many arguments for "__new__" of "object"
        self.foo = 0  # E:  "C" has no attribute "foo"
        return self
C().foo  # E: "C" has no attribute "foo"

The workaround is redundant class-level attribute declarations.

[ilevkivskyi]

The super error is unrelated, it is just typeshed problem, str doesn't have __new__ there.

[gvanrossum]

The super error is unrelated, it is just typeshed problem, str doesn't
have __new__ there.

Okay, but the main problem is the need for duplication for every attribute.

[JukkaL]

Updated priority to high.