Pylint: Incorrect no-member for overridden alternate constructor classmethods

[mgilson]

There are lots of no-member warnings and I'm not sure if this one has been reported yet or if it is related to one of the others, so feel free to close as a dupe....

Steps to reproduce

1. Run pylint on the following code:

"""Module docstring."""

class Foo(object):
    """Foo class."""

    @classmethod
    def bar_clsmethod(cls):
        """Bar method."""
        return cls()

    def a_public_method(self):
        """A public method."""
        print('Hello world.  I am:', self)


class Bar(Foo):
    """Bar class."""

    @classmethod
    def bar_clsmethod(cls):
        instance = super(Bar, cls).bar_clsmethod()
        instance.a_public_method()
        instance.method()  # no-member
        return instance

    def method(self):
        """Say hello to the world."""
        print('Hello World', self)

Current behavior

You'll get a no-member error at the line instance.method() # no-member because pylint thinks that instance is of type Foo. However, since bar_clsmethod is a class method that returns an instance of the class and it is being called from Bar, then instance is an instance of Bar.

Expected behavior

I expect there to be no "no-member" warnings since the classes/instances all have the members that I'm using.

pylint --version output

No config file found, using default configuration
pylint 1.5.6,
astroid 1.4.6
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)]

[PCManticore]

Thanks for reporting this, will look into it.

[samwebster]

Some code with a similar problem and probably the same root cause:

class ParentClass(object):
    def __call__(self):
        return self

class ChildClass(ParentClass):
    def method_that_cannot_be_found(self):
        return 1

def pylint_error():
    instance = ChildClass()()
    # Instance of 'ParentClass' has no 'method_that_cannot_be_found' member (no-member)
    instance.method_that_cannot_be_found()

[mlfz]

I am also running into this problem. We have a series of class hierarchies that has factory methods for generating instances (i.e. similar to @mgilson's example above). Pylint interprets subclass instances generated this way as instance of the superclass, and therefore lacking all the attributes of the subclass. This makes pylint extremely noisy with false-positives for people who uses this pattern.

This pattern is fairly standard for people who writes classes with multiple constructors. (See, e.g. http://stackoverflow.com/a/682545/3658396). Ironically, we want to migrate to this pattern because our old pattern tended to render static analysis tool useless. So we switched to have more useful pylint output.

I apologize for not adding much to the investigation. I really would like to stress how important this particular false positive is. I have been digging into astroid a little bit, but it's been slow-going.

[PCManticore]

Okay, I can increase the priority for this one and trying to tackle "soon". The definition of soon here varies, as soon as we finish what we have in plan for the next major release (Python 3.6 support and some other changes).

[mlfz]

Thanks! BTW, I have found this only affect class constructor that are overloaded by subclass and called using super:

class Base(object):
    @classmethod
    def make(cls):
        return cls()

    make2 = make

class Derv(Base):
    @classmethod
    def make(cls):
        return super(Derv, cls).make()

    def __init__(self):
        self.foo = '.foo is a member of {}'.format(self.__class__.__name__)

if __name__ == '__main__':
    inst = Derv.make()
    print inst.foo # pylint: no-member

    inst = Derv.make2()
    print inst.foo # pylint: OK

So there is at least a workaround: avoid using super.

[mplanchard]

Just wanted to chime in that this also occurs with Python 3 style super(), sans arguments, and also that usage of sub-class methods in instances generated from the class methods are flagged, depending (weirdly) on whether or not the instance of the base class is instantiated during the return statement or prior.

So, as an example:

class A:

    @classmethod
    def new(cls) -> 'A':
        inst = cls()
        return inst

class B(A):

    def print_foo(self):
        print('foo')

    @classmethod
    def new(cls) -> 'B':
        inst = super().new()
        inst.print_foo()  # E1101
        return inst


b_inst = B.new()
b_inst.print_foo()  # E1101

But, if the instance is instantiated during the return:

class A:

    @classmethod
    def new(cls) -> 'A':
        return cls()

class B(A):

    def print_foo(self):
        print('foo')

    @classmethod
    def new(cls) -> 'B':
        inst = super().new()
        inst.print_foo()  # E1101
        return inst


b_inst = B.new()
b_inst.print_foo()  # No E1101!!

Edit: removed empty __init__ methods, since they didn't contribute to the example

Edit2: also, almost certainly related, accessing private members defined on the parent or the child class on instances created in the class methods of the child class warn about access to private members (W0212)

[mplanchard]

Any idea as to whether this is something that is going to be worked on in the near future? E1101 is generally a really useful inspection, so I'd rather not turn it off for the entire codebase if I can avoid it.

[PCManticore]

@mplanchard Most probably not in the near future, e.g not in the following months. Pylint's development is slow lately mostly because I lack the time for open source right now.

[mplanchard]

Thanks for the reply! I totally understand. I pulled down the repo locally
and was trying to figure out the root cause so I could make a PR. It looks
like the node is lacking the class in the parents of the return from
super().

Last time I had time to mess with it, I was trying to figure it whether
that was an astroid bug or a pylint bug, leaning towards astroid but still
trying to figure out how that all works. I'm going to keep at it when I
have time, but since you know it better, does it sound like an astroid bug
to you?

On Thu, Oct 12, 2017 at 2:36 AM Claudiu Popa [e-mail]
wrote:

@mplanchard https://github.com/mplanchard Most probably not in the near
future, e.g in the following months. Pylint's development is slow lately
mostly because I lack the time for open source right now.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PyCQA/pylint/issues/981#issuecomment-336046392, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AGcI4SNEhL_3Fungzol61rMlWaomxBgGks5srcGIgaJpZM4JGdhj
.

[PCManticore]

I think the problems comes from astroid. You should check what exactly the inference engine is inferring and adapt it as needed. From what I see, the minimum piece of code that reproduces this is:

from astroid import extract_node
n = extract_node('''
class A:

    @classmethod
    def new(cls) -> 'A':
        return cls()

class B(A):
   pass

B.new()
''')
print(next(n.infer()))

This prints A, while it should print B, so the cls context is not propagated correctly when .new() is inferred.

[mplanchard]

Thanks for the direction! Hopefully I'll have some time to dig in this
weekend.

On Thu, Oct 12, 2017 at 7:58 AM Claudiu Popa [e-mail]
wrote:

I think the problems comes from astroid. You should check what exactly the
inference engine is inferring and adapt it as needed. From what I see, the
minimum piece of code that reproduces this is:

from astroid import extract_node
n = extract_node('''
class A:

@classmethod
def new(cls) -> 'A':
    return cls()

class B(A):
pass

B.new()
''')
print(next(n.infer()))

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PyCQA/pylint/issues/981#issuecomment-336126660, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AGcI4S1L7tQe-F5CKMS1pE6xkJMCUn2dks5srg0MgaJpZM4JGdhj
.

[dmulter]

+1

[mplanchard]

Turns out I never had time to look into this, but I did run into it again today and found this issue, which I had forgotten about.

The workaround I've come up with for now looks like this:

class Base:
    @classmethod
    def new(cls):
        return cls()


class Child(Base):
    @classmethod
    def new(cls):
        return Base.__dict__["new"].__func__(cls)

    def foo(self):
        pass


c = Child.new()
c.foo()

pylint can't figure out the return type, so no errors. From there, adding type hints allows good editor suggestions and autocomplete.

Edit: Python 3.7.4, pylint 2.3.1, astroid 2.2.5

[drmaples]

🙏 please 2 fix

[PCManticore]

@drmaples We'll get to it at some point, unless you have some time to contribute? 😄 Happy to help you navigate around the codebase in figuring this out. Given we're almost at 500 issues and less and less volunteer time to dedicate to this project, we can take any help any of you can offer.