Pylint: false positive no-member on numpy imports

[pylint-bot]

Originally reported by: Trey Greer (BitBucket: trey_greer)

---

With pylint 1.0.0, astroid 1.0.0, common 0.60.0, and python 3.3.2 I'm getting no-member errors on all imports from numpy.

#!python
''' demonstrate numpy import false positive '''
import numpy

A1 = numpy.array([1, 2, 3])
print(A1)

gives

#!

No config file found, using default configuration
************* Module test
E:  4, 5: Module 'numpy' has no 'array' member (no-member)

---

• Bitbucket: https://bitbucket.org/logilab/pylint/issue/58

[pylint-bot]

_Original comment by_ Dan Blanchard (BitBucket: daniel_blanchard):

---

I'd like to add that I am also seeing this issue, and it's pretty frustrating if you use numpy a lot.

[pylint-bot]

_Original comment by_ Erik Bjäreholt (BitBucket: ErikBjare, GitHub: @ErikBjare?):

---

Also seeing this issue

[pylint-bot]

_Original comment by_ Alexander Sideropoulos (BitBucket: asiderop, GitHub: @asiderop?):

---

This is really far more general than numpy; the issue is present for any library which dynamically creates definitions, like numpy, zmq, and twisted. So I suggest changing the title of this issue to be more generic.

@sthenault, not sure how you are planning to fix this, but I think a nice approach would be to add an --ignore-module option at the CLI/pylintrc level which is implemented in the same vein as this StackOverflow answer to a similar question. I am willing to code up a pull request if you want.

Cheers.
--ap

[pylint-bot]

_Original comment by_ Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

another solution being to add description for those module to the pylint-brain project, though I agree some way to tell 'suppose this module has everything we try to access to' would be nice for cases where a description isn't available yet. So yes, a patch for the option you propose would be welcome.

[pylint-bot]

_Original comment by_ BitBucket: Juanlu001, GitHub: @Juanlu001?:

---

I work extensively with scientific libraries (which depend on NumPy and the like), and it's still annoying having to use a patched function of PyLint. Besides, I don't feel like recommending it to other scientific Python users because it creates so much noise. Is there a chance a simple option can be implemented to ignore these kind of errors?

[pylint-bot]

_Original comment by_ Mykola Sakhno (BitBucket: mkola):

---

Is it possible to simply dynamically load this modules and than check for member existence?
Create option like
--dynamic-module=numpy
to clarify imports from which modules should be tested dynamically.

[pylint-bot]

_Original comment by_ BitBucket: Corentin_Hamel:

---

I see this bug has been opened for 4 months now, any progress on it?

[pylint-bot]

_Original comment by_ Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

I'm afraid no. At least on my side.

[pylint-bot]

_Original comment by_ Holger Peters (BitBucket: HolgerPeters, GitHub: @HolgerPeters?):

---

I have created a pull request #100 which is a move towards a solution ignoring the member access of certain modules.

[pylint-bot]

_Original comment by_ Antony Lee (BitBucket: anntzer, GitHub: @anntzer?):

---

I prefer Mykola's suggestion, as these modules can generally be imported safely without any side-effect.

[pylint-bot]

_Original comment by_ Holger Peters (BitBucket: HolgerPeters, GitHub: @HolgerPeters?):

---

Both approaches have their merrit. In general, as a programmer, I do not want to check whether import of such a package is side-effect free, so ignoring a module is a valid solution. For packages that I know and trust I would not be opposed to a dynamic solution. Both solutions could exist in parallel since they are solving different situations.

[pylint-bot]

_Original comment by_ Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

Anyway, the best way off all would be to discover what changed since numpy
1.6 that did broke pylint support. And either makes pylint support it or
ask numpy to do it another way.

I suspect some dynamic 'all' computation to come in the way.

[pylint-bot]

_Original comment by_ Sam King (BitBucket: samking, GitHub: @samking?):

---

I suspect that dynamically modifying __all__ broke support.

I have the same issue with App Engine ndb imports. Eg, I get

#!python

Module 'google.appengine.ext.ndb' has no 'StringProperty' member

when I have

#!python

from google.appengine.ext import ndb
ndb.StringProperty()

The App Engine __init__.py is available here. The members are added dynamically here.

[pylint-bot]

_Original comment by_ Andrea Riciputi (BitBucket: mrrech, GitHub: @mrrech?):

---

Found same behaviour even using the standard library:

#!python

import multiprocessing
data_manager = multiprocessing.Manager()
migration_status = data_manager.dict()

gives

#!python

[no-member] Instance of 'SyncManager' has no 'dict' member

Just to let you know.

[pylint-bot]

_Original comment by_ Ignas Anikevicius (BitBucket: gns-ank, GitHub: @gns-ank?):

---

I am also seeing this on Arch for NumPy. I can also see it with flake8 checker, which suggests that pylint is not the only project suffering from this.

[pylint-bot]

_Original comment by_ Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

Probably due to the fact that multiprocessing does this:

#!python

globals().update((name, getattr(context._default_context, name))
                 for name in context._default_context.__all__)
__all__ = context._default_context.__all__

[pylint-bot]

_Original comment by_ alex lord (BitBucket: rawrgulmuffins, GitHub: @rawrgulmuffins?):

---

I'm seeing this behavior in flask extensions.

#!python

from flask.ext.classy import FlaskView
from app import app

class TestView(FlaskView):
    """ """
    def index(self):
        return "hello world"

TestView.register(app)

generates

E: 10, 0: Class 'TestView' has no 'register' member (no-member)

[pylint-bot]

_Original comment by_ Sergio Oller (BitBucket: zeehio, GitHub: @zeehio?):

---

This issue seems to be caused by the use of

#!python

from __future__ import absolute_import

I have written a simple bash script that creates two similar simple python modules named
module1abs and module1rel and two python scripts using members from both modules.

As you can see if you run this (please place the script in an empty directory), when using the module with absolute_import pylint will raise a non-member error.

Knowing that the cause is the use of absolute_import, I believe that this issue will appear more often as more modules start writing code compatible with both python2 and python3 in the same code base.

I imagine the issue can be fixed in checkers/typecheck.py, in the visit_getattr(self, node) method; or maybe inside the astroid module... although I am not familiar enough with the code base nor with pdb to be able to fix it or debug the code.

#!sh

#!/bin/sh
# Pylint-1.2.1 complains about no-member if absolute_import is used
# under some conditions such as using imports with wildcards.
# numpy-1.8 seems to be affected by this.
# numpy-1.6 is not affected.

# Author: Sergio Oller <[email protected]>
# No copyright is kept on this script as it is quite simple

# $pylint --version
# No config file found, using default configuration
# pylint 1.2.1, 
# astroid 1.1.1, common 0.61.0
# Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
# [GCC 4.8.2]

rm -rf module1abs module1rel useabs.py userel.py

# Create a module that uses absolute import (numpy 1.8 does that)
mkdir module1abs
echo "import sys" > module1abs/core.py
echo "from __future__ import absolute_import" > module1abs/__init__.py
echo "from . import core" >> module1abs/__init__.py
echo "from .core import *" >> module1abs/__init__.py
echo "print sys.version" >> module1abs/__init__.py

# Create a module that uses relative import (numpy 1.6 does that)
mkdir module1rel
echo "import sys" > module1rel/core.py
echo "import core" >> module1rel/__init__.py
echo "from core import *" >> module1rel/__init__.py
echo "print sys.version" >> module1rel/__init__.py

# Create a script that uses a module with absolute import:
echo "import module1abs" > useabs.py
echo "print module1abs.sys.version" >> useabs.py

# Create a script that uses the relative import version:
echo "import module1rel" > userel.py
echo "print module1rel.sys.version" >> userel.py

echo ""
echo "Using a module with absolute_import pylint reports an error:"
pylint ./useabs.py | grep "^E:"
#E:  2, 6: Module 'module1abs' has no 'sys' member (no-member)
echo ""
echo "Using a module without absolute_import does not report errors:"
pylint ./userel.py | grep "^E:"

[pylint-bot]

_Original comment by_ Til Birnstiel (BitBucket: birnstiel, GitHub: @birnstiel?):

---

This thread is now more than 10 month old. Now also scipy doesn't work anymore (tried from scipy.optimize import curve_fit). If very common libraries like numpy and scipy or even standard library packages like multiprocessing can't be handled by pylint anymore, pylint is pretty much unusable. Is there still no patch or workaround for this? Or can someone recommend a replacement?

[pylint-bot]

_Original comment by_ Pieter Rautenbach (BitBucket: parautenbach, GitHub: @parautenbach?):

---

Could anybody that's contributed to the discussion so far provide a patch? It may expedite the process of including this fix into a next release.

[pylint-bot]

_Original comment by_ Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

@zeehio good catch, it seems indeed related to relative wildcard import in "absolute import mode", as found all around in numpy when using python 3. At least that problem should be fixed in a few minutes :)

[pylint-bot]

_Original comment by_ Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

Removing version: 1.0 (automated comment)

[pylint-bot]

_Original comment by_ Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

Fixed by https://bitbucket.org/logilab/astroid/commits/83d78af4866be5818f193360c78185e1008fd29e (in astroid)

[pylint-bot]

_Original comment by_ Luke Lee (BitBucket: durden, GitHub: @durden?):

---

I tested this out in a project that heavily uses numpy without any issues. However, I've seen this issue with pytest. Unfortunately, this isn't quite the same thing that's previously mentioned in this post because pytest does a lot of dynamic import magic because of its' plugin system.

For example, using pytest to test that an assertion is raised causes problems because pytest.raises is dynamically imported.

Is this fixable? If not, is there a way to globally ignore this in my pylint config instead of adding an ignore line to every instance of this usage in the code base?

[pylint-bot]

_Original comment by_ Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

Could you open a new issue, please?

[pylint-bot]

_Original comment by_ Luke Lee (BitBucket: durden, GitHub: @durden?):

---

No problem, new issue here.

[pylint-bot]

_Original comment by_ BitBucket: dwiel, GitHub: @dwiel?:

---

I just tried this with pylint 1.4.0, astroid 1.3.2, python 2.7.8, numpy 1.8.2 and I still get this problem. I also get it after installing master branch of both pylint and astroid. Am I doing something wrong?

[~/pylint] # cat ~/fail.py
''' demonstrate numpy import false positive '''
import numpy

A1 = numpy.array([1, 2, 3])
print(A1)
[~/pylint] # ./bin/pylint ~/fail.py
No config file found, using default configuration
************* Module fail
C:  5, 0: Unnecessary parens after 'print' keyword (superfluous-parens)
E:  4, 5: Module 'numpy' has no 'array' member (no-member)
...
[~/pylint] # ./bin/pylint --version
No config file found, using default configuration
pylint 1.4.0, 
astroid 1.3.2, common 0.63.0
Python 2.7.8 |Anaconda 2.0.0 (64-bit)| (default, Aug 21 2014, 18:22:21) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]

[pylint-bot]

_Original comment by_ Czarek Tomczak (BitBucket: Czarek, GitHub: @Czarek?):

---

I've been working on a patch to add "dynamic-modules" option that was suggested in one of comments here. I got it working. For details see Issue #413 and the pull request #208.

[pylint-bot]

_Original comment by_ Matthew Grimes (BitBucket: SuperElectric, GitHub: @SuperElectric?):

---

I uninstalled the Ubuntu repo version of pylint (and astroid), pulled up-to-date versions of these from the pylint repo, updated, and installed. Like dwiel, I still get missing-member errors in numpy, unless I include the following in .pylintrc:

[MASTER]
extension-pkg-whitelist=numpy

I also get missing-member errors in nose. For example in the following line:

#!python

from nose.tools import assert_raises_regexp, assert_equal

I get the pylint errors:

/home/mkg/projects/simplelearn/simplelearn/tests/test_utils.py:2: error (E0611, no-name-in-module, ) No name 'assert_raises_regexp' in module 'nose.tools'

/home/mkg/projects/simplelearn/simplelearn/tests/test_utils.py:2: error (E0611, no-name-in-module, ) No name 'assert_equal' in module 'nose.tools'

These don't go away even when I amend my .pylintrc to include

[MASTER]
extension-pkg-whitelist=numpy,nose,nose.tools

[pylint-bot]

_Original comment by_ Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

Matthew, could you open another ticket for nose, if it isn't already an existing one? The problem with nose is not the same problem as with numpy (in 1.4 extension packages aren't loaded anymore when the source can't be found, that's why you should use extension-pkg-whitelist). The problem with nose is the following piece of code, which can't by detected by static tools.

#!python

for at in [ at for at in dir(_t)
            if at.startswith('assert') and not '_' in at ]:
    pepd = pep8(at)
    vars()[pepd] = getattr(_t, at)
    __all__.append(pepd)

[pylint-bot]

_Original comment by_ Matthew Grimes (BitBucket: SuperElectric, GitHub: @SuperElectric?):

---

Thanks Claudiu. There doesn't seem to be an issue already open on the nose problem, so I've opened one.

[pylint-bot]

_Original comment by_ Dan Stromberg (BitBucket: dan_stromberg):

---

Is there a release of pylint that includes this fix yet? I'm using:
pylint 1.4.1,
astroid 1.3.4
common 0.63.2
numpy 1.9.1

...but I'm still getting:
E: 20,20: Module 'numpy' has no 'ones' member (no-member)
E: 20,49: Module 'numpy' has no 'bool_' member (no-member)

?

[pylint-bot]

_Original comment by_ Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

Dan, just use '--extension-pkg-whitelist=numpy' and it solves your problem (at least the one with ones).

[pylint-bot]

_Original comment by_ Dan Stromberg (BitBucket: dstromberg, GitHub: @dstromberg?):

---

I have a workaround, but I'd of course prefer to have a fix. I like pylint a lot.

[pylint-bot]

_Original comment by_ Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

That's not a workaround, it's the actual fix. We can't get sane behaviour for C based libraries all the time.

[pylint-bot]

_Original comment by_ Tianhui Li (BitBucket: tianhui_li):

---

In response to @samking, I've added a python package for this. https://github.com/tianhuil/gae_pylint_plugin

Please feel free to use + help update + add in numpy methods.

[gergelypolonkai]

I still get this error, even using --extension-pkg-whitelist=numpy (or its RC file counterpart). Adding unsafe-load-any-extension=yes also doesn’t help.

Creating a PyLint module as suggested by Tianhui Li provides a solution, but it feels a bit awkward for me…

I’m using pylint 1.7.1 (from PyPI) and numpy 1.12.1, if it matters.

Here is a minimal example to reproduce (I filtered out docstring-relate warnings for readability’s sake):

import numpy

def my_func():
    var = 1

    return isinstance(var, numpy.float64)

$ pylint --extension-pkg-whitelist=numpy test
************* Module test
E: 14,20: Module 'numpy' has no 'float64' member (no-member)

[gergelypolonkai]

Sorry for the noise, it seems this is already documented in #779