Pylint: Improve the situation for `no-name-in-module` for C extensions and modules using dynamic features

[chrisspen]

I have the ProxyTypes package installed and have a file mymodule.py that imports it like:

from peak.util.proxies import CallbackProxy

However, pylint throws this false error for it:

E:  8, 0: No name 'util' in module 'mymodule.peak' (no-name-in-module)

Pylint should reference the peak package installed into my virtualenv, and not assume it's a local non-existent module.

[PCManticore]

Please provide a minimal working example, I'm not sure I completely understand what is happening, especially since the error you posted indicate something more complex going on than the first import.

[chrisspen]

You can easily reproduce it with the following procedure:

$ mkdir test
$ cd test
$ virtualenv .env
Running virtualenv with interpreter /usr/bin/python2
New python executable in test/.env/bin/python2
Also creating executable in test/.env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
$ .env/bin/pip install pylint ProxyTypes
Collecting pylint
  Using cached pylint-1.7.1-py2.py3-none-any.whl
Collecting ProxyTypes
  Downloading ProxyTypes-0.9.zip
Collecting six (from pylint)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting mccabe (from pylint)
  Using cached mccabe-0.6.1-py2.py3-none-any.whl
Collecting singledispatch; python_version < "3.4" (from pylint)
  Using cached singledispatch-3.4.0.3-py2.py3-none-any.whl
Collecting isort>=4.2.5 (from pylint)
  Using cached isort-4.2.15-py2.py3-none-any.whl
Collecting astroid>=1.5.1 (from pylint)
  Using cached astroid-1.5.3-py2.py3-none-any.whl
Collecting configparser; python_version == "2.7" (from pylint)
Collecting backports.functools-lru-cache; python_version == "2.7" (from pylint)
  Using cached backports.functools_lru_cache-1.4-py2.py3-none-any.whl
Collecting enum34>=1.1.3; python_version < "3.4" (from astroid>=1.5.1->pylint)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting lazy-object-proxy (from astroid>=1.5.1->pylint)
  Using cached lazy_object_proxy-1.3.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting wrapt (from astroid>=1.5.1->pylint)
Building wheels for collected packages: ProxyTypes
  Running setup.py bdist_wheel for ProxyTypes ... done
  Stored in directory: /home/chris/.cache/pip/wheels/e3/8f/4c/5d7b9f8e251cc9567b730e7f9e2f74a9ea36a4d9189e0d2864
Successfully built ProxyTypes
Installing collected packages: six, mccabe, singledispatch, isort, enum34, lazy-object-proxy, wrapt, backports.functools-lru-cache, astroid, configparser, pylint, ProxyTypes
Successfully installed ProxyTypes-0.9 astroid-1.5.3 backports.functools-lru-cache-1.4 configparser-3.5.0 enum34-1.1.6 isort-4.2.15 lazy-object-proxy-1.3.1 mccabe-0.6.1 pylint-1.7.1 singledispatch-3.4.0.3 six-1.10.0 wrapt-1.10.10
$ echo -e "from peak.util.proxies import CallbackProxy\n" > mymodule.py
$ ls -lah
total 16K
drwxrwxr-x  3 chris chris 4.0K Jun 20 16:54 .
drwxr-xr-x 11 chris chris 4.0K Jun 20 16:52 ..
drwxrwxr-x  7 chris chris 4.0K Jun 20 16:53 .env
-rw-rw-r--  1 chris chris   45 Jun 20 16:54 mymodule.py
$ .env/bin/pylint mymodule.py
No config file found, using default configuration
************* Module mymodule
C:  2, 0: Trailing newlines (trailing-newlines)
C:  1, 0: Missing module docstring (missing-docstring)
E:  1, 0: No name 'util' in module 'mymodule.peak' (no-name-in-module)
E:  1, 0: Unable to import 'peak.util.proxies' (import-error)
W:  1, 0: Unused CallbackProxy imported from peak.util.proxies (unused-import)

------------------------------------------------------------------------
Your code has been rated at -120.00/10 (previous run: -120.00/10, +0.00)


$ .env/bin/python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from peak.util.proxies import CallbackProxy
>>>

[meniluca]

I see the same behaviour with another library. Any idea how to fix this?

[andrewxiwu]

Same problem here. Any idea/reference to fix this issue? It is really annoying to have these false positives.

[neocogent]

Same problem here with gevent.ssl.
pylint says SSLContext and PROTOCOL_SSLv23 don't exist but they clearly do.

Python 2.7.12. gevent 1.2.2

Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gevent.ssl import SSLContext, PROTOCOL_SSLv23
>>> 

And another very common one also reported as non-existent, in this case as no-member,

[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb as db
>>> db.Error
<class '_mysql_exceptions.Error'>
>>> 

Have to disable no-name-in-module for now.

Maybe it's related error but I also get a warning on this,
from backports.functools_lru_cache import lru_cache
It says 'relative-import' when it's clearly not.

Incidentally I am not using virtualenv here. This is system wide python.

[soundlake]

It seems duplicated. https://github.com/PyCQA/pylint/issues/1138

[soundlake]

@neocogent --unsafe-load-any-extension=y worked for me.

[rawrgulmuffins]

I'm running into a similar issue at my work with a pure python sub-package. --unsafe-load-any-extension has no effect in this case.

[jcezarms]

I'm an year late, but still seeing the exact same behaviour with tensorflow.python.lib.io, even though both the library and its __init__.py file exist.

As I'm using vscode, this issue causes workspace errors to render in a misleading and incorrect way. To quickly reproduce, install tensorflow and type from tensorflow.python.lib.io import file_io in a newly created file:

Running pylint v1.9.2 from the command line on the above file points the same

$ pylint some.py
No config file found, using default configuration
************* Module some
C:  1, 0: Final newline missing (missing-final-newline)
C:  1, 0: Missing module docstring (missing-docstring)
E:  1, 0: No name 'python' in module 'tensorflow' (no-name-in-module)
W:  1, 0: Unused file_io imported from tensorflow.python.lib.io (unused-import)

-----------------------------------------------------------------------
Your code has been rated at -70.00/10 (previous run: -40.00/10, -30.00)
```` 
For these specs
```bash
$ pylint --version
No config file found, using default configuration
pylint 1.9.2,
astroid 1.6.5
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 03:37:03) [MSC v.1900 64 bit (AMD64)]

@PCManticore I'd consider the above a minimal working example for this issue, can you review it to maybe consider reopening this one?

[brycepg]

• peak issue is due to #1361 I think
• gevent.ssl is doing copy_globals(_source, globals()) (wontfix / maybe astroid brain fix)
• tensorflow seems to be a valid issue

[rafaelmendy]

Same problem in importing PyQt5 libraries.

The code works but pylint send errors.

[eirikgje]

Another datapoint here:

from netCDF4 import Dataset
leads to
[pylint] E0611: No name 'Dataset' in module 'netCDF4' (no-name-in-module)

[PCManticore]

Folks, before commenting here, do make sure to do the following:

• try running pylint first with pylint --extension-pkg-whitelist=<the package that gives you trouble>. e.g pylint --extension-pkg-whitelist=netCDF4 <your files> This way we can make sure it's not dealing with a C extension package that pylint cannot understand
• if that didn't work, it's most likely something related to how that package is structured, in which case we'll need separate issues per each package, as each own can have a different particularity (e.g some of them are namespace packages, some of them uses globals(), others import from an inner C extension etc). Creating a separate issue in this case means we don't conflate a problem in a package with another one for this issue.

[rawrgulmuffins]

Tested --extension-pkg-whistlist and in my case and it did not solve the problem.

I'll open an issue for mine but it's unfortunately not an open source project causing this issue so I won't be able to point to a specific project link. =(

[gaziqbal]

Is there any guidance on how to bypass the tensorflow.python issue mentioned by @jcezarms above? i'd like to avoid disabling the no-name-in-module error just to get past this false positive.

[chm123]

Is there any guidance on how to bypass the tensorflow.python issue mentioned by @jcezarms above? i'd like to avoid disabling the no-name-in-module error just to get past this false positive.

In VSCode, I had to set

    "python.linting.pylintArgs": [
        "--ignored-modules=tensorflow",
        "--extension-pkg-whitelist=tensorflow"
    ],

to get rid of the false positives.

[danielbraun89]

any updates?

[ynarwal]

It happens with specific names of python modules, not sure why though.
I renamed module factories to model_factories warning went away.

[Jack-CHS-PWD]

also happens with

'from win32api import GetShortPathName'
which throws
'No name 'GetShortPathName' in module 'win32api' pylint(no-name-in-module)'

getshortpathname is called later in the code and does work correctly.

[PCManticore]

This error usually happens because the given module is either a C extension, so a static analysis tool can't read that source code to figure out the members, or the module tries to be too smart by using dynamic features such as globals().update to update the global of one or more files. pylint can't really help with these kind of modules, and there are some workarounds that you can use:

• use --ignored-modules= to ignore the module that is giving you the error
• use --generated-members to specify runtime specific attributes
• use --extension-pkg-whitelist= to let pylint know that it should import a C extension to build the AST from the live module

[micklat]

A suggestion to others who encounter this: ensure that your LD_LIBRARY_PATH (or similar) contains paths to all the dependencies of the module you're trying to import. I had failures like the above when importing a C extension that depended on other extensions, and fixing LD_LIBRARY_PATH apparently made the problem go away.

[kevlarr]

I'm running into this issue (and several others) but only on our build machine. Both use pylint 2.4.4.

Given...

from pydantic import BaseModel

Locally (MacOS Mojave 10.14.6) I see no errors.

However, on our build machine (Ubuntu 18.04.1 LTS) I see:

E0611: No name 'BaseModel' in module 'pydantic' (no-name-in-module)

[djelekar]

from pydantic import Json

E0611: No name 'Json' in module 'pydantic' (no-name-in-module)

pylint=2.5.3 pydantic=1.6.1

The same thing happens here. Works on Windows, while fails on Ubuntu. What can we do about it?

[lesinigo]

Same happens with setproctitle, at least since dvarrazzo/py-setproctitle#31 (early 2014).

AFAIK it is reproducible on every kind of system, but if you want a simple test scenario you could get one with this:

$ sudo docker run -ti --rm alpine:3.12
... here we are inside the container ...
/ # apk add py3-pip py3-setproctitle
... lots of packages installed ...
/ # pip install pylint
... pylint installed (it's currently not included in Alpine 3.12 packages) ...
/ # pylint --version
pylint 2.5.3
astroid 2.4.2
Python 3.8.5 (default, Jul 20 2020, 23:11:29)
/ # pip list -v | grep setproctitle
setproctitle      1.1.10     /usr/lib/python3.8/site-packages

/ # echo 'from setproctitle import setproctitle' > example.py
/ # pylint -E example.py
************* Module example
/example.py:1:0: E0611: No name 'setproctitle' in module 'setproctitle' (no-name-in-module)

Installing setproctitle through pip instead of apk does not make any difference:

$ sudo docker run -ti --rm alpine:3.12
/ # apk add gcc musl-dev python3-dev py3-pip
/ # pip install pylint setproctitle
/ # pip list -v | grep setproctitle
setproctitle      1.1.10     /usr/lib/python3.8/site-packages
/ # pylint --version
pylint 2.5.3
astroid 2.4.2
Python 3.8.5 (default, Jul 20 2020, 23:11:29)

/ # echo 'from setproctitle import setproctitle' > example.py
/ # pylint -E example.py
************* Module example
/example.py:1:0: E0611: No name 'setproctitle' in module 'setproctitle' (no-name-in-module)

[craigtrim]

@djelekar same situation. If you discover a better solution than just putting
# pylint: disable=no-name-in-module
on each file with this import, I'd like to hear it!