Pylint: Pylint fails to run, exits with a PicklingError

[Nadock]

Whenever I run pylint on my project since the update to 2.7.1, pylint crashes shortly after starting with a PicklingError (see below). This also effects 2.7.0 but not 2.6.2.

Steps to reproduce

Run pylint in my project via:

$> pylint src/

The closest I've gotten to a source line for the error is a 3rd party import we are using:

from redbot.core import utils

Current behavior

Traceback (most recent call last):
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 866, in check
    check_parallel(
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/parallel.py", line 138, in check_parallel
    for (
  File "/Users/$HOME/.pyenv/versions/3.8.3/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x107ee19d0>'. Reason: 'PicklingError("Can't pickle <enum 'Context'>: attribute lookup Context on astroid failed")'

Expected behavior

pylint should run without error.

pylint --version output

Result of pylint --version output:

$> pylint --version
pylint 2.7.1
astroid 2.5
Python 3.8.3 (default, Aug  5 2020, 20:24:44)
[Clang 11.0.3 (clang-1103.0.32.62)]

Further system details

Intel MacBook Pro
MacOS BigSur 11.2.1 (20D74)

$> pip freeze
aiohttp==3.6.2
aiohttp-json-rpc==0.13.2
aiosqlite==0.15.0
appdirs==1.4.4
apsw-wheels==3.30.1.post3
argcomplete==1.12.2
astroid==2.5
async-timeout==3.0.1
attrs==19.3.0
Babel==2.8.0
black==20.8b1
chardet==3.0.4
click==7.1.2
colorama==0.4.3
commonmark==0.9.1
contextlib2==0.6.0.post1
dateparser==1.0.0
discord.py==1.5.1
fuzzywuzzy==0.18.0
idna==2.10
iniconfig==1.1.1
isort==5.7.0
lazy-object-proxy==1.5.2
Markdown==3.2.2
mccabe==0.6.1
multidict==4.7.6
mypy==0.812
mypy-extensions==0.4.3
packaging==20.9
pathspec==0.8.1
pluggy==0.13.1
py==1.10.0
Pygments==2.6.1
pylint==2.7.1
pyparsing==2.4.7
pytest==6.2.2
python-dateutil==2.8.1
python-Levenshtein-wheels==0.13.1
pytz==2020.1
PyYAML==5.3.1
Red-DiscordBot==3.4.5
Red-Lavalink==0.7.1
regex==2020.11.13
rich==9.5.1
rope==0.18.0
schema==0.7.2
six==1.15.0
toml==0.10.2
tqdm==4.48.0
typed-ast==1.4.2
typing-extensions==3.7.4.2
tzlocal==2.1
uvloop==0.14.0
wrapt==1.12.1
yarl==1.5.1

[Nadock]

Upon further testing, I remember I had jobs set to 0 in my .pylintrc, which was hiding a much more detailed stack trace:

$> pylint -j 1 src/
Traceback (most recent call last):
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/decorators.py", line 34, in cached
    return cache[func]
KeyError: <bound method ClassDef.slots of <ClassDef.AsyncFilter l.53 at 0x10c089e20>>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 862, in check
    self._check_files(
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 896, in _check_files
    self._check_file(get_ast, check_astroid_module, name, filepath, modname)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 922, in _check_file
    check_astroid_module(ast_node)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1054, in check_astroid_module
    retval = self._check_astroid_module(
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1099, in _check_astroid_module
    walker.walk(ast_node)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 72, in walk
    callback(astroid)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/checkers/variables.py", line 1191, in visit_importfrom
    self._check_module_attrs(node, module, name.split("."))
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/pylint/checkers/variables.py", line 1887, in _check_module_attrs
    module = next(module.getattr(name)[0].infer())
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 551, in getattr
    result = [self.import_module(name, relative_only=True)]
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 646, in import_module
    return MANAGER.ast_from_module_name(absmodname)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/manager.py", line 191, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/manager.py", line 100, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/builder.py", line 139, in file_build
    return self._post_build(module, encoding)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/builder.py", line 159, in _post_build
    self.delayed_assattr(delayed)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/builder.py", line 235, in delayed_assattr
    if not _can_assign_attr(inferred, node.attrname):
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/builder.py", line 60, in _can_assign_attr
    slots = node.slots()
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/decorators.py", line 36, in cached
    cache[func] = result = func(*args, **kwargs)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 2838, in slots
    slots = list(grouped_slots())
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 2823, in grouped_slots
    for cls in self.mro()[:-1]:
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 2909, in mro
    return self._compute_mro(context=context)
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 2898, in _compute_mro
    unmerged_mro = list(clean_duplicates_mro(unmerged_mro, self, context))
  File "/Users/$HOME/.local/share/virtualenvs/$PROJECT/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 110, in clean_duplicates_mro
    raise exceptions.DuplicateBasesError(
astroid.exceptions.DuplicateBasesError: Duplicates found in MROs (AsyncFilter), (_GenericAlias, _Final, object), (_GenericAlias, _Final, object), (_GenericAlias, _GenericAlias) for <ClassDef.AsyncFilter l.53 at 0x10c089e20>.

Considering the error is actually coming from asteroid, the fix probably needs to happen there but I'll leave this issue here until someone more familiar with the two projects can comment.

[cdce8p]

The issue is somewhat similar to #4131 and #4093
It's currently being worked on and should hopefully be fixed with the upcoming release 2.7.2

[Nadock]

Ah yep, they do look like duplicates, I didn't think to try searching the second error - only the first one.

I'll leave it to the maintainers if they want to keep this issue open, or close it as a dupe. Technically the other issues (#4131 and #4093) don't have the same first error but that's only because I'm running pylint with jobs: 0.

[cdce8p]

@Nadock Pylint 2.7.2 has just been released. That should have fixed the issue. Just to be sure, can you test if everything works?

[Nadock]

@cdce8p I can confirm Pylint 2.7.2 has fixed this issue, it no longer crashes instead it exits 0 as expected. Thanks for your hard work and quick turn around on this one 👏

[jackton1]

@cdce8p I can confirm that this doesn't work yet.

pylint...................................................................Failed
- hook id: pylint
- exit code: 1

Traceback (most recent call last):
  File "/.../.virtualenvs/content_library-2/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/.../.virtualenvs/content_library-2/lib/python3.6/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/.../.virtualenvs/content_library-2/lib/python3.6/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/.../.virtualenvs/content_library-2/lib/python3.6/site-packages/pylint/lint/pylinter.py", line 870, in check
    files_or_modules,
  File "/.../.virtualenvs/content_library-2/lib/python3.6/site-packages/pylint/lint/parallel.py", line 147, in check_parallel
    ) in pool.imap_unordered(_worker_check_single_file, files):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 735, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x106882320>'. Reason: 'PicklingError("Can't pickle <enum 'Context'>: attribute lookup Context on astroid failed",)'

$ pylint --version
pylint 2.7.2
astroid 2.5.1
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

[cdce8p]

@jackton1 This might be a different error. Please run pylint with -j 1. That should print the correct error message.

[Pierre-Sassoulas]

The hard to read error message is due to #4165