Pylint: RecursionError when checking module using with pandas.merge

[Dobiasd]

pylint crashes with

RecursionError: maximum recursion depth exceeded while calling a Python object

(full output here)

on this minimal .py file:

import pandas
pandas.merge(None, None)

Here is a Dockerfile to reproduce the problem:

FROM ubuntu:18.04

RUN apt-get update -y
RUN apt-get upgrade -y
RUN apt-get dist-upgrade -y

RUN apt-get install -y python3.7
RUN ln -s python3.7 /usr/bin/python3

RUN apt-get install -y python3-pip
RUN pip3 install --upgrade pip

RUN python3.7 -m pip install pylint
RUN python3.7 -m pip install pandas

RUN echo "import pandas;" > pylint_pandas_crash.py
RUN echo "pandas.merge(None, None)" >> pylint_pandas_crash.py

RUN cat pylint_pandas_crash.py

RUN python3.7 --version
RUN pylint --version
RUN python3.7 -c 'import pandas; print(pandas.__version__)'

RUN pylint pylint_pandas_crash.py

• python version: 3.7.1
• pylint version: 2.3.1
• astroid version: 2.2.5
• pandas version: 0.24.2

[PCManticore]

Thanks for the report!

[RebeccaRoberts]

Having the same problem, fixed by reverting to older version of pandas.

[kylrth]

Encountering the same issue.

pylint 2.3.1
astroid 2.2.5
Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)]

[PCManticore]

This seems to have been fixed with pylint 2.4.3 and astroid 2.3.2. Can you give it a go and let us know if you can still reproduce the issue?

[sbardPHI]

Hello, I'm still encountering this error with:
Python 3.7.5
pylint 2.4.3
astroid 2.3.2
pandas 0.24.2

[PCManticore]

Thanks @sbardPHI I can reproduce the bug still using the versions you've mentioned.

[karltk]

Not sure if this is related, but I also get a stack overflow with Pandas. In my case, the minimal example I managed to make is:

import pandas as pd
pd.to_datetime(None).dt

If I drop the .dt dereference, it works fine.

This is with

pylint 2.4.4
astroid 2.3.3
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0]

[timhunderwood]

Using:

pylint 2.4.4
astroid 2.3.3
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)]
pandas 0.25.3

I am not able to reproduce the original error using the code:

import pandas
pandas.merge(None, None)

I can however reproduce the issue, if I am using the pylint.extensions.redefined_variable_type plugin, with the below code:

import pandas
x = pandas.to_datetime("2019-01-01")

>>>pylint --load-plugins=pylint.extensions.redefined_variable_type pandas_script.py

I've done some debugging and it seems that the recursion error occurs when running astroid.inference.infer_call.
It creates a deep recursion of calls to stroid.scoped_nodes.FunctionDef.infer_call_result
astroid.decorators.raise_if_nothing_inferred, which eventually raises the error.

Is anyone able to reproduce the reported issue without the redefined_variable_type plugin enabled on the latest version of pylint / pandas?

I can look into the issue in more detail if helpful.

[esba1ley]

This seems to have reared its ugly head again with:

pandas 1.1.3
pylint 2.6.0
astroid 2.4.2
Python 3.8.5 (default, Sep 4 2020, 02:22:02)

I can reproduce the errors with

import pandas
pandas.merge(None, None)

Things were working fine until I upgraded my environment, and now I get these pylint errors on my code that merges pandas dataframes.

[Dobiasd]

I also ran into this again, even in a project not using pandas.

But increasing the recursion limit helps in my cases. One way to do so it by having the following to the pylintrc file:

[MASTER]
init-hook='import sys; sys.setrecursionlimit(8 * sys.getrecursionlimit())'

---

Dockerfile to reproduce:

FROM python:3.9.1

RUN pip install pip==20.3.1
RUN pip install pylint==2.6.0 pandas==1.1.5 astroid==2.4.2

RUN echo "import pandas;" > main.py
RUN echo "pandas.merge(None, None)" >> main.py

# TODO: Enable the following two lines to prevent the RecursionError from happening.
#RUN echo "[MASTER]" > pylintrc
#RUN echo "init-hook='import sys; sys.setrecursionlimit(8 * sys.getrecursionlimit())'" >> pylintrc

RUN pylint main.py