Pylint: PyLint incorrectly identifying certain classes as tuples

[gandhis1]

See this issue in another repository: https://github.com/DonJayamanne/pythonVSCode/issues/497
The PythonVSCode owner indicates this seems to be an upstream issue with PyLint.

The function pd.read_csv, from the Pandas module, returns a pd.DataFrame. However, it appears PyLint is identifying this as a tuple and/or list, and is flagging my usage of pd.DataFrame.rename as invalid.

Here's a screenshot below of the code excerpt and PyLint message (or see the link above).

[PCManticore]

Hi,

Can you provide more details, regarding the version of pylint & astroid you used?

Using the latest available pandas, in combination with pylint 2.0 & astroid 1.5.0, this works for me:

$ cat a.py
from astroid import extract_node
n = extract_node('''
import pandas as pd
pd.DataFrame()
''')
i = next(n.infer())
print(i) 
$ python a.py
<Instance of pandas.core.frame.DataFrame at 0x75820592>

[PCManticore]

Oh, I see from the linked issue that you used pylint 1.6.4. Can you try with pylint and astroid's master branches and check if it works for you? You can find astroid's repository at https://github.com/PyCQA/astroid.

[gandhis1]

I will test these latest versions and report back...

[gandhis1]

Haven't gotten around to testing the latest branch, but there's another issue that may or may not be related. cx_Oracle most certainly does have a connect member.

[PCManticore]

Can you try with pylint --extension-pkg-whitelist=cx_Oracle? We don't analyze correctly extension packages, unless the user explicitly ask us to dynamically load it and construct an analysis from there

If this does not work as well, please open another, separate issue for this problem.

[gandhis1]

Thanks so much. I'm asking these questions in the context of the Python extension for Visual Studio Code, which comes with various default configurations for linting using PyLint.

Let's say, hypothetically, the extension was modified such that it identified imported extension packages, and automatically added all of them to the whitelist argument when calling PyLint. Are there any unforseen consequences of this, performance-oriented or otherwise?

[PCManticore]

Yes. One drawback would be, as you said, a performance hit, due to importing every extension package and building a dynamic AST that pylint can understand from raw objects. Secondly, this could be, theoretically, a security problem, since you might have a malicious extension package, which does something outside of your control during its import. But these might not be a problem, depending on what kind of extensions you are dealing with.

[ghost]

pandas dataframe still could not work OK with pylint 2.0 and astroid 1.5

Here I have:

pylint 2.0.0,
astroid 1.5.0
Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:42:40)

and my test code is:

import pandas as pd
df = pd.read_csv("/tmp/")
df.to_csv("/tmp/", index=False)

it gaves:

ID:no-member Instance of 'tuple' has no 'to_csv' member instance.py /TryInstanceVariable line 19 PyLint Problem

ID:no-member Instance of 'list' has no 'to_csv' member instance.py /TryInstanceVariable line 19 PyLint Problem

Still, recognize it as tuple or list...

[PCManticore]

@weiliz which version of pandas and on what OS? Might be useful for reproducing it, when I first tried, this worked.

[gandhis1]

I am running Pandas 0.19.0 on Windows 7, with the latest version of Anaconda 64-bit, on python 2.7.12. I do not have the latest versions of PyLint and astroid, but I do get this issue. I believe I get this on my personal computer (Windows 10, Python 3, Pandas 0.19.0) as well (will check later).

[ghost]

I am using pandas 0.19.1 on ubuntu 16.04.1
Name: pandas
Version: 0.19.1
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

I'm kinda of found a way to bypass this problem, based on @PCManticore 's working code.

@PCManticore 's working code shows structure produced by pandas.DataFrame() can be correctly parsed, however, structure returned by pandas.read_csv() cannot be recognized.

So I just get around by:
a = pd.DataFrame(pd.read_csv("\path\to\file"))
and it could be recognized.

I have tried various of data types in @PCManticore 's code returned by pandas and numpy to see if pylint can recognize. pandas.read_csv() is one of those cannot be recognized, and numpy.array() is another one..just wondering if there is a way to solve this.

[gandhis1]

The read_csv function is defined indirectly in pandas by calling another function _make_parser_function which itself returns a function. Perhaps this indirection is confusing pyLint?

[PCManticore]

We already have problems with numpy.array. Regarding pd.read_csv, the best solution I can think now, without checking how that function is implemented, is to write an astroid brain tip for it, as in: https://github.com/PyCQA/astroid/tree/master/astroid/brain

[pcorpet]

Have you ignored any modules or classes for typecheck in your pylintrc or commandline? It's a good practice with pandas, and then you're probably running into #1276.

[tomwanzek]

Is there an update on this issue? It still replicates on:

• Windows 10 machine conda virtual environment
• python (3.5)
• pylint (1.6.5) astroid (1.4.9)
• pandas (0.19.2)

(for completeness VS code 1.10, Python Extension 0.58)

Thx in advance.

[gusennan]

I am able to reproduce this issue on:

• Linux Fedora 4.10.8-200.fc25.x86_64
• python 3.5.3
• pylint 1.6.5
• asteroid 1.4.9
• pandas 0.19.2

== EDIT ===
After upgrading pylint to 1.7.1, this issue doesn't occur anymore.

[keiv-fly]

I had the same problem, but after updating pylint from 1.6.x to 1.7.1 the problem disappeared.
My system: Windows 7, python 3.5.2

[tomwanzek]

I can confirm that pylint 1.7.1 fixed the issue I was experiencing when using pandas.

[gandhis1]

Will check this myself and close if it's been fixed.

[pmav99]

I also confirm that the new version correctly recognizes pd.read_csv:

• pylint 1.7.1
• linux x64
• python 3.6

[predicador37]

Hi all:

I know it's not exactly the same issue, but the same problem occurs with pd.read_sql_table() from pandas

[KokoDoko]

Same problem here, when importing datasets from sklearn:

from sklearn import datasets
digits = datasets.load_digits()

## Instance of 'tuple' has no 'images' member
digits.images[2] 

pylint version 2.3.1

[juanmigutierrez]

Same problem using python:
from sklearn import datasets data = datasets.load_iris() X = data.target

pylint version 2.3.1

[alzulas]

Same issue is still occurring for pandas data frames and .loc or .iloc

Error: Generator 'generator' has no 'loc' member

pylint 2.4.4
astroid 2.3.3