Pylance-release: Missing info for numpy

[tekumara]

Environment data

Language Server version: v2020.7.2

OS and version: macOS

Python version (& distribution if applicable, e.g. Anaconda): Python 3.6.10

Expected behaviour

When I hover over np.ndarray I expected to see (class) ndarray

Actual behaviour

When I hover over np.ndarray it says (import) ndarray: Unknown

Code Snippet / Additional information

import numpy as np

def f(n: np.ndarray):
    pass

numpy 1.18.1

[jakebailey]

Much of numpy is compiled. We currently don't have a scraper that will create type stubs for compiled modules. Therefore, for now, a fix would be for numpy to ship stubs / annotations to describe their code, which I believe that they have merged. I'd be curious if this works better in numpy 1.19 (which I think has the stubs, but I'm unsure).

[tekumara]

Thanks for the tip @jakebailey ! I've upgraded to numpy 1.19.1 but unfortunately, I'm still encountering the above.

[jakebailey]

I'm going to retitle this and use this to deduplicate numpy issues.

[neighthan]

Note: there's already been a PR merged into numpy (https://github.com/numpy/numpy/pull/16515) that adds type stubs (these used to be in a separate repo), but the commit isn't in a release yet. So, this should be working better soon.

[ggydush-fn]

There are other packages with this same problem (i.e. Pysam). Is it possible to add a setting to disable is not a known member of module to Pylance?

[aboettcher]

Does this issue also including the problem that doc strings are not displayed for some numpy functions such as numpy.arange? Note: for other functions like numpy.zeros_like the doc strings are displayed correctly.

[jakebailey]

Yes. These functions are from compiled modules; there's no source code we can read to obtain them. Some of these functions are showing up only because numpy has declared they exist in their __all__ expression, but we aren't able to actually find the definition (so assume Unknown).

Numpy's next release is supposed to include stubs, which will make these functions known, however the docstrings are still only going to be in the compiled module where we can't access them without actually running Python and live importing the module to extract it.

[randallpittman]

Looks like the stubs are coming in NumPy 1.20. It's not clear to me if this will help with auto-completion in situations such as the following in VS Code with PyLance:

def test() -> np.ndarray:
    return np.arange(5)

x = test()
# x. <tab> doesn't give me any ndarray completions and the hover-text for test() says it returns Any
# n.b. it doesn't help to annotate x: np.ndarray either.

When I select jedi as the language server I don't have this problem, but I guess jedi must do this by

actually running Python and live importing the module to extract it.

(?)

I think this NumPy issue is especially relevant: ENH: Add annotations to ndarray and generic

[jakebailey]

We were hoping that the new numpy stubs would help immensely (as opposed to the "actually run it and see" method), but I know that a lot of the current numpy stubs right now are just Any, which won't be super helpful. For the most part, it would help in cases where there really isn't any code for us to directly analyze without scraping.

Hopefully they don't go and release with incomplete stubs...

[jakebailey]

I forgot to update this, but as of the most recent release we should be able to find many more things in numpy than previously; we'd appreciate feedback about other things that you're seeing in numpy, however.

Whether or not this will work once numpy releases their own stubs, I'm uncertain.

[kong13661]

I have upgraded to the newest version. But it alse can't catch the type returned by frequently-used api like:

a = np.array(1)
 # Pylance think the type of a is Any

May the support for numpydoc handle more problems?

[jakebailey]

The stubs are generated directly from the compiled numpy code; we don't know anything more than that and just assume Any past there. Better types will come from numpy's new stubs. Parsing docstrings is a lot tricker and not something I would expect us to try for return types.

[dcervenkov]

@jakebailey Since you said feedback about problems we see in numpy is appreciated...

np.linspace(1, 2, 3).tolist()

causes Pylance to report

Cannot access member "tolist" for type "tuple[Any, Any | float]"
  Member "tolist" is unknownPylance (reportGeneralTypeIssues)

[jakebailey]

The next release includes some improvements to our source mapper which should help get numpy docs and navigation:

[wvolkov]

The next release includes some improvements to our source mapper which should help get numpy docs and navigation:

Does that release shows np.logical_and, np.logical_or, etc. as well?
Current version lacks it:

[jakebailey]

Are you on at least numpy 1.20? That function is defined in their stubs and should appear, but may not on older versions of numpy without type info.

[wvolkov]

thanks. upgrading to 1.20 version helped!