Python-language-server: Unresolved imports for project files

[praveen-srinivasan]

Python modules in my workspace show as unresolved imports. I've tried adding paths to python.autoComplete.extraPaths, e.g.

"python.autoComplete.extraPaths": [
"${workspaceFolder}/",
],

to no avail. I'm using the remote development extension (SSH).

[MikhailArkhipov]

Some details and sample code would help.

[egabrum]

@praveen-srinivasan , let us know if your case is different, but what I see, in a quick-and-dirty example is:

Folder with 2 files: a.py and b.py
a.py defines function f1
b.py tries to import function f1 from a.py:
from a import f1
and the import is not recognized.

[jakebailey]

I tried that as described, and it works for me with either Python 2 or 3 activated.

[egabrum]

I'm very confused.
Note that I didn't add this folder to extraPaths, I don't think we are expected to do so when importing from a relative path.

However, if I use from .a

But the interpreter can't run that.

[jakebailey]

That's a different setup than what I did, then. You have it in a subdirectory, but our import system treats the workspace root as the import root, meaning that module needs to be imported as "vscode.a" unless you use extraPaths to specify that folder as another import root.

You can run python b.py such that import a works, but that relies on the fact that any file run that way (as a script) has its sys.path modified at startup to include the script's directory. The language server cannot determine that any one file is a script or not a script and does not emulate this behavior (but can be modified via extraPaths).

(Feasibly this behavior could be loosed to allow this, but it's not simple to achieve at this moment, as we require every module in a workspace to have a single name; in this example that module could be imported as vscode.a from one context, and a from a script in the same directory, breaking that invariant. @AlexanderSher would know better the cost of making that change.)

[jakebailey]

FWIW "import a" is not importing from a relative path to my knowledge of the term.

[egabrum]

Thanks @jakebailey
I realize a big part of the problem is my lack of good understanding of how the import hierarchy works in Python and, specially, in the Language Server.

For illustration, I'll summarize my particular use case:
I share code with colleagues that are not VSCode or LS users. We use multiple 'helper' py files with utilities that we import from our pytests, direcly from the same folder where each pytest is. We also use multiple packages that are in constant evolution (hence the frequent switch between editable and regular installs). This just works in their IDEs (PyDev, pyCharm, VSCode + Jedi) and in Production.

As it is, and with the requirements it seems to have, LS is not an option for me or my colleagues. But I only say this to represent my use case, not in frustration. I feel I'm distracting you guys from your vision, trying to force you to adapt to my needs. I'll give LS another try in the future.

Keep up the good work. Thanks.

[AlexanderSher]

@egabrum, I'm not sure what do you mean by saying it just works, because the way it works in pyCharm, VSCode + Jedi, VSCode + LS and python is different.
Consider a sample file structure:

└─ src
    ├─ package
    │   └─ module_c.py
    ├─ module_a.py
    └─ module_b.py

with the code in module_b:

def f():
    """This is the docstring for f"""
    pass;

and module_c:

def g():
    """This is the docstring for g"""
    pass;

For the following code in module_a:

from module_b import f as f1 
from src.module_b import f as f2
from package.module_c import g as g1
from src.package.module_c import g as g2

• VSCode + jedi shows no errors, documentation and go to definition works:
  

• PyCharm with default project settings highlights unresolved imports module_b and package, as well as functions f and g:
  

• python -B src/module_a.py fails with module resolution error on the second line:
  

• Finally, VSCode + PLS highlights same unresolved imports as PyCharm, but doesn't highlight functions:
  

To make PyCharm errors match python errors, src must be made a content root:

In PLS the same result is achieved by setting "python.autoComplete.extraPaths": ["./src"]

Now, pytest has a discovery mechanism that is different from PEP-420 and from everything above:
https://docs.pytest.org/en/latest/pythonpath.html

We don't have special support for pytest import logic (neither does PyCharm, its test runner overrides sys.path). You can create a feature request on github.

[egabrum]

@jakebailey , thank you for spending the time to provide this detailed explantation.

[egabrum]

@AlexanderSher I've seen a lot of improvement on the import mechanism in PLS, but we still can't switch from Jedi to PLS because we use Pytest heavily. As you mentioned above, with Pytest one can import from files in the same folder. Jedi seems to work fine with it, but for PLS it's a unresolved-import.

Do you know if this issue/request is already part of the scope of any of the existing tickets around unused-imports? Or should I open a new one specific to Pytest? Thanks.

P.S. I did try to find existing issues, but it's not clear to me if this would be part of their scope. e.g. #1469

[jakebailey]

Just to verify, adding that test folder to extraPaths does not improve the situation? That should be a way to manually specify extra import roots.