Pyright: Import could not be resolved for pip packages and local modules

Created on 5 Apr 2019  路  22Comments  路  Source: microsoft/pyright

Describe the bug
When opening a file that is known to work in python, all pip based modules and local (to the project) modules are flagged, e.g.: Import 'boto3' could not be resolved

To Reproduce
Import a pip based module or local module. Here is my pyrightconfig.json:

{
    "venvPath": ".",
    "venv": ".venv"
}

Note: my virtual environment is in the project directory as ".venv" (so my python is PROJECT_DIR/.venv/bin/python)

Expected behavior
Valid imports do not get flagged.

VS Code extension or command-line
VSCode extension 1.0.8

To Reproduce
macOS 10.14.2
python 3.6.5 (Anaconda)
pyright 1.0.8

Create a new folder and cd into it via the command line
Create a virtual env with:

virtualenv .venv
. .venv/bin/activate
pip install boto3

Load the following file in VSCode as test.py in the root of the project

import boto3

Additional context
Possibly related to #54, but am not sure.

picture TheCleric

Most helpful comment

Yeah, you're correct that relative imports work only in packages. You'd need to add an empty __init__.py for that to work.

Pyright has no way of knowing what files contain a main entry point for a script or what the current directory will be when you execute the code. By default, it assumes that you'll execute it from the workspace root, which is why it adds that to the search path by default. If you plan to execute your script from another directory (such as "dirB" in this case), then you should define an "execution environment" for that directory.

picture erictraut on 24 Jul 2019
👍2

All 22 comments

Can you provide the contents of your ".venv" directory? Does it contain a lib/pythonX.Y/site-packages directory? By any chance, does the lib directory contain two python.X.Y versions?

erictraut picture erictraut on 5 Apr 2019

ls -la .venv

drwxr-xr-x   6 adam  staff   192 Aug  1  2018 .
drwxr-xr-x  32 adam  staff  1024 Apr  5 11:49 ..
drwxr-xr-x  76 adam  staff  2432 Mar  5 11:57 bin
drwxr-xr-x   3 adam  staff    96 Aug  1  2018 include
drwxr-xr-x   4 adam  staff   128 Aug 22  2018 lib
-rw-r--r--   1 adam  staff    59 Aug  1  2018 pip-selfcheck.json

ls -la .venv/lib

drwxr-xr-x   4 adam  staff   128 Aug 22  2018 .
drwxr-xr-x   6 adam  staff   192 Aug  1  2018 ..
drwxr-xr-x   4 adam  staff   128 Aug 22  2018 mypy
drwxr-xr-x  58 adam  staff  1856 Aug  1  2018 python3.6
TheCleric picture TheCleric on 5 Apr 2019

Huh, that all looks fine. I presume that within the python3.6 directory, there's a site-packages subdirectory, and that contains a boto3.py file or a boto3 directory that in turn contains an __init__.py file?

If you look in the Output panel in VS Code and select Pyright, do you see any console output that provides any additional clues?

erictraut picture erictraut on 5 Apr 2019

ls -la .venv/lib/python3.6/site-packages/boto3

drwxr-xr-x   15 adam  staff    480 Sep 27  2018 .
drwxr-xr-x  218 adam  staff   6976 Mar  8 16:01 ..
-rw-r--r--    1 adam  staff   3339 Sep 27  2018 __init__.py
drwxr-xr-x    7 adam  staff    224 Sep 27  2018 __pycache__
-rw-r--r--    1 adam  staff   1490 Sep 27  2018 compat.py
drwxr-xr-x   12 adam  staff    384 Sep 27  2018 data
drwxr-xr-x   16 adam  staff    512 Sep 27  2018 docs
drwxr-xr-x    8 adam  staff    256 Sep 27  2018 dynamodb
drwxr-xr-x    6 adam  staff    192 Sep 27  2018 ec2
drwxr-xr-x    4 adam  staff    128 Sep 27  2018 examples
-rw-r--r--    1 adam  staff   3993 Sep 27  2018 exceptions.py
drwxr-xr-x   11 adam  staff    352 Sep 27  2018 resources
drwxr-xr-x    6 adam  staff    192 Sep 27  2018 s3
-rw-r--r--    1 adam  staff  19614 Sep 27  2018 session.py
-rw-r--r--    1 adam  staff   3095 Sep 27  2018 utils.py

Pyright output:

Pyright language server starting
Received updated settings.
No configuration file found.
venvPath /Users/adam/Development/XPro/${workspaceFolder}/server/.venv is not a valid directory.
venvPath must be used in conjunction with venv setting, which was omitted.
Finding source files
No source files found.

I will say I have VSCode opened at one level up from the server folder so I can edit my backend code and my frontend code at the same time, with the .venv and pyrightconfig.json in the server folder.

That's probably the real culprit here as when I open the server directory in VSCode these errors go away. Sorry about the red herrings.

TheCleric picture TheCleric on 5 Apr 2019

Does it pull the venvPath from the workspace settings.json for that output above, because here's my settings.json (which is valid in VSCode):

{
    "python.pythonPath": "${workspaceFolder}/server/.venv/bin/python",
    "python.venvPath": "${workspaceFolder}/server/.venv",
    "python.envFile": "${workspaceFolder}/.vscode/.env",
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {
        "source.organizeImports": true
    },
    "python.linting.enabled": true,
    "python.linting.lintOnSave": true,
    "python.linting.pylintEnabled": true,
    "yaml.customTags": ["!anything", "!anyint", "!anystr"]
}

I wonder (among me doing this in an odd way with subdirectories) if it's also getting hung up on the ${workspaceFolder} stuff that VSCode supports.

TheCleric picture TheCleric on 5 Apr 2019

Ah, I see the problem. Pyright is picking up your venvPath from your VS Code setting, and that setting includes an unexpanded variable ${workspaceFolder}. I'll need to see if there's a way to call back to VS Code to expand that variable.

erictraut picture erictraut on 5 Apr 2019

Based on how you're concatenating it, it MAY be safe to just sub it for . since you are already prepending the workspace folder to the path.

Not sure if Windows likes that or not though.

TheCleric picture TheCleric on 5 Apr 2019

I don't want to do the substitution in the Pyright code though. VS Code supports a bunch of path variables, so it's best if VS Code expands those variables.

erictraut picture erictraut on 5 Apr 2019
👍1

Looks like VS Code doesn't currently provide such a facility. There's an active discussion about it here. I'll do some additional investigation to see what other extensions are doing today to work around this limitation.

erictraut picture erictraut on 5 Apr 2019

In the meantime, you can work around the problem by manually expanding the variable in your settings.

erictraut picture erictraut on 5 Apr 2019

Other VS Code extensions appear to each do their own ad hoc replacement of some subset of these predefined variables. I've added code to pyright to support replacement of "workspaceFolder" since that seems like a common usage. This change will be in the next published version.

erictraut picture erictraut on 5 Apr 2019
👍1

This is fixed in 1.0.9, which I just published. Thanks again for the bug report.

erictraut picture erictraut on 6 Apr 2019

@erictraut This is still happening for me. It's happening with requests and with a local module.

import requests
import mylocalmodule  # In the same directory

I'm running the VS Code extension, version 1.0.48. Settings > Extensions > Python Configuration > Python: Python Path is set to the defeault, python. I don't have any sort of pyrightconfig file set up.

However, this does not happen on my other computer at work. I will take a look tomorrow and see what the difference is in terms of configuration.

michaelkolber picture michaelkolber on 24 Jul 2019
👍2

I've added a bunch of logging to help diagnose import resolution problems. Check out the "Output" tab, and choose the "Pyright" menu item. I think I've worked out all of the bugs, so it's likely a configuration issue, but if you aren't able to figure it out, let me know.

erictraut picture erictraut on 24 Jul 2019

In the meantime, here are the logs for requests and the local module. Note that the local module only gives an error when both files are in a subdirectory. When they're in the main VS Code directory, there's no error. From the logs, it looks like pyright is not even trying to look in the directory where the main script is.

Error-causing directory structure:

dirA  // Main directory that VS Code is open in
  |- dirB  // Directory the files are in
       |- main.py  // File where the error is
       |- localmodule.py  // File being imported

Good directory structure:

dirA  // Main directory that VS Code is open in
  |- main.py  // Now there's no error
  |- localmodule.py  // File being imported

Requests logging:

Could not import 'requests' in file 'c:\Users\Michael\nobackup\Code\Projects\myproject\myproject\myproject.py'
  Looking for typeshed stdlib path
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7\requests'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7\requests.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7\requests.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6\requests'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6\requests.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6\requests.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5\requests'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5\requests.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5\requests.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3\requests'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3\requests.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3\requests.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3\requests'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3\requests.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3\requests.py'
  Typeshed path not found
  Looking in root directory of execution environment 'c:\Users\Michael\nobackup\Code\Projects\myproject'
  Attempting to resolve using root path 'c:\Users\Michael\nobackup\Code\Projects\myproject'
  Could not find directory 'c:\Users\Michael\nobackup\Code\Projects\myproject\requests'
  Did not find file 'c:\Users\Michael\nobackup\Code\Projects\myproject\requests.pyi' or 'c:\Users\Michael\nobackup\Code\Projects\myproject\requests.py'
  Looking for typeshed path
  Looking for typeshed third_party path
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3\requests'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3\requests.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3\requests.py'
  Typeshed path not found
  No python interpreter search path

Local module logging:

Could not import 'localmodule' in file 'c:\Users\Michael\nobackup\Code\Projects\myproject\myproject\myproject.py'
  Looking for typeshed stdlib path
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7\localmodule'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7\localmodule.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.7\localmodule.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6\localmodule'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6\localmodule.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.6\localmodule.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5\localmodule'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5\localmodule.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3.5\localmodule.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3\localmodule'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3\localmodule.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\3\localmodule.py'
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3\localmodule'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3\localmodule.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\stdlib\2and3\localmodule.py'
  Typeshed path not found
  Looking in root directory of execution environment 'c:\Users\Michael\nobackup\Code\Projects\myproject'
  Attempting to resolve using root path 'c:\Users\Michael\nobackup\Code\Projects\myproject'
  Could not find directory 'c:\Users\Michael\nobackup\Code\Projects\myproject\localmodule'
  Did not find file 'c:\Users\Michael\nobackup\Code\Projects\myproject\localmodule.pyi' or 'c:\Users\Michael\nobackup\Code\Projects\myproject\localmodule.py'
  Looking for typeshed path
  Looking for typeshed third_party path
  Attempting to resolve using root path 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3'
  Could not find directory 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3\localmodule'
  Did not find file 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3\localmodule.pyi' or 'c:\Users\Michael\.vscode\extensions\ms-pyright.pyright-1.0.48\typeshed-fallback\third_party\2and3\localmodule.py'
  Typeshed path not found
  No python interpreter search path
michaelkolber picture michaelkolber on 24 Jul 2019

I think this is correct behavior. You've specified an absolute import. Python (and pyright) will look for this import in a list of configured paths. Pyright always includes in this list the root path of your workspace ("dirA" in your example above). It won't include "dirB".

Python 2.x allowed implied relative imports, but Python 3.x doesn't.

Here are several ways you can fix this:

  1. Within pyrightconfig.json file, define an "executionEnvironment" that specifies "dirB" as a rootPath. Then any files under this directory will include this root path in its list of directories to search.
  2. Use a full import path that's rooted in your workspace directory: import dirB.localmodule
  3. Use a relative import path: import .localmodule
erictraut picture erictraut on 24 Jul 2019

Forgive my ignorance, but won't Python always add the current directory of the main running script into the list of configured paths? If so, shouldn't pyright as well?

Furthermore, aren't relative/absolute imports only for packages? I'm not dealing with a package here, just two Python files. There's no __init__.py. Besides suggestions 2 and 3 not working in this case, this is the only way to even access this module. There's no other standard way (that I know of) to import the module besides just its name.

michaelkolber picture michaelkolber on 24 Jul 2019

Yeah, you're correct that relative imports work only in packages. You'd need to add an empty __init__.py for that to work.

Pyright has no way of knowing what files contain a main entry point for a script or what the current directory will be when you execute the code. By default, it assumes that you'll execute it from the workspace root, which is why it adds that to the search path by default. If you plan to execute your script from another directory (such as "dirB" in this case), then you should define an "execution environment" for that directory.

erictraut picture erictraut on 24 Jul 2019
👍2

I think this is correct behavior. You've specified an absolute import. Python (and pyright) will look for this import in a list of configured paths. Pyright always includes in this list the root path of your workspace ("dirA" in your example above). It won't include "dirB".

Python 2.x allowed implied relative imports, but Python 3.x doesn't.

Here are several ways you can fix this:

  1. Within pyrightconfig.json file, define an "executionEnvironment" that specifies "dirB" as a rootPath. Then any files under this directory will include this root path in its list of directories to search.
  2. Use a full import path that's rooted in your workspace directory: import dirB.localmodule
  3. Use a relative import path: import .localmodule

I have a little more complicated dir structure:

<root-dir>
.git\
.vscode\
myPrj\
env\
.gitignore
myPrj.code-workspace

myPrj:

src\
docs\
tests\
LICENSE
README.rst
release
requirements.txt
setup.cfg
setup.py

1) pyright was happy when I used the absolute path - myPrj.src, but python - not. It uses as root the src, not myPrj.src.
2) If I define "executionEnvironment" that specifies myPrj.src pyright does not see dirs inside myPrj.src (surprisingly I have some). When I added them into "extraPaths" pyright did not show errors in its log (except "No source files found"), but anyway it said that import "could not be resolved".
3) I did not try relative import.

My conclusion that pyright is not usefull tool. I spent day but did not make it works as I want.

Michal-D4 picture Michal-D4 on 14 Aug 2019

@Michal-D4

Yes, tools are generally not useful if they're configured incorrectly :).

Without the proper configuration, pyright has no idea how you plan to invoke your python code (which directory you'll run it from, your PYRIGHTPATH environment variable setting, which platform you're going to run it on, what python interpreter version number, etc.). It can infer some of that information, but not all.

By default, pyright assumes that you're going to run your python code from the same directory as your workspace. In your case, your workspace directory is . I can't tell for sure what directory you plan to run your python code from, but I'm guessing it's /myPrj. If that's a correct assumption, then you should configure an executionEnvironment with the rootPath set to "myPrj". Then pyright and the python interpreter will both be happy if you import from a path starting with "src".

Alternatively, you could open the "myPrj" directly within VS Code, in which case that will be the workspace root, and pyright will use that directory as the default execution environment root.

erictraut picture erictraut on 15 Aug 2019

Try to restart vscode after pip install. Restarting vscode worked for me.

hakkikonu picture hakkikonu on 1 Nov 2020
👍1

Try to restart vscode after pip install. Restarting vscode worked for me.

worked for me as well! nothing else did. Thank you!!

rairishi764 picture rairishi764 on 3 Dec 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

letrec picture letrec  路  3Comments
tamuhey picture tamuhey  路  3Comments
tiangolo picture tiangolo  路  3Comments
tamuhey picture tamuhey  路  3Comments
christopher-hesse picture christopher-hesse  路  3Comments