Python-language-server: Unresolved import for builtin packages after reload

[DonJayamanne]

@schlamar commented on Mon Jun 03 2019

Environment data

• VS Code version: 1.34.0
• Extension version (available under the Extensions sidebar): 2019.5
• OS and version: Windows 7
• Python version (& distribution if applicable, e.g. Anaconda): 2.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: N/A

Expected behaviour

No errors for builtin packages.

Actual behaviour

Import of builtin packages like time, itertools and operator are sometimes marked as unresolved import.

Steps to reproduce:

Don't know, this happens randomly.

Logs

Starting Microsoft Python language server.
##########Linting Output - flake8##########
[Info  - 12:19:26] GetCurrentSearchPaths C:\Python27\python.exe 
[Info  - 12:19:26] Python search paths:
[Info  - 12:19:26]     c:\python27\dlls
[Info  - 12:19:26]     c:\python27\lib
[Info  - 12:19:26]     c:\python27\lib\lib-tk
[Info  - 12:19:26]     c:\python27
[Info  - 12:19:26]     c:\python27\lib\site-packages
[Info  - 12:19:26] Configuration search paths:
[Info  - 12:19:26]     ...
[Info  - 12:19:33] Microsoft Python Language Server version 0.2.87.0
[Info  - 12:19:33] Initializing for C:\Python27\python.exe

(Right now, I don't have a Console Log in case of this error. I will submit them as soon as this happens again.)

[thernstig]

This happens to our project as well, e.g. for the sys module, but other modules work just fine. Example of our imports:

from __future__ import (absolute_import, division, generators, nested_scopes,
                        print_function, with_statement)

import os
import re
import subprocess
import sys

I am using the language server (i.e this is our setting "python.jediEnabled": false,).

As it is quite a basic functionality to be able to get IntelliSense for builtin packages it hampers development speed by quite a bit.

[johnybx]

Same problem with python 3.6. I do not really think that this problem is related to python version though I did not test all versions. I noticed this problem after using

"python.autoComplete.extraPaths": [ "${workspaceFolder}/src" ],
and using autosave with autopep8 few times (seem like some pep8 formating must occure but not sure). Simple folder structure:

root/
    \_env
    \_src
    \_.vscode

Before the problem occurs all autocomplete functions works well. After the problem sys, time and other builtin packages are unrecognized.

Edit:
Seem to happen always just after python log [Info - 7:16:23 PM] Reloading modules... which seem to happen always if autopep8 makes some changes to files.]

Edit2:
I tried to disable autopep8 and instead tried to use this macro:
"macros": { "saveAndLintFile": [ "workbench.action.files.save", {"command": "runInTerminal.run", "args": {"cmd": "/usr/bin/env autopep8 -i --max-line-length 120 \"${file}\"", "match": ".*"}}, "workbench.action.files.save" ] }
using runInTerminal and macros which seem to work fine and sys does not throw unresolved import . Also language server was not reloaded which seem to be related. In trace log from language server first run analyses shows Analysis of sys(CompiledBuiltin) queued but when reload occurs no builtin are analyzed even though other packages seem to be analyzed again.

[jakebailey]

Interesting. The reloading is due to our file watcher detecting changes on disk to reload files (for package installation and so on, see also #1076 for the issue on improving that mechanism).

It might be the case that when we go reload modules, some builtins are getting lost.

[jakebailey]

@johnybx Also, we'd appreciate it if you could try the daily build, which includes a few race fixes among other things that we think could have fixed the missing builtins issues:

"python.analysis.downloadChannel": "daily"

[johnybx]

@jakebailey I already tried

"python.analysis.downloadChannel": "daily"

but it did not resolve this issue.

[need47]

I have the same issue with builtin 'sys' and 'time', and I tried the daily build. It did not help. Python version is 3.7.3. So it's not limited to 2.7.

[pcolmer]

I'm occasionally hitting this problem. I'm currently running 1.36.0-insider. I'm developing against a folder opened inside a container.

Version: 1.36.0-insider (user setup)
Commit: f291714906cdcb02830b8cfe2485a3c23415b5fe
Date: 2019-06-12T05:19:53.054Z
Electron: 4.2.3
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Windows_NT x64 10.0.17763

Starting Microsoft Python language server.
##########Linting Output - flake8##########
Downloading https://pvsc.azureedge.net/python-language-server-stable/Python-Language-Server-linux-x64.0.2.96.nupkg... complete
Unpacking archive... done
[Info  - 13:08:17] Analysis cache path: /root/.cache/Microsoft/Python Language Server
[Info  - 13:08:17] GetCurrentSearchPaths /usr/bin/python3 
[Info  - 13:08:17] Python search paths:
[Info  - 13:08:17]     /usr/lib/python3.6
[Info  - 13:08:17]     /usr/lib/python3.6/lib-dynload
[Info  - 13:08:17]     /usr/local/lib/python3.6/dist-packages
[Info  - 13:08:17]     /usr/lib/python3/dist-packages
[Info  - 13:08:17] Configuration search paths:
[Info  - 13:08:17]     /usr/local/lib/ea-lib
[Info  - 13:08:18] Microsoft Python Language Server version 0.2.96.0
[Info  - 13:08:18] Initializing for /usr/bin/python3
[Info  - 13:08:25] Reloading modules... 
[Info  - 13:08:25] done.
[Info  - 13:08:25] Analysis restarted.

{
    "resource": "/workspaces/exit-automation/bamboo.py",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": "unresolved-import",
    "severity": 4,
    "message": "unresolved import 'sys'",
    "source": "Python",
    "startLineNumber": 20,
    "startColumn": 8,
    "endLineNumber": 20,
    "endColumn": 11
}

[jakebailey]

Now that I've fixed the path watcher to watch the correct things, I've observed this happening for some libraries, in my case array and itertools. When the reload happens, it seems like some modules are getting lost. Needs testing.

[jakebailey]

This might have been fixed in #1402. See also #1345.

[johnybx]

@jakebailey Problem still occurs - builtin packages are unresolved after reload of language server.

[johnybx]

@AlexanderSher @jakebailey still occurs in Microsoft Python Language Server version 0.3.47.0.

[MikhailArkhipov]

Changes are not in stable feed yet. "python.analysis.downloadChannel": "daily" is 0.3.59. We do not push builds with large changes into stable immediately, so when an issue is closed, it does not mean the fix shows up in stable next day.

[pcolmer]

when an issue is closed, it does not mean the fix shows up in stable next day.

That is certainly a good point to make but, as someone who does use the stable build, I get frustrated by the following:

1. The comment in https://github.com/microsoft/python-language-server/pull/1446 says "may fix #1156) (my emphasis on "may").

2. There isn't any clarification from the PR/merge process as to which version of Python Language Server PR#1446 is getting merged into, so people using the stable build don't know which version to keep an eye out for.

3. Given the "may" ... perhaps the issue ought to be left open until it is definite that #1156 is fixed by that PR?

[MikhailArkhipov]

It is hard to say which version will include particular change since version is assigned by build and does not directly relate to PR or a date. 0.3.59 is published to stable.

"may" fix reflects the fact that we found that earlier PR was not complete and needed additional changes that affect the imports area, but since we do not have stable repro internally, we have to wait and see if changes applied indeed fix the case. Agree, #1156 should stay open until verified.

[pcolmer]

It is hard to say which version will include particular change since version is assigned by build and does not directly relate to PR or a date. 0.3.59 is published to stable.

Is it possible to do more with the Releases feature in GitHub in that respect? Presumably a build takes a specific point in time in the git history and therefore that could be tagged with the version assigned by the build?

I realise I'm going off the point of this issue now so I'll leave it there. I'm just trying to see if there is any way for consumers of the Python Language Server to get a better insight into what each version has fixed or had added so that we know when to try and test issues like this one.

Thanks.

[johnybx]

@AlexanderSher @jakebailey @MikhailArkhipov issue still occurs in Microsoft Python Language Server version 0.3.61.0 after Analysis restarted..
I also tried to delete .cache/ but did not help.
It's pretty easy to reproduce just install some module for example pip (un)install flake8 right after that there will be Reloading modules... and builtins are lost somewhere. search paths seem to be correct (they are same as in first analyses).

[jakebailey]

Is it possible to do more with the Releases feature in GitHub in that respect? Presumably a build takes a specific point in time in the git history and therefore that could be tagged with the version assigned by the build?

At the moment, the version numbers aren't "true" semver; the last number is actually just a build number we trigger manually (versions may skip or repeat commits). Improving this is something that'd be good once we're more stable.

[jakebailey]

We believe this to be fixed. We'd appreciate if anyone with this issue could try out the beta and see if the issue goes away.

"python.analysis.downloadChannel": "beta"

[anatolyburakov]

@jakebailey i am experiencing this with RemoteSSH connection. The above line doesn't appear to work (the line in the config is dimmed, as if the option does not exist), so i can't test with the beta/daily version. I am using VSCode Insiders build. How do i get the latest Python extension?

[jakebailey]

The option is not documented, which is why VS Code does not think it exists. If you set beta and reload, the extension should download an updated copy. Beta is currently 0.3.72.

The language server and the extension are versioned separately; you don't need to get the latest extension in order to get the latest language server.

[anatolyburakov]

@jakebailey i believe the issue is fixed for me, thanks! at least i haven't seen any problems so far.

[jonathanwall]

We believe this to be fixed. We'd appreciate if anyone with this issue could try out the beta and see if the issue goes away.

"python.analysis.downloadChannel": "beta"

this fixed it for me so far

[jakebailey]

The fix is now out in stable, so you don't need to set this variable to get this fix (though we always like testers).

[noamsto]

@AlexanderSher I'm still seeing this issue with 'itertools' import under centos, is it a known problem?
Sys info:
OS: CentOS Linux release 7.7.1908
Code: 1.39.2
Python: 2.7.16 (also on 2.7.5)
Python-language-server: 0.4.71.0

[jakebailey]

Unless this occurs only after a reload (package install), then it's likely not related to this old issue. I'd appreciate it if you could file a new issue with your info.