Ptvs: refreshing db not working

[ryansmccoy]

Any idea why this is preventing me from refreshing?

I've had this issue in past and fixed renaming anaconda directory, reinstalling anaconda, and copying env to c:\anaconda\envs, but seeing this has happened a few time now I'd be interested in understanding what is causing.

[zooba]

One thing to check is whether restarting VS clears the button. I've seen an intermittent issue where the button remains visible even when it shouldn't, but restarting VS gets rid of it.

[ryansmccoy]

So far, have tried restarting VS, restarting computer, reinstalling Anaconda or doing Visual Studio repair.

[zooba]

Can you copy the info from Tools, Python Tools, Diagnostic Info into a gist or email it to [email protected]?

[ryansmccoy]

Done to email.

[zooba]

Got it. While there are clearly some issues there (to do with Anaconda not behaving like regular Python wrt virtual environments), they aren't affecting the one shown in the screenshot.

Can you also send through C:\Users\<username>\AppData\Local\Python Tools\CompletionDB\14.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\3.5\AnalysisLog.txt? It should have been included in the log, but maybe it was cut off somehow.

[ryansmccoy]

Alright sent to email in zip. I had just finished uninstall python/ptvs from vs when I saw your reply (wasn't expecting a reply), so did a system restore which might have been reason why stuff maybe off.

Yea, virtenv sounds about right.

[zooba]

Okay, so there's one issue here that's our fault:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.2\PythonScraper.py", line 385, in generate_member
    return 'method', generate_builtin_function(obj, True)
  File "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.2\PythonScraper.py", line 270, in generate_builtin_function
    function_table['overloads'] = BuiltinScraper.get_overloads(function, is_method)
  File "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.2\BuiltinScraper.py", line 514, in get_overloads
    extra_args,
  File "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.2\BuiltinScraper.py", line 471, in get_overloads_from_doc_string
    res = parse_doc_str(doc_str, mod, decl_mod, func_name, extra_args, obj_class)
  File "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.2\BuiltinScraper.py", line 262, in parse_doc_str
    ret_types = get_ret_type(ret_type, obj_class, mod)
  File "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.2\BuiltinScraper.py", line 181, in get_ret_type
    return [type_name_to_typeref(ret_type, mod, RETURN_TYPE_OVERRIDES)]
  File "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.2\BuiltinScraper.py", line 139, in type_name_to_typeref
    for mod_name, mod in sys.modules.items():
RuntimeError: dictionary changed size during iteration

But it looks like your real problem is that Python won't run at all:

System.InvalidOperationException: Cannot obtain search paths ---> System.InvalidOperationException: Cannot obtain list of paths
Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'

This probably means you've got a corrupted registry (Anaconda and CPython share the same registry keys here, so installing/uninstalling either can break the other) or missing files on disk. Either way, doing a repair install on Anaconda should resolve it, and the long-term fix is for Python to generally stop using the registry like this (which hopefully Python 3.6 will achieve, but 3.5 and 2.7 cannot at this stage).

[ryansmccoy]

Any suggestions on registry keys to remove because reinstall of Anaconda did not do the trick (Searching for anaconda in registry with regscanner, going to delete everything).

[KevinLucidyne]

Python Tools for Visual Studio 2013 CompletionDB never refreshes

When I'm in the interactive window of PVTS and I enter

import sys

I get the error message

The completion DB needs to be refreshed.
Open Python Environments for more information.

When I open Python Environments I see the message

Completion DB needs refresh

When I press "Refresh DB" the "Refreshing DB" status bar starts but never completes.

PTVS Diagnostic Info

Projects: 
    Project: HelloWorld.pyproj
        Kind: Python
        InterpreterVersion: 2.7
        StartupFile: C:\Users\Kevin\dev\python\HelloWorld\HelloWorld.py
        WorkingDirectory: .
        PublishUrl: <undefined>
        SearchPath: 
        CommandLineArguments: <undefined>
        InterpreterPath: <undefined>

Environments: 
    Microsoft.PythonTools.Interpreter.ConfigurablePythonInterpreterFactoryProvider
    Microsoft.PythonTools.Interpreter.CPythonInterpreterFactoryProvider
        Id: 9a7a9026-48c1-4688-9d5d-e5699d47d074
        Factory: Python 64-bit 2.7
        Version: 2.7
        Arch: Amd64
        Prefix Path: C:\Anaconda2
        Path: C:\Anaconda2\python.exe
        Windows Path: C:\Anaconda2\pythonw.exe
        Lib Path: C:\Anaconda2\lib
        Path Env: PYTHONPATH

    Microsoft.IronPythonTools.Interpreter.IronPythonInterpreterFactoryProvider
Launchers:
    Launcher: Microsoft.PythonTools.Project.Web.PythonWebLauncherProvider
        Launches web sites using the Python debugger. This enables launching and starting a web browser automatically, using the commands specified in the project file.
        Web launcher

    Launcher: Microsoft.PythonTools.Project.DefaultLauncherProvider
        Launches and debugs Python programs. This is the default.
        Standard Python launcher

    Launcher: Microsoft.PythonTools.Django.Debugger.DjangoLauncherProvider
        Launches Django web sites using the Python debugger. This enables launching and starting a web browser automatically. It launches using manage.py and passes the runserver flag as well as additional options to configure the port and other settings.
        Django launcher

    Launcher: Microsoft.IronPythonTools.Debugger.IronPythonLauncherProvider
        Launches IronPython scripts using the .NET debugger.  This enables debugging both IronPython code as well as other .NET code such as C# or VB.NET.
        IronPython (.NET) launcher

Logged events/stats:
Installed Interpreters: 1
Configured Interpreters: 0
Debug Launches: 0
Normal Launches: 0



System events:
Time: 2016-07-15T16:36:20
Application: devenv.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.Runtime.InteropServices.InvalidComObjectException
   at System.StubHelpers.StubHelpers.GetCOMIPFromRCW(System.Object, IntPtr, IntPtr ByRef, Boolean ByRef)
   at Microsoft.VisualStudio.Shell.Interop.IVsHierarchy.UnadviseHierarchyEvents(UInt32)
   at Microsoft.VisualStudio.Progression.CodeProviderNativeProjectHelper+ProjectOutputMap.Close()
   at Microsoft.VisualStudio.Progression.CodeProviderNativeProjectHelper.Complete()
   at Microsoft.VisualStudio.Progression.CodeProvider.Dispose(Boolean)
   at Microsoft.VisualStudio.Progression.CodeProvider.Dispose()
   at Microsoft.VisualStudio.Progression.ProviderCreator.Dispose(Boolean)
   at Microsoft.VisualStudio.Progression.ProviderCreator.Finalize()

Loaded assemblies:
...

Global Analysis:
2016-07-14T15:44:27 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
FAIL_STDLIB: (-3) "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
This interpreter is already being analyzed.
FAIL_STDLIB: (-3) "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
This interpreter is already being analyzed.
2016-07-14T16:38:30 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2016-07-15T09:48:26 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2016-07-15T12:05:09 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2016-07-18T13:32:01 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2016-07-19T10:51:28 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
FAIL_STDLIB: (-3) "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
This interpreter is already being analyzed.
FAIL_STDLIB: (-3) "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
This interpreter is already being analyzed.
2016-07-19T11:07:17 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Anaconda2\python.exe /library C:\Anaconda2\lib /outdir "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""


Environment Analysis Logs: 
Python 64-bit 2.7
Error reading: System.IO.IOException: The process cannot access the file 'C:\Users\Kevin\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean checkHost)
   at System.IO.File.InternalReadAllText(String path, Encoding encoding, Boolean checkHost)
   at System.IO.File.ReadAllText(String path)
   at Microsoft.PythonTools.Interpreter.PythonInterpreterFactoryWithDatabase.GetAnalysisLogContent(IFormatProvider culture)

[KevinLucidyne]

PyCharm works just fine on the same configuration that PTVS chokes on above.

[zooba]

@KevinLucidyne It looks like you had an analysis running when you grabbed that log - can you wait for it to finish and get the diagnostic info again? I'd expect it to be too long to reasonably post in an issue (hence my usual gist recommendation).

[zooba]

@ryan413 Probably HKEY_CURRENT_USER\Software\Python\PythonCore\3.5, but have a poke around and see if you can definitively identify it as pointing at the wrong place. Simply deleting the keys won't fix anything - you'll need to reinstall just one Python 3.5 interpreter.

[KevinLucidyne]

@zooba It says I'm running an analysis even after I restart the machine and open the Python environment as the first thing I do.

[KevinLucidyne]

@zooba Here's what the complete diagnostic dump looks like after I run "Refresh DB" and it stops analyzing and goes back to the "Completion DB needs refresh" prompt.

Bug1439.txt

[zooba]

Thanks! It looks like something in the version of astropy you have is causing us to overflow the call stack. Can you get me the exact version so I can reproduce it on my own machine?

[KevinLucidyne]

astropy 1.2.1. The same as what is packaged with the latest version of Anaconda2.

[ryansmccoy]

just installed vs 15 enterprise 5 and refresh not working again...

[zooba]

I think we know about one issue here where the button visibility doesn't update properly, but restarting VS should fix that.

If you change that "Overview" dropdown to "Intellisense" it will show you the actual status. If everything has a tick next to it then you're probably hitting this problem.

[ryansmccoy]

Well.. it was either that or deleting appdata\Python Tools folder that did trick. Works now.

[Nilster]

In my case, I created a new python environment and pointed to the existing python set up and then refreshed the intellisence and that worked. I think the problem in my case was that I had upgraded visual studio to a newer version and existing intellisense db was thinking of it as older version of the visual studio. Try deleting the current environment and creating a new one with exact settings.

[zooba]

@Nilster You may be right on that one. We did change how we identify that a db is out of date, and while it shouldn't be a problem for normal environments it may still be an issue with older virtual environments. I'll check that out

[zooba]

The workaround of clearing the DB entirely seems to work well enough for now, and we hope to remove the completion DB entirely in the near future.