Vscode-jupyter: Intermittent ECONNREFUSED when starting debugger in Python Interactive window

[AMALCGCG]

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur

1. conda create -n test python=3.8 jupyter
2. Launch VS Code, create a main.py file, and select the "test" Conda environment as the Python interpreter
3. Create two cells in main.py

# %%
def foo():
    a = 2 + 2
    return a

# %%
b = foo()
b

1. Click "Run Cell" in the first cell
2. Click "Debug Cell" in the second cell --> Will give ECONNREFUSED error
3. Exit from ECONNREFUSED error (click Cancel)
4. Click "Restart IPython kernel" in the Python Interactive Window
5. Repeat steps 4 and 5 --> Will work
6. Can detach/reattach debugger, and debug/run cells as often as desired
7. Restart kernel (repeat step 7), and then repeat steps 4 and 5, and will get ECONNREFUSED error again
8. This time restarting the IPython kernel won't fix it, need to restart VS Code, and repeat steps 4-8 (i.e. it will fail the first time after restarting VS Code, but restarting the kernel will fix it)

Actual behavior

ECONNREFUSED errors requiring restart of IPython kernel (the first time), and both VS Code and the IPython kernel (subsequent times)

Looking at the output of lsof -nP -iTCP | grep LISTEN, there are several python processes listening on ports, but none of them are on the port the debugger is trying to connect to (causing the ECONNREFUSED error). There appear to be python processes listening on the two subsequent ports, though. For example the debugger will show an ECONNREFUSED error on port 61910, but lsof -nP iTCP | grep LISTEN will show python processes listening on both ports 61911 and 61912. When it does work, I see processes listening on the desired port, plus the two additional ones, for example the Out variable will indicate port 62050, and there will be python processes listening on ports 62050, 62051, and 62055. (not 62052).

Note, the non-interactive (standard Python) debugger works fine every time. This is just when trying to perform "Debug Cell" or "Debug Current File in Python Interactive Window"

Expected behavior

Debugger attaches every time when working in Interactive Python mode.

Your Jupyter and/or Python environment

_Please provide as much info as you readily know_

• Jupyter server running: Local
• Extension version: 2020.8.101144
• VS Code version: 1.48.0
• Setting python.jediEnabled: false (disabled via python.dataScience.runStartupCommands, see below)
• Setting python.languageServer: None
• Python and/or Anaconda version: Python 3.8.5, Conda 4.8.4
• OS: Windows | Mac OS X 10.15.6
• Virtual environment: conda
• settings.json:

  "python.dataScience.askForKernelRestart": false,
  "python.dataScience.runStartupCommands": [
    "%config IPCompleter.use_jedi = False",
    "import warnings",
    "warnings.simplefilter(action='default')"
  ],
  "python.experiments.optInto": ["DeprecatePythonPath - experiment"],
  "python.formatting.provider": "black",
  "python.formatting.blackArgs": ["--line-length", "120"],
  "terminal.integrated.inheritEnv": true,
  "python.dataScience.interactiveWindowMode": "perFile"

Python Output

(attached)

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

[AMALCGCG]

python.log

[rchiodo]

Thanks for logging a bug. It looks like debugpy is not being found on the path (this is is in the log):

debugpy not found in path

Can you check this folder and see if there is a 'debugpy' folder there?

/Users/<username>/.vscode/extensions/ms-python.python-2020.8.101144/pythonFiles/lib/python

[rchiodo]

If it is there, can you run the first cell then run this code:

import debugpy
print(debugpy.__version__)

[AMALCGCG]

Hi - thanks for your reply and for suggesting some things to try. First, I verified that debugpy was in the extension folder - it was. Also I updated the extension version to ms-python.python-2020.8.10360, (from build 101144) and restarted VS Code to make sure the old extension folder was removed. I then verified that /Users/<username>/.vscode/extensions/ms-python.python-2020.8.103604/pythonFiles/lib/python/debugpy was there (it was).

Here is what I did (including timestamps to make it easier to navigate the attached log). This behavior appears to be deterministic, but strange - and by that I mean that it _always_ seems to: not work the first time, work the second time, then not work again on subsequent times. And the cycle repeats after restarting VS Code.

Looking at the output of lsof -nP -iTCP | grep LISTEN shows 14 instances of Python running (which all shut down when I exit my single running instance of VS Code), and the log does show some python commands from various Python interpreter locations (including other Conda environments!).

I'm happy to run other tests you suggest or send other logs.

---

python-attempt2.log

---

Launched VS Code from Mac Spotlight (not command line)

Seeing a bunch of lines of commands being run from the stuff Conda environment, which isn't the environment for this project (the correct environment is test)

Startup completes at 18:35:31

Open main.py at 18:36:04

Click "Run Cell" on first cell at 18:36:26

In Python interactive window, run import debugpy; print(debugpy.__version__) at 18:37:02

That gives "1.0.0rc2"

Click "Debug Cell" on second cell at 18:37:58

That gives ECONNREFUSED, and when I exit the modal at 18:38:06, the status bar just spins with "Executing Cell"

Click "Restart kernel" in the Python interactive window at 18:39:42

Click "Run Cell" on first cell at 18:40:19

In Python interactive window, run import debugpy; print(debugpy.__version__) at 18:40:47

That gives "1.0.0rc2"

Click "Debug Cell" on second cell at 18:41:30

That works, and the debugger attaches

Click "Continue" in debugging controls to exit debugger at 18:42:30

Click "Restart kernel" in the Python interactive window at 18:42:54

Click "Run Cell" on first cell at 18:43:28

In Python interactive window, run import debugpy; print(debugpy.__version__) at 18:44:04

That gives "1.0.0rc2"

Click "Debug Cell" on second cell at 18:44:41

That gives ECONNREFUSED, and when I exit the modal at 18:38:06, the status bar just spins with "Executing Cell"

---

I've attached the Python log for the above.

[AMALCGCG]

@rchiodo - is there any other debugging information I could provide?

[rchiodo]

If it is there, can you run the first cell then run this code:

import debugpy
print(debugpy.__version__)

What was the result of this print statement? It may be that debugpy is being picked up somewhere else first.

[AMALCGCG]

Here is the output you requested:

import debugpy
print(debugpy.__version__)

1.0.0rc2

I also checked where the module was located:

print(debugpy.__file__)

/Users/<username>/.vscode/extensions/ms-python.python-2020.8.105369/pythonFiles/lib/python/debugpy/__init__.py

That does appear to match the Python extension version I'm currently running.

Also, when I restart the kernel, I see the following warning:

/Users/<username>/miniconda3/envs/test/lib/python3.8/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: should_run_async will not call transform_cell automatically in the future. Please pass the result to transformed_cell argument and any exception that happen during thetransform in preprocessing_exc_tuple in IPython 7.17 and above.
and should_run_async(code)
'3.8.5 (default, Aug 5 2020, 03:39:04) \n[Clang 10.0.0 ]
(6, 1, 1)

Is that related?

Are you able to reproduce this issue on a Mac with Python 3.8 and the latest version of VS Code and the Python extension?

[rchiodo]

No we haven't been able to repro this issue yet. Your debugpy version looks up to date.

@karthiknadig, @int19h can you suggest how to debug ECONNREFUSED with the debugger? Is this something that the debugger logs would show a root cause for? Thanks.

[rchiodo]

No we haven't been able to repro this issue yet

Sorry spoke too soon. This helped:

   "python.dataScience.runStartupCommands": [
        "%config IPCompleter.use_jedi = False",
        "import warnings",
        "warnings.simplefilter(action='default')"
      ]

After I set that the debugger no longer worked. This line here is messing it up:

        "warnings.simplefilter(action='default')"

[int19h]

Sounds like you might be hitting https://github.com/microsoft/debugpy/issues/351?

[AMALCGCG]

Interesting. I have that line in there to work around issue https://github.com/microsoft/vscode-python/issues/11970.

For this issue, I can confirm that removing the "warnings.simplefilter(action='default')" startup command fixes the debugger.

Is there an alternate way to address https://github.com/microsoft/vscode-python/issues/11970 without breaking the debugger?

[rchiodo]

I don't believe there is a way to get the debugger to work with this warning value set. We explicitly disable warnings on startup so that the ipython kernel will run. (See this bug: https://github.com/Microsoft/vscode-python/issues/8496)

[AMALCGCG]

Ok - I see. I took that line out of the startup commands, restarted VS Code, started the Interactive Python session (by running a cell), and then manually ran import warnings; warnings.simplefilter(action="default") in another cell. The debugger seems to still be working, although I'm seeing the DeprecationWarning: should_run_async will not call transform_cell automatically in the future. message, which I can live with.

Given the above, is is possible to have the extension re-enable warnings after the Interactive Python sessions starts? Not having the IDE hide warnings is very important to us, and as an example: one of my colleagues recently lost the better part of his day because the extension swallowed a SettingwithCopyWarningwarning.

[rchiodo]

is possible to have the extension re-enable warnings after the Interactive Python sessions starts

Maybe? Can you up vote the microsoft/vscode-python#11970 issue? We might be able to do something with running code on startup but as you can see you're initial workaround is breaking stuff.

Basically we clear PYTHONWARNINGS when launching the kernel so that the kernel will actually start even if warnings are set to error. The problem then becomes how to turn it back on.

[AMALCGCG]

Just upvoted https://github.com/microsoft/vscode-python/issues/11970. I wish I could upvote more than once, as this is a big one for us. :-) Should we keep this issue open while we wait for that one to be resolved? Thanks for your help tracking down what what happening, and thanks for your work on the extension!

[rchiodo]

Actually I'm going to close this one. If the debugger can handle the warning problem it should be resolved by https://github.com/microsoft/debugpy/issues/351. Otherwise we'll track the warning problem with microsoft/vscode-python#11970

[int19h]

It can absolutely be solved, we just didn't have a specific reason to prioritize this higher before.

[AMALCGCG]

I'll keep an eye on both microsoft/debugpy#351 and https://github.com/ipython/ipykernel/issues/540. Thanks again