Vscode-jupyter: Debugging cells doesn't use the actual file with older versions of IPython

[rchiodo]

Start with this code here:

#%%
import torch 
import numpy as np 
import matplotlib.pyplot as plt 
%matplotlib inline
# some test comments here! 
#%% 
from torchvision import datasets, transforms, models

transformations = transforms.ToTensor()

transformations = transforms.Compose([transforms.ToTensor(),
                                      # for normalizing note we used 0.5, that , 
                                      # is needed since mean and std requires a tuple
                                      transforms.Normalize(mean=(0.5,),std=(0.5,))]) 
# 
dataset_train = datasets.MNIST(root='MNIST', train=True, transform=transformations, download=True)
dataset_test = datasets.MNIST(root='MNIST', train=False, transform=transformations, download=True)

import torch.utils.data as data
dataloader_train = data.DataLoader(dataset_train, batch_size=32, shuffle=True, num_workers=2)
# we do the same thing for test 
dataloader_test = data.DataLoader(dataset_test, batch_size=32,shuffle=False,num_workers=2)
print(f'test dataloader size: {len(dataloader_test)}')

See the gif here provided by @Coderx7

[rchiodo]

@Coderx7 could you run this command and tell me the output?

import ptvsd
print(ptvsd.__version__)

It might be that our ptvsd we're shipping is not being picked up.

[Coderx7]

You might try removing %matplotlib inline as the line/cell magics can also cause this problem. (Although the second cell works fine for me).

removing the %matplotlib inline, doesnt change anything!

By the way here is the console log
and the output for :

import ptvsd
print(ptvsd.__version__)

is 4.3.0

[rchiodo]

Hmm. Yeah the hashes definitely don't match. It looks like it thinks there's less lines in the real code than Jupyter does. The jupyter file shows 19 lines, whereas your cell has only 18.

console.ts:134 [Extension Host] Info Python Extension: 2019-08-07 21:33:03: start debugging
console.ts:134 [Extension Host] Info Python Extension: 2019-08-07 21:33:04: Adding hash for 7 = 4017fc73d7d6 with 18 lines
console.ts:134 [Extension Host] Info Python Extension: 2019-08-07 21:33:04: Cached data exists getEnvironmentVariables, <ipython-input-7-09441925b6a6>

Can you try adding a blank line into the real code and see what happens?

Unfortunately I don't get the same error though :(

[Coderx7]

No luck there either! it doesn't affect anything!

[rchiodo]

Another thing to try is to copy your code from the editor into this:
https://passwordsgenerator.net/sha1-hash-generator/

Then play with it until it generates the hash shown in the non highlighting error - 09441925b6a6

[rchiodo]

We're messing up something in the generation of the hash code (missing bits of text).

Perhaps you have a different encoding than utf-8?

[rchiodo]

Actually it might be a lot easier to just try a really simple line instead:

#%%
print('hello')

And see if that works.

[Coderx7]

Nothing changed!
I wrote both

#%%
print('hello')

and also

#%%
print('hello')
print('there!')

but nothing changes!
by the way looking at the console log, why do I get :
[Extension Host] Info Python Extension: 2019-08-07 22:14:06: Adding hash for 29 = c8d3efd6778c with 3 lines
is that 29 refering to the lines generated by vscode debugging?
As far as I remember it should be utf-8 . where should I check this again?

[rchiodo]

So the print cells didn't work either? Totally weird.

The '''29''' is referring to the cell execution count. Jupyter uses a scheme like for the file names. So we need both the hash and the execution count to match the file name.

To change a file's encoding, go to the command palette and type 'Change File Encoding', pick Save with encoding and pick utf-8. At least that's what mine are set to.

[Coderx7]

saving as utf-8 didnt do any good.
The hash for the generated file includes the breakpoint() as well and the full hash is :

breakpoint()
print('hello')
print('there!')

**C8D3EFD6778C**48E296799D0E3B505A2D0AE1DD46
while in the vscode only the first 12 values are shown which are : c8d3efd6778c
Is this the actual value that is being used for comparison between hashes or, is it just a trimmed version for brevity in the log file? aside from that, shouldn't the hash be calculated without including breakpoint()? since the actual file doesn't have breakpoint()
Also I couldnt create the hash vscode calculates for the file!(i.e. ipython-input-2-bb5b204da0d4). there is no hash tha starts with bb5b204da0d4!

[rchiodo]

The value is a trimmed version. The first 12 characters. breakpoint() is part of the code sent to Jupyter (so that it breaks on the first line).

Maybe jupyter isn't using sha1. What's your jupyter version number?

notebook.__version__

in a cell should print it out.

[Coderx7]

its '4.3.1'

[rchiodo]

Mine is 6.0.0

[rchiodo]

Can you create another environment with the 6.0.0 version and see if it works there?

[Coderx7]

thats exactly what I'm doing right now. I'll report back in a moment

[rchiodo]

This code actually lives in iPython. So that version matters more:

import IPython; print(IPython.__version__)

And it looks like this used to be MD5.

Old version:
https://github.com/ipython/ipython/blob/2961b531f0e7d137128e61ed6250074cc50a023a/IPython/core/compilerop.py#L58

Latest:
https://github.com/ipython/ipython/blob/1879ed27bb0ec3be5fee499ac177ad14a9ef7cfd/IPython/core/compilerop.py#L59

[Coderx7]

Updating the notebook to the latest version solved all the issues! there is no more a temporary file being generated, and the debugging happens inside the original file!
Thanks a lot for your time and generous help!

Edit:
For the record, my IPythonversion was 5.1.0 which was updated to the latest version as well. Basically these are the affected packages that were updated and thus made the issue get resolved:

Successfully installed Send2Trash-1.5.0 backcall-0.1.0 ipykernel-5.1.1 ipython-7.7.0 jedi-0.14.1 jsonschema-3.0.2 jupyter-client-5.3.1 jupyter-core-4.5.0 notebook-6.0.0 parso-0.5.1 pickleshare-0.7.5 prometheus-client-0.7.1 prompt-toolkit-2.0.9 pyrsistent-0.15.4 pywinpty-0.5.5 pyzmq-18.0.2 terminado-0.8.2 tornado-6.0.3

[greazer]

Upgrading IPython is considered a sufficient workaround.