Notebook: All notebook data is lost after renaming

[Avel7884]

So after trying to rename notebook like some franchise movie "Notebook: Return of the Jedi.ipynb" jupyter rename it just "Notebook" and made it empty file, basically deleting it. Same happened in .ipynb_checkpoints. Apparently it happened because name contained ":" symbol, that is prohibited in file names in windows. But still, I lost several hours of work and i'm sad.

[kevin-bates]

@Avel7884 - I'm really sorry to hear about this, its very unfortunate. In looking into this, you've entered the realm of "Alternate Data Streams" on Windows. I found this SO link helpful and its solution includes a more detailed link regarding ADS.

In playing around with this, I found the rename operation _from within the notebook_ prevents renames that include slashes ('/', '\') and colon (':'). Unfortunately, the Rename operation from the general directory listing does not - and that's a bug. I'm hoping someone may be able to submit a pull request to address that - since the two areas should be similarly behaved.

However, in reading about ADS, there may be hope to salvage your content. Within a Windows command prompt, please navigate to the folder in which "Notebook: Return of the Jedi.ipynb" should reside and enter dir /r. This exposes hidden files such as ADS-based files. In that listing, I suspect you'll see Notebook (with 0 bytes) but (hopefully) a file named "Notebook: Return of the Jedi.ipynb$DATA" (with non-zero bytes). If so, try to open the file (without the $DATA suffix) using notepad...

notepad Notebook: Return of the Jedi.ipynb

I found no double-quoting was necessary.

You should then see your raw notebook data. Use notepad to 'Save as...' and provide a name like 'hope_this_works.ipynb'. By default, it looks like this goes to the Documents folder, so you might want to change the path to where your notebook files reside.

Once saved, go back into Jupyter Notebook and you should see 'hope_this_works.ipynb' in the directory listing. Open that notebook and (fingers crossed), your notebook should be restored.

Of course, if dir /r didn't produce non-zero byte content for "Notebook: Return of the Jedi.ipynb$DATA", then you might be out of luck, unless others have ideas.

Good luck.

[Avel7884]

Yes, it worked! Thank you, you really saved the day!

[kevin-bates]

That's great news! Had you deleted the zero-length "Notebook" file, I believe the ADS would have been removed as well. Really glad to hear you got your work back.

I think this issue needs to be addressed in the server code as well - as a catch-all and for cases where folks are submitting requests directly via the REST api. I can look into that.

Hopefully someone with more front-end knowledge (than my zero) can add some validation in this area or related code path.

[jasongrout]

Perhaps validation on the server side could be added around https://github.com/jupyter/notebook/blob/43df5af2b614088b4b297fae90a70b6505b9bf84/notebook/services/contents/filemanager.py#L564-L566

[kevin-bates]

Thanks @jasongrout. I was thinking this would be done in the handler patch method since we don't know what kind of contents manager is in use. One could argue this is a file-specific issue, but because the front-end isn't in tune with what contents manager is in play, one could also argue behaviors should be consistent between front and back.

I've also noticed that slashes can be entered when renaming a folder from the notebook list and that results in multiple directories being created. As such, the set of invalid characters would likely vary depending on the model's type (directory or file).

To my first point, it might be best to forgo validation in the front end, then only add character validation to the specific contents managers - since some contents managers (even those that are file-based) may wish to allow things like colons, slashes, etc..

What are your thoughts?

[kevin-bates]

I've gone ahead and submitted a PR for this, implementing this directly in the FileContentsManager (since other managers will have differing sets of invalid characters).

I added a few more invalid chars relative to Windows, but we _could_ ratchet things back to only include ':' since the others will/should be caught at the OS level.

[Ignacio-Ibarra]

God bless @kevin-bates , you're a genius!!! You saved my work