Jupytext: Images in the output cells are not being reflected to python code

[rcshubhadeep]

What is the issue?

When I run jupytext --to py my_notebook.ipynb to convert a notebook as a python file, it removes the information from the output cells (in this case some images are there as a hex string) and thus when I convert them back to ipynb using jupytext --to notebook my_notebook.py the produced .ipynb is not the same as the one I started with (it misses the images).

Which version am I using?

version - 1.6.0 (Installed from pip)

How to reproduce this?

I am attaching a .ipynb file with this issue report. If you do exactly what I said above (convert it to python and then convert it back to .ipynb) then you will see the final .ipynb file misses in the inline (output cell) images that the original .ipynb had. (because of Guthub's issue of not letting me upload direct .ipynb files, I had zipped the original file)

Additional information

I am very new to jupytext. I tried to look online and in the documentation but could not find such a problem or anything like that. I am not sure what am I missing here. But maybe something. Please let me know if it is something trivial that I may have missed. In that case, I am sorry.

dream_exp.ipynb.zip

[mwouts]

I @rcshubhadeep , thanks for reporting this.

If you want to use Jupytext at the command line, then give a try to the --update flag. It should do what you are looking for, i.e. preserve the existing outputs in the .ipynb file.

Also, and especially since you mention that you're new to Jupytext, maybe you could give a try to paired notebooks - that's a mode when the synchronisation is taken care of automatically by Jupytext in Jupyter.

[rcshubhadeep]

Hello @mwouts Thank you so much for the reply. Here are the things

1.> I tried with the update optional parameter (like so - jupytext --update --to py dream_exp.ipynb which produced dream_exp.py file. And then I turned it back to ipynb using - jupytext --update --to notebook dream_exp.py which produced dream_exp.ipynb file) however the original images were not there anymore. (I guess you will be able to reproduce it doing what I am doing)

2.> I have not tried the sync but unfortunately, in my setting, it won't be an option. In my setting, I want to run code analysis on some code (and some of them are in jupyter) so I will convert them (hopefully) using jupytext to python file, run analysis on them and then convert them back to .ipynb files. And this process may take place minutes, hours, or even days after the original notebook is written (certainly won't take place when the notebook is being written, where the sync option should be used, if I understood properly.) Please let me know if that makes sense.

Thanks a lot in advance and looking forward 👍

[mwouts]

I tried with the update optional parameter (like so - jupytext --update --to py dream_exp.ipynb which produced dream_exp.py file. And then I turned it back to ipynb using - jupytext --update --to notebook dream_exp.py which produced dream_exp.ipynb file) however the original images were not there anymore

You need the --update parameter only for the conversion back to .ipynb. But this will not recreate the images that are missing in the .ipynb file - did you restore them after having the initial issue?

I have not tried the sync but unfortunately, in my setting, it won't be an option

FYI Paired notebooks are also available at the command line (give a try to the --set-formats ipynb,py:percent and then --sync commands)

I will convert them (hopefully) using jupytext to python file, run analysis on them and then convert them back to .ipynb files

Oh you could also be doing that with jupytext --pipe. You will learn more about that in the docs :)

[rcshubhadeep]

You need the --update parameter only for the conversion back to .ipynb. But this will not recreate the images that are missing in the .ipynb file - did you restore them after having the initial issue?

I am not sure what you mean. What I did is to first convert the .pynb to .py After that I ran non-invasive code analysis on those files (meaning that I do not change them in any way) and then I turn them back to .ipynb notebooks to .py

But here is the problem. When you convert the .ipynb to .py in the first place it does not put any information about images in the .py file. So when I turn them back to .ipynb it does not restore them back. (They were present in the first, original .ipynb file)

[rcshubhadeep]

I will check the pointers you mentioned.

[mwouts]

When you convert the .ipynb to .py in the first place it does not put any information about images in the .py file

Agreed! The .py or .md files don't include the outputs - we don't want them to be as big as the .ipynb files.

But again, jupytext --sync or jupytext --to ipynb --update will preserve the outputs that exist in the .ipynb file, so it is indeed possible to work on the .py file, and propagate that work to the .ipynb file without loosing the outputs.

[rcshubhadeep]

OK i see. However, after some experiments, I think I will be using jupytext
as a python library and work on the return of jupytext.read

But in any case, thanks a lot for your help and also for creating this
amazing library :))

On Wed, Nov 11, 2020, 12:01 AM Marc Wouts [e-mail] wrote:

When you convert the .ipynb to .py in the first place it does not put any
information about images in the .py file

Agreed! The .py or .md files don't include the outputs - we don't want
them to be as big as the .ipynb files.

But again, jupytext --sync or jupytext --to ipynb --update will preserve
the outputs that exist in the .ipynb file, so it is indeed possible to
work on the .py file, and propagate that work to the .ipynb file without
loosing the outputs.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mwouts/jupytext/issues/675#issuecomment-725019281,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIZ56HLT2GJZ7S6EZ5ISKTSPHA6BANCNFSM4TQQMRTA
.

[rcshubhadeep]

I could solve it also (apart from using Jypytext as a library) as you mentioned in the last post @mwouts Thanks again.

[mwouts]

Great news! And thank you for your feedback. much appreciated!