Notebook: UI for adding images (etc.) to notebook file

[stevengj]

The notebook format can already include images etc. via "output_type" : "display_data" cells. However, there is currently no UI to add such cells to a notebook. As a result, if you want an image in a notebook then you need to either (a) write a code cell to read and display it (b) link to an external image file that you must then carry around with the notebook, or (c) insert an <img src="data:image/png;base64,...> tag with the base64-encoded image data (although this no longer seems to work in Jupyter 4?).

It would be much nicer to have:

• A menu item _Insert image file_ or similar that prompted you for a file name, then uploaded the file, base64-encoded it, and stuck the resulting data as a display_data cell in the notebook after the current cell.
• The ability to copy-paste and drag-and-drop images into the notebook, using the same display_data mechanism.

Note that I'm suggesting the images will appear as additional cells, not inline in markdown code (inlining the base64 data is too messy for large images, although I suppose it would be cool to be able to refer to a named output item in Markdown as if it were a separate image file), just like how display_data messages appear. The kernel is not involved.

Questions about including images in notebooks come up frequently, e.g. here, here, here, here, and in ipython/ipython#4111. Using an external file seems like the best option at the moment, but it defeats the goal of having the notebook be a stand-alone file that contains everything.

[ellisonbg]

While I like this idea I don't see a way to support it that doesn't break
some fundamental abstractions in the notebook:

• If we put it into output, the next time that cell runs, the output gets
  replaced and the image goes away.
• If we put the base64 encoded image direction into a markdown cell, that
  markdown cell becomes difficult to manage and edit.

We have added a "restart-kernel-and-run-all-cells" action in 4.1 as we want
to encourage more folks to rerun their notebooks to make sure they are
reproducible.

Obviously, we could upload the image to the direction where the notebook
sits and insert a link to a markdown cell. Would that satisfy this usage
case?

On Fri, Oct 16, 2015 at 2:41 PM, Steven G. Johnson <[email protected]

wrote:

The notebook format can already include images etc. via "output_type" :
"display_data" cells. However, there is currently no UI to add such cells
to a notebook. As a result, if you want an image in a notebook then you
need to either (a) write a code cell to read and display it (b) link to an
external image file that you carry around the notebook, or (c) insert an src="data:image/png;base64,...> tag with the base64-encoded image data
(although this no longer seems to work in Jupyter 4?).

It would be much nicer to have:

-

A menu item _Insert image file_ or similar that prompted you for a
file name, then uploaded the file, base64-encoded it, and stuck the
resulting data as a display_data cell in the notebook after the
current cell.
-

The ability to copy-paste and drag-and-drop images into the notebook,
using the same display_data mechanism.

Note that questions about including images in notebooks come up
frequently, e.g. here
http://stackoverflow.com/questions/10628262/inserting-image-into-ipython-notebook-markdown,
here
https://mail.scipy.org/pipermail/ipython-dev/2014-February/013162.html,
here
https://groups.google.com/d/msg/julia-users/gQxbpWXpTXk/mzYUoCgSldcJ,
here
http://stackoverflow.com/questions/32370281/how-to-include-image-or-picture-in-jupyter-notebook,
and in ipython/ipython#4111
https://github.com/ipython/ipython/issues/4111. Using an external file
seems like the best option at the moment, but it defeats the goal of having
the notebook be a stand-alone file that contains everything.

—
Reply to this email directly or view it on GitHub
https://github.com/jupyter/notebook/issues/613.

Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[julienr]

What about adding a new cell type that would act like markdown but could only contain inline media (base64 images, videos? and maybe others).

[ellisonbg]

That is a possibility, but adding new cell types is something we are _very_
conservative about.

On Sun, Oct 18, 2015 at 2:47 PM, Julien Rebetez [email protected]
wrote:

What about adding a new cell type that would act like markdown but could
only contain inline media (base64 images, videos? and maybe others).

—
Reply to this email directly or view it on GitHub
https://github.com/jupyter/notebook/issues/613#issuecomment-149037925.

Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[stevengj]

@ellisonbg, I understand that, but it seems like supporting embedded media that are not code outputs is something you will eventually want to support. That is, some kind of cell or notebook item that can contain an arbitrary mimebundle, but which is inserted/deleted independent of code.

Instead of creating a new cell type, how about adding a flag to display_data cells that prevents them from being cleared with other output? (Of course, you would still need a UI for deleting them, e.g. selecting them and pressing "delete" or hitting a close box or something.) But it really seems cleaner just to have a new cell type.

[stevengj]

Uploading the image to a notebook site does not satisfy this need. There are lots of ways to share notebooks besides nbviewer. I want to just be able to email someone a file, or copy it from one place to another, without worrying that I'm forgetting some file that I need to send along with the notebook.

(Especially since the only way to know what files I need is to open the notebook and look through the source of every single markdown cell. Of course, one could write a tool for this, but it still seems pre-historic. When I email someone a word-processor document, I don't need to send separate files for every image in the document.)

[ellisonbg]

OK, what about the notebook having a a section for mime type data that is
separate from the main cells that allows the storage of different mime
types by key. Then the Markdown cells could just refer to those by those
keys. It would keep the Markdown cells free of that actual data and still
embed the data into the notebook.

On Sun, Oct 18, 2015 at 7:27 PM, Steven G. Johnson <[email protected]

wrote:

Uploading the image to a notebook site does not satisfy this need. There
are lots of ways to share notebooks besides nbviewer. I want to just be
able to email someone a file, or copy it from one place to another, without
worrying that I'm forgetting some file that I need to send along with the
notebook.

(Especially since the only way to know what files I need is to open the
notebook and look through the source of every single markdown cell. Of
course, one could write a tool for this, but it still seems pre-historic.
When I email someone a word-processor document, I don't need to send
separate files for every image in the document.)

—
Reply to this email directly or view it on GitHub
https://github.com/jupyter/notebook/issues/613#issuecomment-149064196.

Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[stevengj]

@ellisonbg, even simpler, you could just have a string-keyed dictionary of mime-typed data in the Markdown cell data itself, and each markdown cell could only refer to data in its own cell by key. This avoids the need to introduce a new cell type, and I don't think it would be common to need the same image in multiple markdown cells (and in the rare case that this is needed, you could always include multiple copies of the image, the same as if you wanted to paste the same image more than once into a word-processor document).

[ellisonbg]

I like that idea. Marking this as 4.2.

[taldcroft]

I don't think it would be common to need the same image in multiple markdown cells

I find myself wanting this functionality frequently but in a slightly different context. I have a notebook with some technical analysis that generates some plots. Then I want to write a summary up top that describes key results and includes a couple of the generated plots.

Is there currently any way to include an image which is a cell output into other cells, or might this new functionality being discussed handle this case?

[stevengj]

@taldcroft, I think that would be different functionality from what is being discussed here.

[stevengj]

Closed by jupyter/notebook#621?

[julienr]

@stevengj Yes, I think so.

[minrk]

Yup, thanks!