Jupyter-book: Inquiry: Plotly interactive plots in a Jupyter Books?

[mathieuboudreau]

Has anyone tried this? Or is there any known reason why this wouldn't be feasible?

As far as I know, Plotly generates standalone javascript/html as an output in the ipynb, and the instructions here say this but are also a bit vague on if there's other compatibility issues:

Interactive outputs
We can even do the same for interactive material. Below we’ll display a map using ipyleaflet. When the notebook is converted to Markdown, the code for creating the interactive map is retained.
Note that this will only work for some packages. They need to be able to output standalone HTML/Javascript, and not depend on an underlying Python kernel to work.

[choldgraf]

I've never tried, but you should! :-) let us know how it goes and we can try to figure something out if anything is finnicky

[mathieuboudreau]

@choldgraf I gave it a shot ! : https://mathieuboudreau.github.io/introML-book/01/mvp_sos.html

After running all the cells, the output is only a blank canvas:

Unlike the very nice interactive figure I get running the same script in a Jupyter Notebook:

I know because Plotly uses javascript, that by default MyBinder sessions don't render it (notebooks aren't trusted by default); it only displays if your rerun the notebook. I'm wondering if there's anything permission-wise in the Jupyter Book implementation that might be blocking that Javascript? Or maybe the javascript generated by Plotly doesn't render for another reason?

[mathieuboudreau]

p.s. that Jupyter Book might appear funky to you; it's because I'm using the ScriptOfScript kernel, which allows multiple languages to be used in a notebook, so it's actually running code in Octave/MATLAB then transfering data to Python. Had to manually tweek a few config/MD files to get it working, but just an FYI in case it caught you off guard.

[choldgraf]

ahhh I see, I thought you meant just embedding the output of a cell in the jupyter-book markdown (like here: https://jupyter.org/jupyter-book/features/notebooks.html#interactive-outputs)

Using thebelab is a bit trickier, since Thebelab needs to build in the ability to return HTML-like data, and currently it only works for PNGs. I think this issue might be the place to check: https://github.com/minrk/thebelab/issues/111

[mathieuboudreau]

Ah ha, ok thanks for clarifying! Will keep an eye on the topic!

[emdupre]

Looking through the book -- it looks like the output is not rendering even on the initial execution. Is that right, @mathieuboudreau ?

[mathieuboudreau]

Yeah @emdupre - it's like that in my Jupyter Notebooks as well. For those, from what I understood it's that MyBinder makes all notebooks "not trusted", so they never display. I assumed this might be contributing to that as well here, but maybe there's other things at play (e.g. missing javascript blocks in the HTML?).

[mathieuboudreau]

On the other hand, here is a link to an HTML outputted by SoS and Jupyter, which renders Plotly fine: https://s3.ca-central-1.amazonaws.com/qmrlab-blogs/InversionRecovery.html

[choldgraf]

yeah in general plotly should work fine on Binder I think?

[emdupre]

@choldgraf should correct me if I'm wrong, but if the initial execution generates valid HTML (that kramdown can parse), then I'd think it should appear in the initial HTML -- even if it's not (yet) re-executable ? You don't have any kind of error output, though, which I find a little odd. Can you share what your intermediate markdown file looks like for that cell ?

[mathieuboudreau]

Here's a link to the markdown file: https://github.com/mathieuboudreau/introML-book/blob/master/content/01/mvp_sos.ipynb

And for reference, here's a link to the working exported HTML (SoS+Jupyter): https://s3.ca-central-1.amazonaws.com/qmrlab-blogs/mp2rage/mvp_sos.html

The relevant block of code in the markdown too huge to past here because of the raw data in it.

[mathieuboudreau]

Oops, the above link is to the notebook, here is the markdown file: https://github.com/mathieuboudreau/introML-book/blob/master/_build/01/mvp_sos.md

[mathieuboudreau]

Is it possible that this block of code (present in both HTML and md files) isn't being executed/parsed correctly for the markdown version?

<div markdown="0" class="output output_html">
<script type="text/javascript">window.PlotlyConfig = {MathJaxConfig: 'local'};</script><script type="text/javascript">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: "STIX-Web"}});}</script><script>requirejs.config({paths: { 'plotly': ['https://cdn.plot.ly/plotly-latest.min']},});if(!window._Plotly) {require(['plotly'],function(plotly) {window._Plotly=plotly;});}</script>
</div>

[emdupre]

In the linked notebook (thank you !), all the cells except the Plotly cell have output. Is that expected ? Is there a way to get output for that cell in the notebook itself ?
How did you render the HTML ? Just with nbconvert --to html ?

[mathieuboudreau]

In the linked notebook (thank you !), all the cells except the Plotly cell have output. Is that expected ?

By default, MyBinder makes all notebooks "not trusted", so the plotly output will not show. You need to rerun all the cells for it to appear in the Jupyter Notebook running on MyBinder. So I think that's expect, if that's what you were asking?

As for generating the HTML, after you've run the notebook, save (disk icon, top left), then click the dropdown menu on the left panel, and select the "%sossave --to html force" option. See this timestamped videoclip for an example: https://youtu.be/wD694KBiY-E?t=171

[mathieuboudreau]

@emdupre Oops, if you're in the notebook right now, you will need to reload the docker instance in a few minutes. I forgot to save a tag for that cell; it needs to have report_output for it to be exported in html.

[mathieuboudreau]

Ok, the updated commit of my repo has the correct tag on it now, if you want to try and export the HTML with my instructions above in a new MyBinder session. Sorry about that.

[mathieuboudreau]

Ok, so @TommyBoshkovski is amazing.

He was able to get it running, with two caveats. 1) it still doesn't show on first load, and 2) this implementation requires that Plotly secret keys are executed in a call in the first cell, so obviously I can't commit and push them in my repo.

But at least, it's possible.

Two changes are required. First cell needs to be replaced with:

%use python3
import matplotlib.pyplot as plt
import numpy as np
import plotly.plotly as py
import plotly as pl
import plotly.graph_objs as go
from plotly import __version__
from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
pl.tools.set_credentials_file(username='SECRET_NAME', api_key='SECRET_KEY')
#init_notebook_mode(connected=True)
config={'showLink': False, 'displayModeBar': False}

init_notebook_mode(connected=True)

from IPython.core.display import display, HTML

And in my last cell, iplot needs to be replaced by py.iplot.

[mathieuboudreau]

Still can't get it running using Plotly's offline mode (free), but if you think this is sufficiently resolved my original issue topic which wasn't that specific, feel free to close it since at least we know it's possible.

[choldgraf]

hmmm, do you think that:

1. This is satisfying solution to your problem? If no, then...
2. Will this be actionable at all on jupyter-book's side, if "yes" then...

let's keep it open.

Otherwise, let's close it :-)

[mathieuboudreau]

Those are both unknowns to me for now, I will get back in touch when I know! Thanks!

[mathieuboudreau]

Also note that I'm discussing this in a thread on the Plotly forums as well: https://community.plot.ly/t/plotly-graphs-in-jupyter-books-possible/18901/4

[jonmmease]

Below we’ll display a map using ipyleaflet ...

ipyleaflet displays visualizations as a jupyter widget, and if Jupyter Book can handle widgets, then it might be worth trying to display the plotly figure using a FigureWidget (https://plot.ly/python/figurewidget/). In this case you don't use iplot, you use a go.FigureWidget instance and let it display itself.

[mathieuboudreau]

Thanks for the tip @jonmmease !

I just tried now in my live demo (https://mathieuboudreau.github.io/introML-book/01/mvp_sos.html).

A few notes:

1 - The widget works fine in Jupyter Notebook running on MyBinder (link at the top of the live demo)
2 - Only a blank canvas is shown with the Jupyter Book is first loaded.
3 - When trying to run using the Interactive Inline mode in my Jupyter book, I get the following error:

Error displaying widget
No renderer could be found for output. It has the following MIME types:

4 - Looking at the markdown file generated by make book, the output of the widget from the Jupyter Notebook seems to only be partially converted, only the following lines are shown after the python code block:

{:.output .output_data_text}
```
FigureWidget({
'data': [{'colorscale': 'Greys',
'name': 'Signal',
'showscale':…
```

Which in contrast, the Jupyter Book ipyleaflet example seems to have exported a proper iframe wrapped around some divs (see here)

@choldgraf any clues on why the Plotly widget isn't being converted to an iframe like ipyleaflet is with make book, or why the interactive inline is complaining about the renderer?

[mathieuboudreau]

Hmm comparing the two raw ipynb, looks like it's because ipyleaflet outputs to html, whereas the widget is just what it is: a widget but not html. I can (and have) export my widget to html (which loads fine in a browser) using Widgets -> Embed Widgets in the Jupyter Notebook toolbar, which could then be displayed as using iframe. But to do this automatically, it appears that this is a work in progress that might be merged soon..

p.s. (@jonmmease I saw this related post you made on the plotly forums; it worked for me, but I would need that the HTML output be produced only for the single widget cell, not the whole document).

[mathieuboudreau]

SOLVED.

I found out by looking at the Plotly documentation that while plotly.offline.iplot() does not output HTML, plotly.offline.plot() does by saving it to a file. So by then loading that file with display(HTML("FILENAME.html")), it worked, meaning 1) the plot is now displayed in the Jupyter Book when it is first loaded (live demo), 2) the Plotly HTML is loaded/displayed correctly by running the inline code in the Jupyter Book,and 3) credentials aren't needed (still in Plotly offline mode), so no worries about public keys being shared.

e.g. this line:

iplot(fig, filename = 'figure_1.html', config = config)

was replaced with these lines:

plot(fig, filename = 'figure_1.html', config = config)
display(HTML('figure_1.html'))

Note that the Plotly figure does not display when MyBinder Jupyter Notebook is first opened, but it didn't for iplot either. It also does not display using the above commands when the cells of the Jupyter Notebook are run, so iplot... should be used instead (either an if..else, or have it commented it out and instruct the user to switch them).

Thanks @choldgraf @emdupre @jonmmease @TommyBoshkovski for all your input and help!

[choldgraf]

That's great information! Any chance that you'd be willing to add a little bit about this to the FAQ page? https://jupyter.org/jupyter-book/guide/06_faq.html :-)

[mathieuboudreau]

Will do!