Notebook: Bug switching backend between inline and qt or notebook

[tritemio]

I routinely switch back and forth between notebook and inline backends. Occasionally the notebook backend does not show the figure but an empty space, and a static image is shown in the output area below.

Usually when this happens I get messages like this in the terminal:

[IPKernelApp] ERROR | No such comm: c1fa5e1e9de8413692476a2c8fdf34bb

It seems to me that the probability of seeing this bug increases with "heavy" plots. I tried to create a small notebook showing the problem:

https://gist.github.com/tritemio/a4554323fe0faca85c57

However most of the times the previous notebook only triggers the No such comm message but the notebook figure still shows correctly.

I have a much more complex code that triggers the bug much more often though.

attn: @tacaswell

[tacaswell]

Switching backends closes all open figures which does not really fit with the model of the notebook backend.

Why are you doing this? If you just hit the x (soon be be power button) the live figure will turn into a png.

Have you tried this with mpl1.5.0rc1?

[tritemio]

@tacaswell, my use-case is an interactive session, for some heavy plots I'm still using the inline backend, then I switch to notebook when I need interactivity to explore a graph. The issue is that sometimes I execute a cell with a plot and instead of the interactive figure I get a blank space and a static image (see figure below). I'm not talking about the previous figures getting closed when I switch to inline. In principle, I could always use notebook adding a close() command to all the "heavy" plots, then remove it when I want interactivity. In this way I would avoid switching backend.

I'm using matplotlib 1.5rc1, python 3.4, jupyter notebook 4.04 (I don't think this is a regression, I had this kind of hiccups for a long time and this issue slowed me down to switch to notebook).

Here a screenshot:

[tacaswell]

Seems like a weird race condition. That is fun.

Are you seeing performance issues between inline and notebook? They are running identical rendering code (well, notebook uses a high default DPI so it might be a bit slower) and are both sending you a png over a comm.

[tritemio]

I think I have a more reproducible example: run %matplotlib inline twice in two separate cells then run %matplotlib notebook. The next plot should show the issue like in the previous screenshot. On top of this there are 8-10 lines of [IPKernelApp] ERROR | No such comm: ....

• Updated notebook showing the issue

EDIT: typos

[tritemio]

In addition this exact bug happens also switching between inline and qt backends.

This is a bigger problem for me because:

a. I need to switch to QT for some fast visualization widgets (i.e. scrolling big timetraces, etc..)
b. I cannot use the notebook backend in this case (only inline allows switching back and forth)
c. It's easy to run %matplotlib inline twice in a row during an interactive session, thus triggering this bug.

[tritemio]

@tacaswell, can you reproduce the bug? Regarding generating static images with the notebook backend (i.e. close the figure but display the png) what is the right approach? Currently I'm adding:

fig = plt.gcf()
plt.close('all')
display(fig)

to plot cells. This seems cumbersome and leaves al lot of empty space like in the previous screenshot.

[tritemio]

@tacaswell

Are you seeing performance issues between inline and notebook?

The performance issue is when a plot uses a lot of memory so I just want to create (and display) a PNG, without keeping the figure open. Also, since notebook uses PNG rendering, things like fast scrolling will not work and I still need to resort to qt backend.

[ellisonbg]

@tacaswell is there anything for us to do on this on our side?

[Carreau]

Hi all, this has been dormant for many moth now, what's the status ?
Otherwise I suggest we close it to keep the list of issue we work on clean. Reopening if needed.

Thoughts ?

[tritemio]

@Carreau, it is still a bug for me. You can try to reproduce it yourself by running the previously linked notebook, pasted again here for your convenience:

• Updated notebook showing the issue

[tritemio]

If nobody can work on it at the moment you can tag it for a "future" release and also add a tag like help-wanted, or something.

[Carreau]

Marked as 5.0. It might be an IPython thing though as well.

[thomasaarholt]

I thought I'd bring this up again. I too routinely switch between inline and qt4, and I find I need to run

%matplotlib
%matplotlib qt4

in a cell a few times before it "catches", when switching away from inline. On mac, I haven't gotten the interactive %matplotlib notebook working yet, so I'm stuck using those two.

[tritemio]

Update: this bug is still present using pyqt and qt version 5 (QT5), the current version in anaconda.
If anything it got even worst. Now I can reproduce it on macOS and Windows using any plot function.

[astrojuanlu]

I'm also having this problem, and someone reported a similar issue on Stack Overflow: http://stackoverflow.com/questions/41125690/matplotlib-notebook-showing-a-blank-histogram

[tritemio]

Just a temporary workaround. When switching from inline to notebook backend, you can run two cells with %matplotlib notebook and plt.plot([]) respectively. Rerun the previous 2 cells (usually 3-4 times) until the figure appears. Once the figure appears, Matplotlib will work without issues for the rest of the session. Not a great solution, but better than nothing if you desperately need an interactive backend.

[thomasaarholt]

@tritemio try just calling

%matplotlib
%matplotlib notebook

a bunch of times in a cell, holding ctrl whilst pressing return. That should do the same as you suggest, but without having to wait for the plot.

[ncclementi]

Is this issue still an issue? any temporary solutions?
I'm trying to go from %matplotlib inline to %matplotlib notebook and the second display doesn't render.

[thomasaarholt]

@ncclementi That should render if the first %matplotlib you call is %matplotlib notebook. Otherwise it'll initiate qt4, qt5 or tk depending on what you have installed.

After calling the notebook first, you can freely switch between it and the inline one.

[ncclementi]

@thomasaarholt
So if I do first %matplotlib inline and then switch to %matplotlib notebook that won't work. Because that is what I needed it. :(
Also I tried to do different plots using just %matplotlib notebook but they all overwrite the initial cell, is that the way it is suppose to behave. How can I have separate cells plots all interactive?

[thomasaarholt]

Try notebook followed by inline in the first two lines of a cell?

[ncclementi]

• Tried notebook and then inline in separate cells, and the notebook one doesn't render.
• Then I tried notebook followed by inline in the first two lines of a cell, and the result is like having only inline, the plots are not interactive.

[thomasaarholt]

Hang on - if you just want interactive plots, you shouldn't be using the inline at all.

The naming is slightly confusing - both inline and notebook are "inline" plotting backends - that is, they plot within the page. However, only notebook is interactive.

So all you should need to do is %matplotlib notebook, unless I am misunderstanding what you want to do.

[ncclementi]

At the beginning I needed both. It was for the purpose of teaching both possibilities. Then I tried just to use %matplotlib notebook, but that result in all my plots being plot on the same cell even though I have them on separate ones; and I need them separately. I figure that to that that I can shut down interactivity as I go, but that is inconvenient for my purpose.

[tacaswell]

If you want a new figure using notebook do

fig, ax = plt.subplots()

and then use ax for you subsequent plotting. This way you can control what goes to which figure and update the same plot from multiple cells. This behavior is much closer to the 'normal' behavior in the IPython terminal.

[ncclementi]

Hi @tacaswell

Thanks a lot, that worked!!

[ImportanceOfBeingErnest]

There is a nice analysis about the switching to %matplotlib notebook part in https://github.com/shoeffner/cvloop/issues/20#issuecomment-340292628

[egorananyev]

I thought I'd bump this. This is still very much an issue, and it looks like a lot of people would benefit from a solution!