Spyder: IPython console can't run VPython code

[BruceSherwood]

The following issue might be an IPython issue, or a VPython issue, but it seems possible that it's a Spyder issue. I've also posted this issue to the IPython repository.

I'm the principal developer of the new VPythons (plural), with the same API in the browser (glowscript.org, using the RapydScript-NG Python-to-JavaScript transpiler), in a Jupyter notebook using true Python, or, as of two days ago, in IDLE or Spyder (see vpython.org for details).

I can import vpython and run in Spyder if I choose the deprecated Python console or an external System terminal.

If I choose the IPython console, I get this output, and no 3D display:

runfile('C:/Users/Bruce/.spyder-py3/temp.py', wdir='D:/Documents/0GlowScriptWork/LocalServer')

I don't have any difficulties running from IDLE.

Since the Spyder Python console is going away, can you offer any insight into what could be the problem with trying to run a program that imports vpython, when using the IPython console? The vpython module when running outside the Jupyter notebook environment sets up an http server to serve image files (for textures on 3D objects) and font files (for extruding 3D text), and it sets up a websocket server for rapid two-way communications between the Python program and a JavaScript program in the browser that calls upon the GlowScript library for WebGL rendering.

The vpython module when not running in a Jupyter notebook makes no references to IPython other than to check whether the user is or is not running in a notebook, so the output shown above suggests that it is the IPython console itself that is unhappy.

Thanks for any advice you can offer!

[ccordoba12]

Hey @BruceSherwood, let me first say that I'm a great admirer of VPython! I'm a physicist and I used to teach simulation of dynamical systems with it. So I'm really glad that you want to solve this issue between VPython and Spyder.

If you can run VPython in our Python console but not in the IPython one, then I think you need to change the GUI backend before running any VPython code.

For that, please run

In [1]: %gui tk

or

In [2]: %gui qt5

and try again.

If that doesn't work, please post a simple VPython example so we can try it in our side. Thanks!

[BruceSherwood]

I'm delighted that you're already familiar with VPython. I tried %gui qt5 and got the same error message as before. I tried $gui tk and nothing worked. (I'll mention that I'm brand-new to Spyder, and having deleted an IPython console during these tests I don't know how to create one again -- what do I need to do?) To try this yourself:

Execute pip install vpython

Here's a minimal program that should create a white cube in a black canvas.

from vpython import *
box()

Right button drag or Ctrl-drag to rotate "camera" to view scene.
To zoom, drag with middle button or Alt/Option depressed, or use scroll wheel.
On a two-button mouse, middle is left + right.
Touch screen: pinch/extend to zoom, swipe or two-finger rotate.

Here's a zip file with lots of examples:

https://github.com/BruceSherwood/vpython-jupyter/blob/master/VPythonDemos_no_notebook.zip?raw=true

[Carreau]

The reason why is that this check in VPython make the assumption "running in a kernel == running in a notebook", which , in the case of the IPython console of Spyder is untrue. The IPython console is a QTconsole, which does not support Javascript, but uses a kernel.

Is there a way to tell VPython "no really I'm not in a notebook" ? More generally, the "I am in a kernel" and "I am in a notebook" and "I can display javascript" should be completely orthogonal questions. nteract, SageMathCloud, and JupyterLab are all notebook based and will have more restriction on JS than classic notebook.

There are hacky way to detect in which kind of notebook you are, but i'd like to avoid that for now.

I believe Console > Open an IPython Console (or ctrl-T) can open a new console.

I have other issues running VPython by forcing non-notebook :

File ~jupyter-vpython/vpython/vpython.py", line 373, in __init__
    comm.on_msg(self.handle_msg)

AttributeError: 'NoneType' object has no attribute 'on_msg'

But will try to debug this.

[BruceSherwood]

Thanks for explaining how to reacquire an IPython console. I don't understand your comment about forcing non-notebook, since vpython.py line 373 is in an if statement to be executed only if in a notebook. To force non-notebook requires setting _isnotebook = False in__init__.py and near the start of vpython.py. When I do that, and set the Spyder run configuration to "Execute in current Python or IPython console", vpython runs, but outputs to the deprecated Python console. I don't understand what I can do about the warning "NOTE: The Python console is going to be REMOVED in Spyder 3.2. Please start to migrate your work to the IPython console instead." I don't see any option in the preferences to require an IPython console.

[Carreau]

I don't understand your comment about forcing non-notebook, since vpython.py line 373 is in an if statement to be executed only if in a notebook.

My comment is that this check is not right. It does not check whether you are in a notebook. It checks wether you are in a kernel. And Spyder IPython's Console does also use a kernel. So by definition if you import vpython in Spyder's IPython console it thinks it's in a notebook, and the wrong codepath is taken.

The Kernel Equivalent to Notebook is a widespread confusion, and is a logical fallacy.

• You are in notebook implies you are in a kernel.
  The contraposition is :
• You re not in a kernel, hence you are not in a notebook.
  But the following is false:
• You are in a kernel, so you are in a notebook.
  For proof here is a non-exaustive list of frontends that do not display javascript:
• Hydrogen
• QtConsole / Spyder IPython's console.
• nbconvert
• jupyter console (terminal).

but outputs to the deprecated Python console. I don't understand what I can do about the warning "NOTE: The Python console is going to be REMOVED in Spyder 3.2. Please start to migrate your work to the IPython console instead."

I think you have to think of this warning as not for you as a developer, but for you as a user. So as a developer you cannot do much about that – except making sure your code work in the IPython console.
Spyder will show this warning wether you use vpython or just execute 1+1. It is just a warning to users: get used to the Console > Open an IPython Console and not a Pure Python console.

Does that make sens ?

[BruceSherwood]

Yes, your feedback is helpful; thanks. What check SHOULD I be making in the vpython code, now that you've explained that the check I make is not appropriate?

As for the warning to "migrate your work" I remain puzzled as a user because I don't see anything in the preferences for how to specify/require an IPython console, which would seem to be necessary for testing purposes if and when I learn how to make the right check about kernel.

[ccordoba12]

As for the warning to "migrate your work" I remain puzzled as a user because I don't see anything in the preferences for how to specify/require an IPython console

@BruceSherwood, the only thing you need to do is to give focus to the IPython console, then run a file with F5. Spyder will automatically select it to run code.

We know this is confusing and it's one of several reasons why we are removing the Python console.

[ccordoba12]

What check SHOULD I be making in the vpython code

The problem is any check you can come up with will run in the kernel, and the kernel doesn't know to which frontend is connected to (Qtconsole or Notebook). If fact, it can be connected to several different frontends at the same time.

So in theory (although @Carreau says there are some hacks) you can't reliably detect if your kernel is running in the Notebook or not.

What we could do in Spyder is to add a statement to our sitecustomize (loaded by every console before it starts to run code) that sets

_isnotebook = False

in VPython. We do several tweakings like this for other packages, so there's no problem with that.

Well, the only problem is this issue will continue to be present for plain Qtconsole, so I think it's better to see if @Carreau finds a better solution.

[BruceSherwood]

That's very generous, Carlos. If there turns out to be no better solution, the variable should not be named _isnotebook but something like (say) _isspyder, because the existing vpython code which sets _isnotebook works fine in IDLE or from a terminal. If you set _isspyder I would reference it in a try/except structure, since it wouldn't exist if not running in Spyder.

Dado que eres de Bogotá, menciono que temenos colegas en la Universidad Nacional y la Universidad de Antioquia, quienes han traducido nuestro volumen de “Mecánica Moderna” (la primera parte de “Matter & Interactions”) al español, publicado por Trillas.

[ccordoba12]

Dado que eres de Bogotá, menciono que temenos colegas en la Universidad Nacional y la Universidad de Antioquia, quienes han traducido nuestro volumen de “Mecánica Moderna” (la primera parte de “Matter & Interactions”) al español, publicado por Trillas.

Muy interesante! Gracias Bruce por contarme al respecto!

[BruceSherwood]

Is there perhaps something I could put in a try/except structure that would work or fail definitively if I'm in the Spyder environment?

[ccordoba12]

Bruce, I think you don't understand quite well how the Jupyter kernel/frontend architecture works. Kernels are separate Python processes that run independently of (but are controlled by) frontends.

So each IPython console in Spyder has its own kernel and they have no idea they were started by Spyder.

[BruceSherwood]

It is definitely the case that I understand almost nothing about the IPython machinery! However, now that you've pointed out that giving focus to an IPython console will override the Run preferences, I successfully ran in that console by unconditionally setting my variable _isnotebook to False, which is very good news So if you are indeed willing to set some variable such as _isspyder, I can use its existence in a try/except structure to set _isnotebook to False, and vpython will work. Perhaps _isspyder might be of use in other modules, too.

[BruceSherwood]

Is there any progress on what I could do to detect when Spyder is being used, such as Spyder setting some variable?

[ccordoba12]

Sorry, we're busy preparing to release Spyder 3.2.

We'll take a look at this in two months for Spyder 3.2.1

[dbo0676]

I am thankful for this bug/comment as I am using Spyder 3.2.1 and wanted to use VPython but had the same error that Bruce mentions above. I see that targeted version for the fix has been pushed back to Spyder 3.2.3. Is there an estimated date for this release?

[BruceSherwood]

It's been nearly 7 months from when the first milestone was established, which is disappointing. We're in the position of having to tell those who want to use the vpython module that they have to install Spyder 3.1.4, which is unfortunate and counter-intuitive.

[mpkuchera]

I am also disappointed that I am unable to use vpython in spyder. Following!

[ccordoba12]

@BruceSherwood, I'm very sorry for the delay, but I don't know how to solve this issue. The problem, as I mentioned above, is that the kernel doesn't know to what frontend or interface is connected to (i.e. the notebook or Spyder).

Besides, the same kernel can be connected to several frontends at the same time, so we can't rule out the notebook at the expense of Spyder, or vice versa, because now we have spyder-notebook too.

The way a library can deal with this problem (if my understanding of the Jupyter architecture is correct) is by sending several repr's when a computation has finished (e.g. repr_html and repr_text) and allowing the frontend to handle the output according to its capabilities. In the case of the IPython console inside Spyder, it discards repr_html because it can't display HTML and instead it uses other available repr's.

But it seems VPython is not working that way, right? Forgive me if I'm wrong.

[mwcraig]

I got most of the way through making this work with sitecustomize.py (though it took some time to start to wrap my head around how namespaces are handled when spyder kicks off a terminal).

I thought the latest vpython (7.4) had fixed this but can try to pick it up again (likely not until weekend).

The remaining issues I had were ones that required doing some reorganization of imports in vpython, they weren't really spyder related.

[ccordoba12]

Hey Matt, thanks a lot got helping us with this one! PR really welcomed and appreciated :-)

[ytalhatamer]

Import problem still continues with the spyder version 3.2.8 and vpython 7.4.3 . Do you know is there any solution posted for this?

[mwcraig]

@ytalhatamer -- still stuck(mostly I've been too busy). There are discussions off github about updating vpython's import hierarchy, which is fundamentally what is blocking progress on this. My semester effectively ends in a week 😀 so I'm cautiously optimistic there will be a workaround/fix by mid-May (on the vpython side at least). @BruceSherwood

[elibre82]

Hi,
I was also searching for a solution to run vpython in Spyder. I suppose that there is no solution for the moment. Is it the case ?
However, for those interested, vpython works fine with VSCode.

[mwcraig]

@elibre82 -- we are closer to a solution; a couple of the issues on the vpython side that were blocking progress have been resolved. The remaining one has to do with peacefully coexisting with the async event loop set up in Spyder. Our plan is to address that in the upcoming release of vpython 7.5.0. You can track progress at https://github.com/BruceSherwood/vpython-jupyter/issues/80

[elibre82]

Thanks for the news ! And the great job you do !!
Arnaud

[ccordoba12]

These are great news! Thanks Matt!

[bittelli]

Hello Matt, thanks a lot for your great job. I have been using and developing in Visual Python for 10 years and your work to keep it going is wonderful. Any news on running VPython in Spyder ?

[BruceSherwood]

Marco, see the instructions at vpython.org, where you'll see that you can use the old Spyder 3.1.4 with the vpython module with Python versions less than Python 3.7. But Matt is indeed working to enable using current versions of Spyder.

[mwcraig]

Quick note to everyone on this thread....we will be releasing vpython 7.5.0 shortly (Monday of next week?) and it runs in spyder.

If you want to test our beta release please follow the instructions here: https://github.com/BruceSherwood/vpython-jupyter/wiki/Instructions-for-trying-7.5.0-pre-release-versions

Also, please report any failures (or successes) as comments on this issue: https://github.com/BruceSherwood/vpython-jupyter/issues/124

Thanks for your patience while we worked out all of the kinks!

[ccordoba12]

@mwcraig, those are great news!! Thanks for sharing!

Could I ask how did you manage to make VPython to work reliably for both the notebook and Spyder?

[bittelli]

Great news Greg
thanks so much.

Marco

---

From: Matt Craig [e-mail]
Sent: Thursday, February 14, 2019 3:03:27 PM
To: spyder-ide/spyder
Cc: Marco Bittelli; Comment
Subject: Re: [spyder-ide/spyder] IPython console can't run VPython code (#4582)

Quick note to everyone on this thread....we will be releasing vpython 7.5.0 shortly (Monday of next week?) and it runs in spyder.

If you want to test our beta release please follow the instructions here: https://github.com/BruceSherwood/vpython-jupyter/wiki/Instructions-for-trying-7.5.0-pre-release-versions

Also, please report any failures (or successes) as comments on this issue: BruceSherwood/vpython-jupyter#124https://github.com/BruceSherwood/vpython-jupyter/issues/124

Thanks for your patience while we worked out all of the kinks!

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/spyder-ide/spyder/issues/4582#issuecomment-463638050, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYYsbi1TGzGOTQvZxUuoxuJBvcJJ2juCks5vNWyvgaJpZM4N2HL9.

[mwcraig]

@ccordoba12 -- There were a few changes needed. We check whether we are in spyder by checking for any environment variable name that includes SPYDER. Since recent versions of ipython/ipykernel enable asyncio be default and start an event loop running we also had to rewrite our async code so that we make a separate loop in its own thread. This was a change that was overdue anyway.

The notebook code didn't require a ton of updating, except for running our own event loop their too, now.

We also added JupyterLab compatibility, and did a lot of python code refactoring to defer creating a viewer from import time until the user executes code that creates an object. That will improve the experience for spyder users quite a bit because browser tabs used to open for them unexpectedly as soon as they type import vpython.

The next big task is refactoring the javascript to reduce duplication between the notebook, non-notebook and lab versions of the code.

We just tagged 7.5.0, so I think this issue can be closed now.

[ccordoba12]

Ok, thanks a lot for your help to solve this one!