Mne-python: BUG: Mayavi hangs forever

[kingjr]

Impossible to make mayavi/pysurfer work on an ubuntu 16.10

I followed the recommended installation steps:

• conda install mayavi
• pip install PySurfer
• export ETS_TOOLKIT=qt4 and export QT_API=pyqt

But mayavi's scene hangs forever with this:

import mne
from mne.datasets import sample
data_path = sample.data_path()
fname = data_path + '/MEG/sample/sample_audvis-meg'
subjects_dir = mne.datasets.sample.data_path() + '/subjects'
stc = mne.read_source_estimate(fname)
stc.plot(subject='fsaverage', subjects_dir=subjects_dir)

Here is the system info:

mne.sys_info()
Platform:      Linux-4.8.0-49-generic-x86_64-with-debian-stretch-sid
Python:        2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15)  [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Executable:    /home/jrking/anaconda2/bin/python

mne:           0.15.dev0
numpy:         1.11.3 {lapack=mkl_intel_lp64, blas=mkl_intel_lp64}
scipy:         0.18.1
matplotlib:    1.5.1

sklearn:       0.19.dev0
nibabel:       2.1.0
mayavi:        4.5.0
pycuda:        Not found
skcuda:        Not found
pandas:        0.19.2

launching ipython with pylab=qt or qt4 makes PySurfer crash

/home/jrking/anaconda2/lib/python2.7/site-packages/pyface/qt/__init__.py in prepare_pyqt4()
     15     # Set PySide compatible APIs.
     16     import sip
---> 17     sip.setapi('QDate', 2)
     18     sip.setapi('QDateTime', 2)
     19     sip.setapi('QString', 2)

ValueError: API 'QDate' has already been set to version 1

Any idea where I should go from here?...

related issues and pages: https://github.com/mne-tools/mne-python/issues/3280, https://github.com/mne-tools/mne-python/issues/760, http://mne-tools.github.io/stable/install_mne_python.html, http://mne-tools.github.io/stable/advanced_setup.html

[kingjr]

Note that the following troubleshooting:

If you get an error saying:

ValueError: API 'QDate' has already been set to version 1

you have run into a conflict with Traits. You can work around this by telling the interpreter to use QtGui and QtCore from pyface:

>>> from pyface.qt import QtGui, QtCore

doesn't work for me

[agramfort]

maybe a pyface version problem?

I have 5.1.0

[jasmainak]

oh yes, I have the same issue as @kingjr and I haven't used mayavi on my laptop in many months ...

[kingjr]

Same error on an ubuntu 14.04 and this setup

Platform:      Linux-3.13.0-85-generic-x86_64-with-debian-jessie-sid
Python:        2.7.12 |Anaconda 2.3.0 (64-bit)| (default, Jul  2 2016, 17:42:40)  [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Executable:    /home/jrking/anaconda/bin/python

mne:           0.15.dev0
numpy:         1.11.3 {lapack=mkl_intel_lp64, blas=mkl_intel_lp64}
scipy:         0.19.0
matplotlib:    1.5.1

sklearn:       0.18.1
nibabel:       2.0.2
mayavi:        4.4.0
pycuda:        2016.1
skcuda:        0.5.2
pandas:        0.18.1

Same error with pyface upgrade to 5.1.0

[kingjr]

Note that by default, conda comes with Qt5 now, so we have to explicitly downgrade with conda install pyqt=4 to make it compatible with pysurfer...

[larsoner]

For you Linux folks does it bomb if you do ETS_TOOLKIT=wx?

[larsoner]

then you don't have to mess with Qt at all, but you'll need conda install wxpython

[kingjr]

wx seems to solve the problem for me + ipython --pylab=wx

@jasmainak you can close the issue if it solves it for you (and update the doc troubleshooting ;))

[kingjr]

(although now I regularly get an error that kicks me out of ipython
The program 'ipython' received an X Window System error. This probably reflects a bug in the program. The error was 'BadWindow (invalid Window parameter)'. (Details: serial 159 error_code 3 request_code 15 minor_code 0) (Note to programmers: normally, X errors are reported asynchronously; that is, you will receive the error a while after causing it. To debug your program, run it with the --sync command line option to change this behavior. You can then get a meaningful backtrace from your debugger if you break on the gdk_x_error() function.) )

[jasmainak]

I get this:

(python:11120): Gtk-WARNING **: gtk_disable_setlocale() must be called before gtk_init()
********************************************************************************
WARNING: Imported VTK version (7.0) does not match the one used
         to build the TVTK classes (6.3). This may cause problems.
         Please rebuild TVTK.
********************************************************************************

Xlib:  extension "GLX" missing on display ":0.0".
Xlib:  extension "GLX" missing on display ":0.0".
Xlib:  extension "GLX" missing on display ":0.0".
---------------------------------------------------------------------------
PyAssertionError                          Traceback (most recent call last)
/home/mainak/Desktop/projects/github_repos/mne-python/examples/connectivity/plot_mne_inverse_coherence_epochs.py in <module>()
    143                      time_label='Coherence %0.1f Hz',
    144                      subjects_dir=subjects_dir,
--> 145                      clim=dict(kind='value', lims=(0.25, 0.4, 0.65)))
    146 brain.show_view('lateral')

/home/mainak/Desktop/projects/github_repos/mne-python/mne/source_estimate.pyc in plot(self, subject, surface, hemi, colormap, time_label, smoothing_steps, transparent, alpha, time_viewer, subjects_dir, figure, views, colorbar, clim, cortex, size, background, foreground, initial_time, time_unit)
   1342                                       foreground=foreground,
   1343                                       initial_time=initial_time,
-> 1344                                       time_unit=time_unit)
   1345         return brain
   1346 

/home/mainak/Desktop/projects/github_repos/mne-python/mne/viz/_3d.pyc in plot_source_estimates(stc, subject, surface, hemi, colormap, time_label, smoothing_steps, transparent, alpha, time_viewer, subjects_dir, figure, views, colorbar, clim, cortex, size, background, foreground, initial_time, time_unit)
   1132                       background=background, foreground=foreground,
   1133                       figure=figure, subjects_dir=subjects_dir,
-> 1134                       views=views)
   1135 
   1136     if time_unit == 's':

/home/mainak/anaconda2/lib/python2.7/site-packages/surfer/viz.pyc in __init__(self, subject_id, hemi, surf, curv, title, cortex, size, background, foreground, figure, subjects_dir, views, show_toolbar, offscreen, config_opts)
    393         self._set_window_properties(size, background, foreground)
    394         figures, _v = _make_viewer(figure, n_row, n_col, title,
--> 395                                    self._scene_size, offscreen)
    396         self._figures = figures
    397         self._v = _v

/home/mainak/anaconda2/lib/python2.7/site-packages/surfer/viz.pyc in _make_viewer(figure, n_row, n_col, title, scene_size, offscreen)
    210             # Triage: don't make TraitsUI if we don't have to
    211             if n_row == 1 and n_col == 1:
--> 212                 figure = mlab.figure(title, size=(w, h))
    213                 mlab.clf(figure)
    214                 figures = [[figure]]

/home/mainak/anaconda2/lib/python2.7/site-packages/mayavi/tools/figure.pyc in figure(figure, bgcolor, fgcolor, engine, size)
     83                     return scene
     84             else:
---> 85                 engine.new_scene(name=name, size=size)
     86                 engine.current_scene.name = name
     87         figure = engine.current_scene

/home/mainak/anaconda2/lib/python2.7/site-packages/apptools/scripting/recordable.pyc in _wrapper(*args, **kw)
     43         if not record:
     44             # If the method was not recorded, just call it.
---> 45             result = func(*args, **kw)
     46 
     47         return result

/home/mainak/anaconda2/lib/python2.7/site-packages/mayavi/core/engine.pyc in new_scene(self, viewer, name, **kwargs)
    450                     factory_kwargs[arg] = value
    451 
--> 452             viewer = self.scene_factory(**factory_kwargs)
    453             process_ui_events()
    454 

/home/mainak/anaconda2/lib/python2.7/site-packages/mayavi/core/ui/mayavi_scene.pyc in viewer_factory(size)
     87     viewer.menu_bar_manager = None
     88     viewer.size=size
---> 89     viewer.open()
     90     return viewer
     91 

/home/mainak/anaconda2/lib/python2.7/site-packages/pyface/i_window.pyc in open(self)
    164 
    165         if self.control is None:
--> 166             self._create()
    167 
    168         self.show(True)

/home/mainak/anaconda2/lib/python2.7/site-packages/pyface/ui/wx/application_window.pyc in _create(self)
    149         if AUI:
    150             body = self._create_body(self.control)
--> 151             contents = self._create_contents(body)
    152             body.GetSizer().Add(contents, 1, wx.EXPAND)
    153             body.Fit()

/home/mainak/anaconda2/lib/python2.7/site-packages/tvtk/tools/ivtk.pyc in _create_contents(self, parent)
    402         """ Create the contents of the window. """
    403 
--> 404         self.scene = self._scene_factory(parent)
    405 
    406         return self.scene.control

/home/mainak/anaconda2/lib/python2.7/site-packages/mayavi/core/ui/mayavi_scene.pyc in mayavi_scene_factory(parent)
     66     appropriately set."""
     67     p = get_scene_preferences()
---> 68     s = MayaviScene(parent, stereo=p['stereo'])
     69     set_scene_preferences(s, p)
     70     return s

/home/mainak/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/decorated_scene.pyc in __init__(self, parent, **traits)
     67     ##########################################################################
     68     def __init__(self, parent, **traits):
---> 69         super(DecoratedScene, self).__init__(parent, **traits)
     70         self._setup_axes_marker()
     71 

/home/mainak/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/scene.pyc in __init__(self, parent, **traits)
    283 
    284         # Base class constructor.
--> 285         super(Scene, self).__init__(parent, **traits)
    286 
    287         # Setup the default picker.

/home/mainak/anaconda2/lib/python2.7/site-packages/tvtk/pyface/tvtk_scene.pyc in __init__(self, parent, **traits)
    207         self._def_pos = 1
    208 
--> 209         self.control = self._create_control(parent)
    210         self._renwin.update_traits()
    211 

/home/mainak/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/decorated_scene.pyc in _create_control(self, parent)
    105 
    106         # Ensure the child controls are laid-out.
--> 107         self._sizer.Layout()
    108 
    109         return self._panel

/home/mainak/anaconda2/lib64/python2.7/site-packages/wx-3.0-gtk2/wx/_core.pyc in Layout(*args, **kwargs)
  14839         removed.
  14840         """
> 14841         return _core_.Sizer_Layout(*args, **kwargs)
  14842 
  14843     def ComputeFittingClientSize(*args, **kwargs):

PyAssertionError: C++ assertion "m_widget" failed at ./src/gtk/window.cpp(2737) in DoSetSize(): invalid window

[larsoner]

@jasmainak you have a misconfigured conda -- look at the warning at the top about linking against 6.3 but importing 7.0. I'd expect segfaults or assertion failures from that.

[larsoner]

@kingjr do you get that error just when exiting (i.e., during garbage collection) or during use / commands / viewing?

[kingjr]

I seem to get an error anytime I close the viz window. It seems to run fine as long as a keep updating the same scene.

[kingjr]

(I'm putting this here for the record, perhaps it'll calm down the frustration of those who face similar errors)

[ktavabi]

So in troubleshooting my existential crisis I have:

1. Un-set the matploltlib backend in the rc file
2. Set ETS_TOOLKIT=wx
3. Used %gui wx in ipython console
   using @kingjr 's minimal I now get:

---

WARNING: Imported VTK version (6.3) does not match the one used
to build the TVTK classes (5.1). This may cause problems.
Please rebuild TVTK.

---

Backend agg is non-interactive backend. Turning interactive mode off.
Traceback (most recent call last):
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 7, in
stc.plot(subject='fsaverage', subjects_dir=subjects_dir)
File "/home/ktavabi/Projects/mne-python/mne/source_estimate.py", line 1344, in plot
time_unit=time_unit)
File "/home/ktavabi/Projects/mne-python/mne/viz/_3d.py", line 1134, in plot_source_estimates
views=views)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/surfer/viz.py", line 395, in __init__
self._scene_size, offscreen)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/surfer/viz.py", line 212, in _make_viewer
figure = mlab.figure(title, size=(w, h))
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/mayavi/tools/figure.py", line 85, in figure
engine.new_scene(name=name, size=size)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/apptools/scripting/recordable.py", line 45, in _wrapper
result = func(args, kw)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/mayavi/core/engine.py", line 452, in new_scene
viewer = self.scene_factory(factory_kwargs)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/mayavi/core/ui/mayavi_scene.py", line 89, in viewer_factory
viewer.open()
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/pyface/i_window.py", line 166, in open
self._create()
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/pyface/ui/wx/application_window.py", line 151, in _create
contents = self._create_contents(body)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/tools/ivtk.py", line 392, in _create_contents
self.scene = self._scene_factory(parent)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/mayavi/core/ui/mayavi_scene.py", line 68, in mayavi_scene_factory
s = MayaviScene(parent, stereo=p['stereo'])
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/decorated_scene.py", line 69, in __init__
super(DecoratedScene, self).__init__(parent, *traits)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/scene.py", line 302, in __init__
super(Scene, self).__init__(parent, *traits)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/pyface/tvtk_scene.py", line 204, in __init__
self.control = self._create_control(parent)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/decorated_scene.py", line 103, in _create_control
self._panel)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/scene.py", line 612, in _create_control
renwin = self._renwin = tvtk.to_tvtk(window.GetRenderWindow())
File "tvtk_classes/tvtk_helper.py", line 51, in wrap_vtk
return tvtk_class(obj)
File "tvtk_classes/x_open_gl_render_window.py", line 43, in __init__
tvtk_base.TVTKBase.__init__(self, vtk.vtkXOpenGLRenderWindow, obj, update, *traits)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/tvtk_base.py", line 334, in __init__
self.update_traits()
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/tvtk_base.py", line 482, in update_traits
setattr(self, name, val)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/traits/trait_numeric.py", line 178, in validate
self.error( object, name, value )
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/traits/trait_handlers.py", line 172, in error
value )
TraitError: The 'position' trait of a XOpenGLRenderWindow instance must be an array of int64 values with shape (2,), but a value of '_00000000055336b8_p_void' was specified.
Traceback (most recent call last):
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/scene.py", line 529, in OnPaint
if len(self.renderer.lights) == 0:
AttributeError: 'NoneType' object has no attribute 'lights'
Traceback (most recent call last):
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/tvtk/pyface/ui/wx/scene.py", line 529, in OnPaint
if len(self.renderer.lights) == 0:
AttributeError: 'NoneType' object has no attribute 'lights'

My system config is:

Platform: Linux-4.4.0-78-generic-x86_64-with-debian-stretch-sid
Python: 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Executable: /home/ktavabi/anaconda2/bin/python
CPU: x86_64: 24 cores
Memory: 94.4 GB
mne: 0.15.dev0
numpy: 1.11.3 {lapack=mkl_intel_lp64, blas=mkl_intel_lp64}
scipy: 0.19.0
matplotlib: 1.5.1
sklearn: 0.18.1
nibabel: 2.0.2
mayavi: 4.4.4
pycuda: 2016.1.2
skcuda: 0.5.2
pandas: 0.19.2
pyface: 5.1.0

[larsoner]

That warning about classes and version building mismatches is indicative of
a problem. Your VTK isn't right for your Mayavi / TVTK.

[ktavabi]

Yeah I am not crazy about it either, but I haven't figured out how to remedy it not to mention how it got botched in the first place.

[larsoner]

What does this do:

conda remove vtk mayavi
conda install vtk mayavi numpy scipy matplotlib sklearn pandas

(Only include packages you actually use on the last line)

Extra packages being named seems to help conda not botch the installation.

[ktavabi]

Thanks @Eric89GXL I went with

conda remove vtk mayavi
conda install vtk mayavi numpy scipy matplotlib scikit-learn pandas seaborn ipython jupyter

Things are working again! For now...
I should note current 3D viz environment is not interactive and a little buggy. E.g., I am unable to regenerate new 3D viz after closing initial figure window. I get the following TB...

./src/common/timercmn.cpp(62): assert "Assert failure" failed in Init(): No timer implementation for this platform
./src/common/timercmn.cpp(73): assert "m_impl" failed in SetOwner(): uninitialized timer
Traceback (most recent call last):
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
fig = stc.plot('td411', time_viewer=True, initial_time=.3, hemi='both')
File "/home/ktavabi/Projects/mne-python/mne/source_estimate.py", line 1344, in plot
time_unit=time_unit)
File "/home/ktavabi/Projects/mne-python/mne/viz/_3d.py", line 1134, in plot_source_estimates
views=views)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/surfer/viz.py", line 395, in __init__
self._scene_size, offscreen)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/surfer/viz.py", line 212, in _make_viewer
figure = mlab.figure(title, size=(w, h))
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/mayavi/tools/figure.py", line 85, in figure
engine.new_scene(name=name, size=size)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/apptools/scripting/recordable.py", line 45, in _wrapper
result = func(args, kw)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/mayavi/core/engine.py", line 452, in new_scene
viewer = self.scene_factory(factory_kwargs)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/mayavi/core/ui/mayavi_scene.py", line 89, in viewer_factory
viewer.open()
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/pyface/i_window.py", line 166, in open
self._create()
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/pyface/ui/wx/application_window.py", line 147, in _create
super(ApplicationWindow, self)._create()
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/pyface/i_window.py", line 254, in _create
super(MWindow, self)._create()
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/pyface/i_widget.py", line 86, in _create
self.control = self._create_control(self.parent)
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/pyface/ui/wx/application_window.py", line 175, in _create_control
pos=self.position
File "/home/ktavabi/anaconda2/lib/python2.7/site-packages/wx-3.0-gtk2/wx/_windows.py", line 580, in __init__
_windows_.Frame_swiginit(self,_windows_.new_Frame(args, **kwargs))
PyNoAppError: The wx.App object must be created first!

[mwaskom]

I ended up at the issue while googling something else, but while I'm here, the answer to @kingjr's main problem is to wait to call %gui qt after mayavi is imported. That will avoid conflict for how the Qt API gets set. There's more info here: https://github.com/ipython/ipython/issues/10384

There's still the Qt4/Qt5 issue, but as it sounds like wx is brittle, a conda environment with a downgraded Qt4 may be the easiest solution.

[larsoner]

a downgraded Qt4 may be the easiest solution.

Unfortunately on Anaconda this holds back matplotlib to 1.5, and also NumPy/SciPy last I checked.

It's probably worth trying the integrated Python 3.6 environment file, as it provides latest NumPy/SciPy/matplotlib/Mayavi + PyQt5 + Python 3.6. Our CIs have been running it for a month and so on Linux, OSX, and Windows and so far it has worked pretty well. The downside is that we currently pin to specific commit hashes for Mayavi/traits/pyface, which isn't the nicest thing to do. That requirement should in theory go away once they cut new releases, though, so hopefully it's temporary.

[mwaskom]

Unfortunately on Anaconda this holds back matplotlib to 1.5, and also NumPy/SciPy last I checked.

With base anaconda yes but not with conda-forge, I think. My recipe here gets me Qt4 and matplotlib 2.0.