Taichi: [gui] [possible new feature] Run GUI without X11 requirement

[ehannigan]

Is it possible to save each frame of the gui without a pop-up window? I am trying to run a batch of simulations quickly on a server (and inside a docker image). I keep getting the error:

"RuntimeError: [x11.cpp:create_window@139] Taichi fails to create a window. This is probably due to the lack of an X11 GUI environment. If you are using ssh, try ssh -XY."

I saw this was a recent clarification from issue #1575, but there is no discussion of turning off the window in general.

If this is not already an option, I would be happy to implement it. Possible solutions:

• I know this is possible in matplotlib; you can change the gui backend (matplotlib.use('Agg')). Maybe something similar can be done here.

• Another possible solution is to save the states over the entire simulation and render it later using a separate script. Would this be a better/easier solution?

• I could also just use matplotlib instead. I am currently working with the difftaichi example mass_spring.py, and that would not be difficult to convert to matplotlib.

[archibate]

That's a good idea! We should have multiple GUI backends, X11, Win32, Cocoa, GLFW, Agg... instead of binding a backend to a specific system. Some refactor is to be done for this multi-backend infrastructure to be done. Self-assigning this now :+1:

[yuanming-hu]

That's a good idea! We should have multiple GUI backends, X11, Win32, Cocoa, GLFW, Agg... instead of binding a backend to a specific system. Some refactor is to be done for this multi-backend infrastructure to be done. Self-assigning this now

Thanks, but probably none of these GUI backends will work without X11.

The real solution could be to simply introduce an TI_GUI_EXPORT_FOLDER environment variable and save the GUI frames to that folder instead of popping up a window. That's likely the only thing we could do without X11.

[thinking-tower]

Hey @yuanming-hu @archibate, can I add a new boolean parameter (that says if we should display the GUI or not) to
https://github.com/taichi-dev/taichi/blob/07f407d2723cd613acf042c59239ca82c5dbda82/python/taichi/misc/gui.py#L358-L363
that we can use to add an if statement around redraw() here?
https://github.com/taichi-dev/taichi/blob/07f407d2723cd613acf042c59239ca82c5dbda82/taichi/gui/gui.h#L806-L815

The user can take care of naming the screenshots themselves. A simple example could be just

# Some computation done.
init()
gui = ti.GUI('Something cool')
i = 0
while gui.running:
    # Do some more stuff here.
    gui.show(file="{}.jpg".format(i), show=False)
    i += 1

We should probably rename show or introduce a new method that lets you save without showing the GUI.

[yuanming-hu]

Hi @thinking-tower, that sounds like a good idea! How about naming that function GUI.screenshot(fn : str)?

ti.GUI may still crash on initialization when X11 is not available :-) We need some treatment there. Even if we cannot initialize a window, painting functions and screenshot should still work.

Thanks for helping with this!

[archibate]

We need some treatment there

My opinion is to specify show=False on initialization: ti.GUI('Something cool', show=True).
And GUI::create_window() should not be called in GUI::GUI(), but from Python:

self.show = show
gui.core = ti.core.GUI(...)  # doesn't create X11 window
if self.show:
  gui.core.create_window()  # only create when `show` is on

And all X11 operations should be guarded with if self.show.