X11docker: Tins: Which tools do you recommend?

[marbetschar]

Hi @mviereck 👋️,

I'm currently tinkering with LXD to achieve something similar to what x11docker does (my approach is named "Tins"): The ultimate goal is to enable users to quickly setup a containerized desktop environment for development (the whole container is writeable, so its more of a lightweight virtual machine).

Although I was able to let the Display Manager rendering the login screen using Xephyr as you can see here, I'm puzzled how to go further. I'm overwhelmed by all the different tools involved, which problems they solve and whether they are suitable for my use case or not.

As you have a lot of experience with these tools, I'd highly appreciate your opinion: Which ones would you choose today to achieve which part of the following goals?

Love to hear your thoughts!

Goals

_Host_

• Linux and runs a X11 desktop
• Based on Ubuntu 18.04 LTS or 20.04 LTS (Tins should run on both)

_Container_

• Runs on the same computer as the host system
• Can be any sort of distro / desktop environment combination
• The containerized desktop environment is GPU accelerated (ideally 2D and 3D)

Wish List

_Container_

• Support for Audio
• Support for Copy & Paste
• Support for external Hardware (USB Sticks, Printer, ...)

[mviereck]

I'll look closer a bit later, here just some quick tips:

• Why do you tinker with display managers? You would not need them
• You try to run the Gnome3 desktop. Gnome3 is quite unstable in containers and I discourage it. All other desktops work better.
• Don't use snap in container. Container in container is no good idea and needs too much privileges.
• For GPU support look at the wiki: https://github.com/mviereck/x11docker/wiki/Hardware-acceleration. GPU acceleration is limited to X servers Xorg and Xwayland (and xpra if using Xwayland instead of Xvfb/Xdummy). Wayland works, too.
• For audio look at the wiki: https://github.com/mviereck/x11docker/wiki/Container-sound:-ALSA-or-Pulseaudio
• Clipboard sharing is a pita. xpra and nxagent support it well. For other X servers I run a script in background that checks the clipboard of both X servers repeatedly. Search for function setup_clipboard() in x11docker.
• Support for external Hardware (USB Sticks) is difficult. Compare ticket #245.
• For printer support look at https://github.com/mviereck/x11docker/wiki/CUPS-printer-in-container

[marbetschar]

@mviereck thank you so much for taking time to give some tips!

I thought a display manager is mandatory to start a desktop environment? To be honest my understanding of how the GUI startup process works and how the responsibilities of the different jobs are is quite limited as of now.

Good to know GNOME3 is unstable in containers, will keep this in mind. Although there is no specific reason why I used it for my tests - it just seems to be the easiest starting point as its kind of the default desktop for Ubuntu ;)

[mviereck]

I thought a display manager is mandatory to start a desktop environment?

No, it is just a graphical login screen. You can as well switch to a tty, log in and run e.g. startxfce4. The classic way is to write a file ~/.xinitrc and start X from console with startx.
(Side note: there are others like startlxde that need a running X beforehand while startxfce4 can do this on its own. Display managers also provide the X server, but you would do this yourself.)
For containers it is enough to provide an X server, share the socket, provide DISPLAY (and, if used, XAUTHORITY) and run a command that starts a desktop or window manager.

I want to point at a further setup possibility:
Running a Wayland compositor instead of X is possible. I prefer weston; it runs nested in X as well as on its own from tty. GPU acceleration with Wayland works well.
You can run e.g. Weston on host and Xwayland in container, and on top of Xwayland your desired desktop. Compare https://github.com/mviereck/dockerfile-x11docker-xwayland.

Edit: Instead of running Xwayland in container, you can run it on host as well.

TL;DR: I recommend to run weston+Xwayland on host to provide an X server with GPU acceleration.

[marbetschar]

@mviereck thank soooo much for all this insight! I will definetely give weston + Xwayland a try and we'll see where I end up with.

[eine]

Support for external Hardware (USB Sticks) is difficult. Compare ticket #245.

As an alternative, USBIP is a kernel module. I have successfully used it on Docker Desktop for Windows, in order to share arbitrary USB devices. Unfortunately, I found no open source client for Windows. See https://github.com/ghdl/docker/tree/master/usbip#readme

[mviereck]

Support for external Hardware (USB Sticks) is difficult. Compare ticket #245.

As an alternative, USBIP is a kernel module.

Very interesting! Let's discuss this in #253 .

[toby63]

@mviereck

GPU acceleration with Wayland works well.

Could you share some of your experience?
Like how good is the performance (for example compared to native)?

[mviereck]

Like how good is the performance (for example compared to native)?

It seems to be the same speed as native. Just try it out:

weston &
WAYLAND_DISPLAY=wayland-0 Xwayland :5 &
DISPLAY=:5 glxgears

Edit: You can use x11docker to set up an X server only, without using Docker at all.
That can help to try out setups with LXD, too.

read Xenv < <(x11docker --weston-xwayland --xonly --showenv)
env $Xenv glxgears

Running x11docker with option --debug shows you the generated weston and Xwayland commands.

[toby63]

What can you tell regarding the (transitional) downsides of wayland?
I am a bit confused, because I read that some things might not work:

• Some desktop environments like Xfce have no support yet
• OpenGL is only partly working (?) (mobile variant (OpenGL ES) implemented in weston; otherwise special compiled programs?)
• Wine is only working in a special edition with vulkan and vulkan wrapper for directx9-11: https://github.com/varmd/wine-wayland

Then again I read about XWayland, which gives support for Applications that only run on X.
Maybe you can clarify the situation, what is possible and what not :slightly_smiling_face:.
(At least for the parts you tried yourself)

[marbetschar]

@toby63 I probably start playing with Xwayland & weston later today - with containerized Xfce.

While I don’t have any experience with this yet, from what I understood Xwayland adds the compatibility layer here - so ideally it’s not necessary for Xfce to officially support Wayland cause it does not need to be aware of any difference.

That is, of course, the working theory as of now. We‘ll see if that’s really the case in practice 🤪

[mviereck]

What can you tell regarding the (transitional) downsides of wayland?

While I don’t have any experience with this yet, from what I understood Xwayland adds the compatibility layer here - so ideally it’s not necessary for Xfce to officially support Wayland cause it does not need to be aware of any difference.

That's entirely correct. weston as Wayland compositor only serves to provide Xwayland here. No need to worry about Wayland issues to run a containerized Xfce.

I should clarify one point:
Wayland compositors use a rootless Xwayland while my setup uses Xwayland with a root window.
A rootless setup allows to have X11 windows inside a Wayland environment. There is no difference to the eye between X or Wayland applications.
My example (and x11docker option --weston-xwayland) run Xwayland with a root window (or call it desktop window). The root window allows to have an X11 background, a menu, a panel, desktop icons etc. (It has nothing to do with user root, just the same name).
If you press the super/windows key and the left mouse button, you can move the Xwayland root window around like an arbitrary Wayland client window.

OpenGL is only partly working (?) (mobile variant (OpenGL ES) implemented in weston; otherwise special compiled programs?)
Wine is only working in a special edition with vulkan and vulkan wrapper for directx9-11: https://github.com/varmd/wine-wayland

I am only aware of issues with the closed source NVIDIA driver. nouveau works.

Maybe you can clarify the situation, what is possible and what not slightly_smiling_face.
(At least for the parts you tried yourself)

In practice I do not use Wayland but Xfce. So I can tell how to technically provide Wayland to a container, but not much about downsides in daily use.

Some desktop environments like Xfce have no support yet

There is some interesting discussion of the LXQt developers: https://github.com/lxqt/lxqt/issues/10.
Applications written in GTK3 or QT5 can run on X and on Wayland. Xfce is currently on its way to GTK3. Maybe some day there will be an Xfce Wayland compositor.

[marbetschar]

@mviereck was playing with weston all day long, but was not able to sucessfully run it via Tins. When I execute the weston command in Terminal it works, but not when executed via Vala. Do you have any idea what might be wrong?

This is the weston.ini created:

[core]
shell=desktop-shell.so
backend=x11-backend.so
xwayland=false
idle-time=0

[xwayland]
path=/usr/bin/Xwayland

[shell]
panel-location=none
panel-position=none
locking=false
background-color=0xff002244
animation=fade
startup-animation=fade

[keyboard]

[output]
name=X1
mode=1024x768
scale=1
transform=normal

And the output produced by Tins:

** Starting Compositor [WAYLAND_DISPLAY=wayland-1]:
    /usr/bin/weston
    --socket=wayland-1
    --config=/home/marbetschar/.cache/com.github.marbetschar.tins/Xfce/weston.ini
** Starting X server [DISPLAY=:1]:
    /usr/bin/Xwayland
    :1
    -retro
    +extension
    RANDR
    +extension
    RENDER
    +extension
    GLX
    +extension
    XVideo
    +extension
    DOUBLE-BUFFER
    +extension
    SECURITY
    +extension
    DAMAGE
    +extension
    X-Resource
    -extension
    XINERAMA
    -xinerama
    -extension
    MIT-SHM
    +extension
    Composite
    -extension
    XTEST
    -tst
    -dpms
    -s
    off
    -auth
    /home/marbetschar/.cache/com.github.marbetschar.tins/XfceXauthority.server
    -nolisten
    tcp
    -dpi
    96
Date: 2020-05-26 CEST
[21:31:37.345] weston 3.0.0
               http://wayland.freedesktop.org
               Bug reports to: https://bugs.freedesktop.org/enter_bug.cgi?product=Wayland&component=weston&version=3.0.0
               Build: unknown (not built from git or tarball)
[21:31:37.345] Command line: /usr/bin/weston --socket=wayland-1 --config=/home/marbetschar/.cache/com.github.marbetschar.tins/Xfce/weston.ini
[21:31:37.345] OS: Linux, 4.15.0-1081-oem, #91-Ubuntu SMP Fri May 1 02:03:58 UTC 2020, x86_64
[21:31:37.345] Using config file '/home/marbetschar/.cache/com.github.marbetschar.tins/Xfce/weston.ini'
[21:31:37.345] Output repaint window is 7 ms maximum.
[21:31:37.345] Loading module '/usr/lib/x86_64-linux-gnu/libweston-3/x11-backend.so'
[21:31:37.347] fatal: failed to create compositor backend
could not connect to wayland server
(EE) 
Fatal server error:
(EE) Couldn't add screen
(EE)

[toby63]

Interesting, so everything is the same as you tried in terminal?
Can show us the terminal input and output as well?

It might be to obvious, but just in case, I see two potencial problems:

1. Terminal creates it on host and tins inside container(?)
2. some problem with the user right: what user do run in terminal and what user is tins run with?

[marbetschar]

@toby63 I assume its related to environment variables, since everything is executed on host and under the same user.

I'm currently passing the following environment variables programmatically:

DISPLAY=:1
XAUTHORITY=/home/marbetschar/.Xauthority
XSOCKET=/tmp/.X11-unix/X1
WAYLAND_DISPLAY=wayland-1
XDG_RUNTIME_DIR=/run/user/1001

Obviously these are different when weston is started in Terminal directly. But I'm not sure what is really needed by weston and Xwayland to work properly: some of them need to be different since we are actually creating a new X server, some of them maybe need to be the same because the new X server is nested - and others might be completely missing in the list above.

The code which creates the environment and runs weston is located here:

https://github.com/marbetschar/tins/blob/5dbd4b96f3ff703e814cd175434a5824456de8ca/src/Widgets/ContainerListBox.vala#L179-L287

UPDATE: I guess the first issue to resolve is the fatal: failed to create compositor backend in the above posted output. But I have no clue why weston fails creating the compositor backend.

[toby63]

Hmmm...I just read that weston should work with nearly no config, so you might try to run it with less configuration options :thinking:.

Update:
I see I missunderstood, should've looked at your program code :sweat_smile:.

---

A little off-topic:
I found an interesting project: https://github.com/ackalker/Xweston
It enables one to use other display managers etc.

[mviereck]

For all I assume you try to run weston nested in X.

It seems that you are confusing the DISPLAY numbers:

** Starting X server [DISPLAY=:1]:
    /usr/bin/Xwayland
    :1

I'm currently passing the following environment variables programmatically:

DISPLAY=:1



md5-37b25a67ef900de9defbdf12bdfcf249



Loading module '/usr/lib/x86_64-linux-gnu/libweston-3/x11-backend.so'
[21:31:37.347] fatal: failed to create compositor backend
could not connect to wayland server

This is a bit confusing, I'd assume it should say could not connect to X server because you run nested in X and use x11-backend.so.
Just in case, maybe it helps to specify --backend=x11-backend.so in the weston command instead of the config file.

Some pitfalls:

• Make sure socket wayland-1 is not already in use. Check in $XDG_RUNTIME_DIR.
• Make sure socket X1 for Xwayland :1 is not already in use. Check in /tmp/.X11-unix.

[mviereck]

But I'm not sure what is really needed by weston and Xwayland to work properly: some of them need to be different since we are actually creating a new X server, some of them maybe need to be the same because the new X server is nested

Weston has three most important ways to run:

• nested in X
• nested in Wayland
• directly from tty

Currently you want to run Weston nested in X.
That needs:

• backend x11-backend.so (to be specified in weston command or in weston.ini. Do not rely on auto-detection)
• DISPLAY from the parent X server
• XAUTHORITY from the parent X server (or access based on xhost)
• XDG_RUNTIME_DIR to store the new Wayland socket (normally set up by the system at login)

It generates:

• A nested Wayland window in X.
• A Wayland socket in XDG_RUNTIME_DIR. Its name can be specified on cli with --socket=SOCKETNAME

Xwayland needs:

• WAYLAND_DISPLAY containing the name of the Wayland socket generated by weston.
• XDG_RUNTIME_DIR
• A free display number (check /tmp/.X11-unix)
• To have an easy start, drop many Xwayland options and just use e.g. env WAYLAND_DISPLAY=wayland-1 Xwayland :5

A container needs only DISPLAY and its socket in /tmp/.X11-unix to access Xwayland. If the setup works overall, also set up an X cookie and provide XAUTHORITY for Xwayland access to have restricted access.

[marbetschar]

Thanks @mviereck, the environment variables did the trick! Now weston and Xwayland start sucessfully on host, thanks to this code.

Next I just need to figure out how to correctly map Xwayland into the container and then how to correctly start the desktop environment - hopefully that's not too hard anymore.

Thank you very much for all your help guys - highly appreciate!

[mviereck]

If I understand correctly, you are mapping /tmp/.X11-unix/X1 to /tmp/.X11-unix/X0 in container. Don't do that; access control is based on the display number. Map it to the same path as on host and set DISPLAY accordingly.
Also try e.g. DISPLAY=:1 xterm on host to check if access works at all.

Edit:
If you don't specify -auth COOKIE for Xwayland, it does some xhost access setup on its own. Check the output of DISPLAY=:1 xhost.
If the output only allows access for your current host user, but you have another user in container, the access will fail. To allow e.g. root in container to allow access to Xwayland, run DISPLAY=:1 xhost +SI:localuser:root on host. (Overall I prefer cookie based authentication for better security instead of xhost.)

[mviereck]

Closing here because we rather discuss in https://github.com/marbetschar/tins/issues/16#issuecomment-636472258.
Feel free to open new tickets on anything.