Wlroots: Displaylink support

[Algram]

Hello,

I wanted to ask if displaylink is something that this project wants to support in the future. I've composed a list of resources to make it easier to work on this issue:

Why support displaylink?

We all know that the protocol itself is probably going away at some point in the future. Right now though, there are a lot of companies that have docking stations that ONLY work with displaylink. Supporting the protocol will allow wlroots users access to those adapters.

Current problems (afaik)

• According to sway this should already be supported by wlroots (https://github.com/swaywm/sway/issues/4176#issuecomment-495330845). I think this is not the case, please correct me if I am wrong
• One user reported that they got it working in sway for a short periord with the displaylink package in the AUR (https://github.com/DisplayLink/evdi/issues/58#issuecomment-520019134, https://aur.archlinux.org/packages/displaylink/). It stopped working after an update though and since then it is dead.

How to

There is the evdi (https://github.com/DisplayLink/evdi) kernel module that already works ok. Every wayland compositor needs to implement it though. Mutter has done that already (https://github.com/DisplayLink/evdi/issues/58#issuecomment-495247935), so maybe looking at their implementation helps moving this forward in wlroots.

[ascent12]

Yeah, this is something I eventually want to support. I even bought a displaylink device a while ago for this purpose.
It has been a low priority while some rather drastic backend changes are being made, which should eventually lead to us being able to support devices without their own rendering capabilities, like displaylink ones.

Another comment I made a while ago related to this: https://github.com/swaywm/wlroots/issues/1278#issuecomment-492514451

[Algram]

Alright thanks for your answer! I think I already know the answer, but let me ask anyway: Is there such a thing as a quick and dirty workaround to get at least display output working?

Otherwise I will unfortunately have to switch back to bspwm since my workplace only provides displaylink stations.

Thanks a ton!

[dstoe]

I came here since I tried to get started with sway and my display-link port-replicator. Update from my today's debugging session:

• Sway aborts with Bus Error on initializing the monitor attached via a port-replicator with display-link. Logging output of sway --verbose --debug:

[ ... ]
2020-05-29 09:50:41 - [backend/drm/drm.c:695] Modesetting 'DVI-I-1' with '1920x1080@60000 mHz'
2020-05-29 09:50:41 - [backend/drm/drm.c:591] Initializing renderer on connector 'DVI-I-1'
Bus Error

• Nothing special in the kernel log
• Backtrace hints to this issue which is unrelated to wlroots. Last call in wlroots codebase is eglSwapBuffers in egl.c#L445. Line numbers in the stacktrace may be broken since I added some log statements for debugging.

#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55c6269436f8) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55c6269436a8, cond=0x55c6269436d0) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x55c6269436d0, mutex=0x55c6269436a8) at pthread_cond_wait.c:638
#3  0x00007fb2e9b50a9b in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#4  0x00007fb2e9b5abed in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#5  0x00007fb2e9b5b747 in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#6  0x00007fb2e9558e0b in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#7  0x00007fb2e954ce59 in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#8  0x00007fb2ead01d22 in  () at /lib/x86_64-linux-gnu/libEGL_mesa.so.0
#9  0x00007fb2eacfaa9a in  () at /lib/x86_64-linux-gnu/libEGL_mesa.so.0
#10 0x00007fb2eacf3023 in  () at /lib/x86_64-linux-gnu/libEGL_mesa.so.0
#11 0x00007fb2eeaf48e7 in wlr_egl_swap_buffers (egl=0x55c6269028e8, surface=0x55c627479240, damage=0x0) at ../subprojects/wlroots/render/egl.c:446
#12 0x00007fb2eeb02f3f in drm_fb_acquire (fb=0x55c6269051d8, drm=0x55c6269027f0, mgpu=0x55c6269051b8) at ../subprojects/wlroots/backend/drm/renderer.c:422
#13 0x00007fb2eeafa4b7 in set_plane_props (atom=0x7ffd2874c010, drm=0x55c6269027f0, plane=0x55c626905190, crtc_id=32, x=0, y=0)
    at ../subprojects/wlroots/backend/drm/atomic.c:141
#14 0x00007fb2eeafaa0a in atomic_crtc_commit (drm=0x55c6269027f0, conn=0x55c6273f0510, flags=1025) at ../subprojects/wlroots/backend/drm/atomic.c:232
#15 0x00007fb2eeafcb4f in drm_crtc_commit (conn=0x55c6273f0510, flags=1) at ../subprojects/wlroots/backend/drm/drm.c:340
#16 0x00007fb2eeafcd02 in drm_crtc_page_flip (conn=0x55c6273f0510) at ../subprojects/wlroots/backend/drm/drm.c:365
#17 0x00007fb2eeafda5b in drm_connector_pageflip_renderer (conn=0x55c6273f0510) at ../subprojects/wlroots/backend/drm/drm.c:690
#18 0x00007fb2eeafdc6d in drm_connector_init_renderer (conn=0x55c6273f0510, mode=0x55c6273fcf60) at ../subprojects/wlroots/backend/drm/drm.c:734
#19 0x00007fb2eeafe076 in drm_connector_set_mode (conn=0x55c6273f0510, wlr_mode=0x55c6273fcf60) at ../subprojects/wlroots/backend/drm/drm.c:830
#20 0x00007fb2eeafd63d in drm_connector_commit (output=0x55c6273f0510) at ../subprojects/wlroots/backend/drm/drm.c:581
#21 0x00007fb2eeb4715d in wlr_output_commit (output=0x55c6273f0510) at ../subprojects/wlroots/types/wlr_output.c:585
#22 0x000055c625b91a42 in apply_output_config (oc=0x0, output=0x55c6273e22c0) at ../sway/config/output.c:419
#23 0x000055c625b752cb in handle_new_output (listener=0x55c625be30c0 <server+96>, data=0x55c6273f0510) at ../sway/desktop/output.c:951
#24 0x00007fb2eeb58660 in wlr_signal_emit_safe (signal=0x55c6260d22f8, data=0x55c6273f0510) at ../subprojects/wlroots/util/signal.c:29
#25 0x00007fb2eeb0a118 in new_output_reemit (listener=0x55c627090088, data=0x55c6273f0510) at ../subprojects/wlroots/backend/multi/backend.c:146
#26 0x00007fb2eeb58660 in wlr_signal_emit_safe (signal=0x55c626902818, data=0x55c6273f0510) at ../subprojects/wlroots/util/signal.c:29
#27 0x00007fb2eeb008ef in scan_drm_connectors (drm=0x55c6269027f0) at ../subprojects/wlroots/backend/drm/drm.c:1477
#28 0x00007fb2eeafad00 in backend_start (backend=0x55c6269027f0) at ../subprojects/wlroots/backend/drm/backend.c:26
#29 0x00007fb2eeaf9360 in wlr_backend_start (backend=0x55c6269027f0) at ../subprojects/wlroots/backend/backend.c:36
#30 0x00007fb2eeb09d19 in multi_backend_start (wlr_backend=0x55c6260d22d0) at ../subprojects/wlroots/backend/multi/backend.c:31
#31 0x00007fb2eeaf9360 in wlr_backend_start (backend=0x55c6260d22d0) at ../subprojects/wlroots/backend/backend.c:36
#32 0x000055c625b6fa7e in server_start (server=0x55c625be3060 <server>) at ../sway/server.c:218
#33 0x000055c625b6f036 in main (argc=3, argv=0x7ffd2874c7a8) at ../sway/main.c:394

• sway starts with internal and external monitor enabled if LP_NUM_THREADS=1 as suggested in the issue mentioned above: llvmpipe crashes using kms_swrast_dri.so.
• My system: fresh install of Ubuntu 20.04. Kernel 5.4.0-33-generic

[pati-ni]

I may have a Displaylink device to spare. Is this still under development? Will this be of any use?

[emersion]

It depends which generation of displaylink this is. The more recent generations have a proprietary driver.

[pati-ni]

It is an HP USB-C Universal Dock. Currently, it only works after I spin up gnome 3 with evdi 1.7 module and the proprietary Displaylink v5.3.1.34-4.

Not sure about the protocol version though. I found no way to flush any firmware to this thing which is odd.

[emersion]

Ah, so probably it's one of the newer ones. I'm not really interested in helping with that, but feel free to send pull requests.

[progandy]

There are some options to use/debug the evdi driver without displaylink. Most of the non-displaylink use cases can be covered with compositor support like create_output in sway and wf-recorder / wayvnc, though.

https://github.com/xytovl/evdi-hello
https://github.com/dgarfias/screx

[ascent12]

Even if evdi is "open source", it effectively only exists to push a proprietary driver. Any kind of special support for it should not be in wlroots.

[dstoe]

Spent some time digging into this. I'm not sure to how far this is an evdi issue. The mentioned issue "evdi_dri.so not present" is totally fine, since evdi is an output-only device with no rendering capabilities.

My observations

• First the good story: sway starts perfectly fine if the copy from the primary to the secondary drm device was disabled (writing this message from sway using the displaylink station :heart_eyes: ). Instead, simply re-interpret the buffer from the rendering-device as if it was usable on the output device, too. Hope you forgive this dirty unsafe hacking for testing :see_no_evil: This works with my i915 rendering node which also uses the CPU-RAM. I hacked this into wlroots/backend/drm/renderer.c::drm_plane_lock_surface:

    // [...]
    struct wlr_buffer *local_buf;
    // Change the following check to always use the `else`-path even if a parent-device is used.
    if (drm->parent && 0) {
        // Perform a copy across GPUs
        local_buf = drm_surface_blit(&plane->mgpu_surf, buf);
        if (!local_buf) {
            wlr_log(WLR_ERROR, "Failed to blit buffer across GPUs");
            return false;
        }
    } else {
        local_buf = wlr_buffer_lock(buf);
    }

• The copy-path also works if the LP_NUM_THREADS was set to 1 forcing the software-rasterizer from mesa to work single-threaded. Then, however, the rendering is unusable slow.
• With a multi-threaded software-rasterizer, there seems to be a race-condition which makes the llvmpipe component of mesa crash with a bus-error (see referenced issue in my comment from May 29th).

Open Questions

• How to programmatically check that the output and rendering device operate on the same memory using the same address-space? Any advice for a clean implementation for that case from someone more experienced?
• Why does the drm_surface_blit fail? Isn't it using dma-buf which should work anyway at a slightly higher cost of performing a copy?

[emersion]

How to programmatically check that the output and rendering device operate on the same memory using the same address-space? Any advice for a clean implementation for that case from someone more experienced?

It's a little tricky. Maybe the kernel solution described here can help, but maybe not: https://lists.freedesktop.org/archives/dri-devel/2021-April/303692.html

As of today, all we're left with are heuristics.

We can discover whether a DRM device has rendering capabilities by checking if it has a render node. If it doesn't have a render node, there's a trap: maybe a completely different DRM device can be used to render to buffers allocated on the first device (split render/display SoC).

We can try directly displaying a buffer coming from a different device with KMS. We need to perform an atomic test-only commit to make sure that works. We already support this for client buffers (see drm_connector_test). We could support it for buffers coming from the primary device.

Maybe this will work, but maybe this will be sub-optimal. We can also try to do a CPU copy of the buffer. We don't quite yet have the architecture for this, I suspect we'll need something like DRM dumb buffers. Ref https://github.com/swaywm/wlroots/pull/2700.

Also ref https://github.com/swaywm/wlroots/issues/1347, which is about improving the whole multi-GPU copy situation.

Why does the drm_surface_blit fail? Isn't it using dma-buf which should work anyway at a slightly higher cost of performing a copy?

Yeah, it _should_ work. But from what you wrote it seems like this is just a llvmpipe bug when multi-threading is enabled? And it seems like it works fine when disabling multi-threading?