Obs-studio: Cannot capture entire screen in Wayland

[probonopd]

Expected Behavior

Capture entire screen in Linux.
Works well on Xorg based distributions.

Current Behavior

Cannot capture entire screen in Linux distributions such as Fedora that use Wayland as their display server. Just getting a black screen with a cursor in it.

Works well on Xorg based distributions.

Steps to Reproduce

1. Run Fedora
2. Try to capture the entire screen in OBS Studio

Additional information

Looking for a way that "just works" without having to configure anything or install anything special.

Reference:
https://gitlab.freedesktop.org/wayland/wayland/issues/150

[kkartaltepe]

Wayland is unsupported at this time. You can find a variety of third party plugins that might work to varying degrees depending on your precise DE/WM.

[probonopd]

Thanks @kkartaltepe. Do you know what would need to be changed in Wayland for it to "just work" just like in Xorg?

[Fenrirthviti]

There isn't really something that can just be easily changed. Wayland provides no capture APIs, so you need to go through your specific desktop/window manager for it, which people have done to varying degrees of success. You can find more information and a few very early plugins here:

https://obsproject.com/mantis/view.php?id=719
https://obsproject.com/mantis/view.php?id=1518

Closing this for now, as Wayland is not supported at this time. Please use the Mantis issue for any further discussion.

As a final FYI to anyone coming across this issue in the future, the biggest step to getting Wayland support added to OBS would be to write up and submit an RFC.

[probonopd]

Having each desktop/window manager provide a workaround seems like a lot of unnecessary duplication of work to me. Hence I kindly ask the Wayland team to provide capture APIs, ideally ones that are compatible to Xorg, so that existing screen recording, screen sharing, screen casting applications can work without requiring major changes.

But somehow I doubt they will do anything about it, because it has already been an obvious issue for such a long time.

a few very early plugins

I am looking for a solution that

"just works" without having to configure anything or install anything special.

And I cannot imagine that I am the only one...

a variety of third party plugins that might work to varying degrees depending on your precise DE/WM

is exactly the kind of shaky user experience that makes an otherwise excellent product less excellent.

[kkartaltepe]

ideally ones that are compatible to Xorg

For reference X11 solutions are very close to the worst possible interface one could design.

is exactly the kind of shaky user experience that makes an otherwise excellent product less excellent.

Indeed dekstop linux would be an excellent product if not for the incessant infighting and incompatible implementions at every level of the graphics and windowing stack. Once they can resolve their differences im sure it will be an excellent experience for users and developers alike.

[probonopd]

To create a screencast in Fedora, one can use the key-combination Ctrl + Shift + Alt +R to start your recording. Couldn't the same technology they are using to implement that feature be used in OBS Studio?

[derrod]

That is a Gnome exclusive feature, and there is an OBS plugin to use it: https://github.com/fzwoch/obs-gnome-screencast

[probonopd]

Thanks @derrod. Can it be made so that this plugin becomes part of OBS without the need of installing or configuring anything, and can it be made so that it works on all systems with Wayland, not just those using GNOME?

[kkartaltepe]

Its a pretty good example of how (gnome) wayland has regressed to something harder to use and arguably technically inferior to the worst capture X11 capture.

But since gnome developers are implementing that for KDE it will likely be the target with the largest userbase for screen capture on wayland. And everyone (except Gnome and KDE developers apparently) will complain when it turns their fancy cpu into a space heater.

[Sunderland93]

@probonopd Flatpak-version of OBS Stuido provide new portal plugin, which works on both Wayland and X11, using xdg-desktop-portal and PipeWire

[probonopd]

Thanks but portals seem to be biased toward Flatpak and GNOME. I am looking for a generic solution that works with AppImage and any desktop environment and no need for portals, just as with Xorg.

I have tried to use Flatpak many times but it does not work for me. (If you want to know why, please see https://www.youtube.com/watch?v=0sRYL9IzkRQ)

[Sunderland93]

@probonopd

Thanks but portals seem to be biased toward Flatpak and GNOME

No, it's generic solution and adopted by GNOME, KDE and WLROOTS guys.

[kkartaltepe]

And what do you mean by adopted?

Assuming https://github.com/KDE/xdg-desktop-portal-kde is the actual KDE implementation it supports half the protocol at best. Will probably perform twice as bad (or worse depending on how pipewire is implemented) as X11 xshm (already unacceptably slow for users).

There is no reference of this "adopted" solution in the wlroots or swaywm codebases. So maybe you can clarify on what you mean by adopted. (Their ancient attempt at monitor capture was pretty good and used dma-buf, but of course wlroots doesnt work on nvidia drivers).

Finally gnome, the proposer of the protocol of course has the most complete implementation and appears to even finally be implementing dma-buf support. In the slow path they at least manage to save one useless copy compared to KDE. So depending on pipewire its possible its only as slow X11. Maybe with more work on our side we might even get close to reasonable performance for AMD users by adding special cases for different compositors and hardware.

So while the claim is that this is the one true protocol. It's currently different per major compositors and must be implemented in OBS differently for each compositor since the protocol leaves you guessing as to what is going to work on the system where users might run your code. Some compositors only implement half or less of the protocols JUST for the portion we want to use for screen casting. The whole protocol of course includes everything and the kitchen sync for compositors to worry about. Finally every implementation is AT LEAST as slow X11 without even leaving the compositor. I cant bear to inspect the rest of the layers and what might occur before OBS finally has a chance to do anything. Maybe later one might support dma-buf and we can deal with probing hardware in addition to compositors to try and find a workable solution.

This is just my personal opinion on the topic.

--- edit

Perhaps the wlroots implementation you mean https://github.com/emersion/xdg-desktop-portal-wlr implementing on the barest of bones screencopy and not the supposed standard screencast.

[LordUbuntu]

As a final FYI to anyone coming across this issue in the future, the biggest step to getting Wayland support added to OBS would be to write up and submit an RFC.

Sorry if this is a stupid question but what is an RFC?

[Fenrirthviti]

An RFC is a process designed to propose and discuss major changes, effectively.

An RFC has already been opened for this specific issue, and can be found here: https://github.com/obsproject/rfcs/pull/14

[LordUbuntu]

An RFC is a process designed to propose and discuss major changes, effectively.

Thank you for the explanation. If I understand an RFC is essentially a design proposal to map out the steps necessary to add or complete a feature?

An RFC has already been opened for this specific issue, and can be found here: obsproject/rfcs#14

That's fantastic news to hear!

Thank you for your help.

[Francewhoa]

As a temporary workaround, this challenge can potentially be resolve using either obs-xdg-portal or obs-gnome-screencast free and open source plugins. Attribution & details at https://obsproject.com/forum/threads/running-obs-in-wayland.32165/post-492672

As you know, per the other comments above, the stock OBS does not presently support Wayland. Any volunteers interested to contribute support for OBS to support Wayland? Both me and the Ubertus team would be happy to contribute testing or documentation if needed.

[kkartaltepe]

There is already a complete PR for implementing wayland as a native platform (for the gui) for obs in https://github.com/obsproject/obs-studio/pull/2484 the more people who test and confirm it works as expected the better.

And as you noted there are already third party plugins that can be used to capture wayland windows/screens.