Godot: Add Wayland support

[blackwolf12333]

As you might know there is a new protocol for a display server being developed for linux called Wayland. Currently godot does not support this as far as I know. As this is the future of the linux graphics stack I would very much like to see support for this.

This has no high priority because Wayland is far from production ready.

[reduz]

not a chance in the short or medium term, unless someone contributes it. The fact I use Ubuntu makes it impossible too.

[jerriclynsjohn]

I'm new here, but would like to know whats the effort required to develop support for Wayland into Godot. Any information would be highly be appreciated.

[punto-]

Mostly it'll be re-implementing the stuff in this file:
https://github.com/okamstudio/godot/blob/master/platform/x11/os_x11.cpp

On Thu, Dec 18, 2014 at 8:15 PM, Jerric Lyns John [email protected]
wrote:

I'm new here, but would like to know whats the effort required to develop
support for Wayland into Godot. Any information would be highly be
appreciated.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/576#issuecomment-67574161.

[jerriclynsjohn]

@punto- I would really appreciate if we could collaborate on this development, provided that you could provide me with sufficient information.

[punto-]

@jerriclynsjohn sure, I'd be happy to help.. I don't know anything about wayland, and only a bit about x11 (I believe I only did the joystick and clipboard stuff on that file), but I'm very familiar with the porting process overall. Let me know what you need. Also, we have a developer mailing list, it'll probably be easier to discuss it there: https://groups.google.com/forum/#!forum/godot-engine

[MGageMorgan]

Welp, guys and gals, one of you made the excuse that you're not affected by Wayland because you're on Ubuntu. But you are! Ya boy Shuttleworth is making the jump, Fedora made the jump 6 months ago, and the performance differences are noticeable (in a good way for Wayland). Most stock GNOME apps are Wayland-ready or are getting there. Ubuntu is moving to GNOME, and already switched to Wayland in what, 17.04?

Either way, if the same members as above are still here, this thread is relevant. I'm not in a position to handle this, but I know some probably are. You can roast me all you want, but the truth of the matter is that the editor will run smoother on Wayland than it will on Xorg.

Every other community has seen the light. Unity3d has a Linux port they're working on, and the guys developing it claim to be implementing Wayland support as well (at least last I heard).

[reduz]

There is currently no hurry. For OpenGL there is not much of a difference
and, if you want to deploy a game to a backend that you know it will work
everywhere, X11 is the way to go.

Believe me I'm the first one to experience the pain of the horrible X11 API
and would be more than happy to do away with it, but the transition until
the point we can get rid of the X11 backend will take some years.

On Sun, Jun 11, 2017 at 5:46 PM, M. Gage Morgan [e-mail]
wrote:

Welp, guys ans gals, one of you made the excuse that you're not affected
by Wayland because you're on Ubuntu. But you are! Ya boy Shuttleworth is
making the jump, Fedora made the jump 6 months ago, and the performance
differences are noticeable (in a good way for Wayland). Most stock GNOME
apps are Wayland-ready or are getting there. Ubuntu is moving to GNOME, and
already switched to Wayland in what, 17.04?

Either way, if the same members as above are still here, this thread is
relevant. I'm not in a position to handle this, but I know some probably
are. You can roast me all you want, but the truth of the matter is that the
editor will run smoother on Wayland than it will on Xorg.

Every other community has seen the light. Unity3d has a Linux port they're
working on, and the guys developing it claim to be implementing Wayland
support as well (at least last I heard).

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/576#issuecomment-307655691,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z23PhLpMIZPjj9ewuwtumWnnbngRyks5sDFIMgaJpZM4CLKzr
.

[wmww]

I don't think anyone is suggesting we get rid of the X11 API. Most graphics libraries (Gtk3, GLFW, Qt, etc.) are maintaining X11 and Wayland backends in parallel. Could Godot do that as well, just like it has different backends for different operating systems?

[reduz]

as i mentioned, both editor and release binaries will be distributed with
X11 backend for as long as X11 remains mainstream.
Keeping a Wayland backend that is not used does not make any sense, so we
will only officially implement it after X11 is no longer relevant.

On Wed, Jun 14, 2017 at 3:49 PM, William W Wold [e-mail]
wrote:

I don't think anyone is suggesting we get rid of the X11 API. Most
graphics libraries (Gtk3, GLFW, Qt, etc.) are maintaining X11 and Wayland
backends in parallel. Could Godot do that as well, just like it has
different backends for different operating systems?

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/576#issuecomment-308524251,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z28Lp9OYe8QZAtGaRKBMj5YvO83lGks5sECtVgaJpZM4CLKzr
.

[bojidar-bg]

Wouldn't it be possible to switch the X11/Wayland backends based on what is detected at runtime?

[reduz]

sure, but why bothering to go that far?

On Thu, Jun 15, 2017 at 7:35 AM, Bojidar Marinov [e-mail]
wrote:

Wouldn't it be possible to switch the X11/Wayland backends based on what
is detected at runtime?

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/576#issuecomment-308694184,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z20BStLZj1ELiynu2iHIjVYX0Wgwfks5sEQjggaJpZM4CLKzr
.

[akien-mga]

To clarify things again: We don't have anything against Wayland, but we don't see it as a priority. As flawed as X11 is, it's the one platform that will work everywhere across Linux distros, DEs and graphics servers, thanks to xwayland.

Sure, Wayland support would be nice to have, but it's not that important for us today. We think first and foremost about Godot users who will export their games to the "Linux" platform, and who might not be knowledgeable about all this stuff. If you're a Windows guy and you read in the docs that you need to provide X11, Wayland and Mir (hopefully the last one should go out of the picture soon) binaries to your Linux players, you'll just skip the platform altogether as "meh, too complicated". As game engine developers, we strive to support what will benefit everyone with the least effort from the game developer.

Now, if someone really wants to have Wayland support, you basically just have to write platform/wayland. Check platform/x11 to see how it's done, and reimplement that for Wayland. Then everybody will be able to compile both X11 and Wayland binaries if they care about it. But it will be on a "contributors want it and make it" basis, as mentioned above we don't see an immediate gain for our users in Wayland support, so the core team won't work on it as part of the official roadmap.

[wmww]

Sounds good. I agree it shouldn't use up the time of core contributors. I'm (slowly) learning Wayland, and may implement or help implement this in the future as it is a very important feature to me personally. If anyone starts work on it, please keep this issue up to date.

[minexew]

Wondering why not just use SDL

[reduz]

If anyone wants to submit a wayland or SDL backend, and are willing to
maintain it, then we can close this issue. Meanwhile, it will probably
remain open forever..

On Aug 29, 2017 4:23 PM, "Minexew" [e-mail] wrote:

Wondering why not just use SDL

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/576#issuecomment-325770050,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z25cT_DmQS7Ukx5dMerfnq5lxk9Y2ks5sdGU0gaJpZM4CLKzr
.

[reduz]

I don't mind that such thing never happens myself anyway

On Sep 6, 2017 4:28 PM, "Juan Linietsky" [e-mail] wrote:

If anyone wants to submit a wayland or SDL backend, and are willing to
maintain it, then we can close this issue. Meanwhile, it will probably
remain open forever..

On Aug 29, 2017 4:23 PM, "Minexew" [e-mail] wrote:

Wondering why not just use SDL

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/576#issuecomment-325770050,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z25cT_DmQS7Ukx5dMerfnq5lxk9Y2ks5sdGU0gaJpZM4CLKzr
.

[akien-mga]

For the reference, the work that @hpvb is doing in #16744 will be a pre-requisite for implementing Wayland support, not as a new platform (like the current "x11" one), but as a driver for the new "linuxbsd" platform.

[studioschade]

First let me say _I understand and agree_ with the points about NOT supporting a Wayland as a Godot backend, but I've got a more subtle question/request related to this.

I do research on immersive computing / user interfaces and after exploring many, many, options have landed on Godot as the ideal platform to begin building a new kind immersive window manager for Linux desktop. I explored using X11, met with some experts in the field and with their help, determined that Wayland is the way to go for this purpose with Godot acting as the compositor. I also met with the original author of Wayland - Kristian Høgsberg for feedback on my proposed Wayland/Godot architecture. The good news is I think it makes sense and will work, the bad news is c graphics code, EGL surfaces and contexts, etc are slightly beyond my realm of understanding and I've been stalled at this point for a while.

Something I've learned from these discussions is that its very easy to get confused about what Wayland is and and what it is not, and there is a tendency to over-complicate what a "compositor" is in this context. A key point that can be hard to Grok is that it does not matter what graphical backend Godot uses, X11 is fine. Wayland clients do not know/care how the compositor renders their textures.

The reference implementation of the Wayland protocol includes libraries which can be compiled into C programs. One library is for making Wayland client applications, and another is for compiling into Wayland server/compositor/window mangers.

This library is basically an API for:

• a client application to send its texture/pixmax/gl surface to a compositor via unix file sockets and notify when there are updates.
• a compositor to send mouse and keyboard input back to the client application.

The solution that I'm proposing is to use gdnative to bind the c server library to Godot so that Godot can be used to render arbitrary Wayland clients(and by extension all X11 apps via xwayland) as viewport textures. At the basic level this would enable things like real linux terminals and web browsers inside 2d Godot apps. More interestingly, this would provide a starting point for a new generation of 3d immersive VR/AR user interfaces with godot at the core.

I expect and appreciate skepticism about the end goal, but I'm hoping to get feedback on if what I've proposed is technically feasible and a reasonable way to go about implementing the above. Other options might include implementing the wayland server library as a compiled Module?

Baastions godotvnc application bears some functional similarities and may have useful reference code, specifically for making the textures and sending keyboard/mouse input to the clients, but does not address the core need for a variety of reasons outside the scope of this already rambling post :)

Advice is appreciate, Thanks!

[vnen]

@studioschade that's an uh... _interesting_ project. It certainly isn't trivial. You'll probably need to fork Godot and maintain a set of patches to achieve this. I'm no expert in this stuff by far, but I imagine you need some special boot up code for Godot since it usually relies on the window manager, and since it'll _become_ the window manager, it might need some other way to start. It likely requires another folder in the platform folder of Godot source (hence the need to a fork). I think even if we _did_ have Wayland support, it wouldn't be much of help to you (I might be mistaken though).

What I suggest is to hang around Godot community channels (in particular the IRC channel #godotengine-devel on Freenode, since most devs are active there) to ask some questions. Maybe open another issue here if it is pertinent. You're going into uncharted territory, so my advice is asking the devs to learn how Godot works as it is then see what you need to do to implement your idea (it shouldn't take long to understand Godot, especially when focusing on a single area).

I wish best of luck in your endeavor.

[studioschade]

Thanks for the response and advice, I will start hanging out in the IRC rooms again. You are correct, supporting Wayland as a backend doesn't help this.

A clarification that I think should simplify this this significantly (and eliminate the need for a fork) is that goal isn't to replace the existing window manager with this first step. The goal is to bootstrap the development of the apps that might justify the more drastic changes.

An example of a hierarchy could be:
Ubuntu < X11 < Gnome < 2D Godot Compositor < Wayland Client

This sub-optimal obviously, but I think not as bad as it sounds. The Wayland Client can run headlessly and Godot would be the only thing accessing the clients surface, which its getting directly from GPU memory. There is certainly some overhead but one should still be able to for instance, run high performance games/apps with graphics acceleration in the viewport if desired.

[wmww]

There are already multiple related projects. One is Simula. I don't think its usable yet, but they've made some progress. I have serious doubts about both their choice in language (Haskell) and Wayland library (Weston). I don't know why they don't mention it in their repo, but they are using Godot now.

@toger5 has also expressed interest in such a project. Me and him were planning on connecting the Mir library to Godot with gdnative for this purpose (full disclosure: I work on Mir). Some code for this had been written a while ago, but needs updating. If you're also interested in this, we may have reached the critical mass where it makes sense to make communication channel dedicated to Wayland compositors with Godot. Do you use Matrix?

[toger5]

@studioschade your first post tremendusly elevated my mood!
This is exactly what i was taking into consideration with @wmww as he mentioned already.

Some excited blablabla... :

I really see your vision of having godot as a tool to create compositors, from simple 2d compositors with unbeatable development speeds because of godots ruch set of tools (I'm especially refereing to the easy to use api's)
To the option of having 3d interactive environments where you can run linux apps.
A vr modeling tool (created with godot) could run Krita or Gimp to edit textures on a virtual screen inside the vr env.
(Or obviously a vr desktop/window manager based on godot)

About the mir approach

Mir started as a alternative protocol but nowadays it's a "framework for a compositor" kind of thing. It handles all the low lvl madness of handling/ creating shared graphics buffers, and starting as a standalone compositor which can run without any server (like Xorg).

I also was considering the "Ubuntu < X11 < Gnome < 2D Godot Compositor < Wayland Client" approach. But for beeing a wayland compositor you need to be able to allocate shared graphic buffers so you can acces the buffers from other applications and than show their content. I also was planning the same approach you mentioned but the mir guys pointed out that this would not work because of acces restrictions to the graphics buffer.

@vnen it would be best to make it work with as little changes to godot core as possible to be able ro benefit from all the changes made to godot. Maybe it would be enough to create a module for all the ne nodes required to expose the window manager api. And as the other big part of the project there would be a bew backend (os folder) for mir. This backend than would start the godot instance, the wayland server, and would be capable to run as a display server directly on ubuntu for example.

[toger5]

I think such a project can be really benefitial for godot itself too.
I would definitly work on the user interfaces to make them even more solid to build a desktop.
And also the input system would be solidified and worked on.

[Sslaxx]

@toger5 - it may also distract a bit too much from Godot's primary purpose. At that point it essentially becomes a general-purpose development environment. Whilst taking cues from those isn't a bad thing, Godot isn't really meant for (or optimal for) non-game-related application development.

[studioschade]

@toger5 (and @wmww) the feeling is mutual, I've been pursuing this in one form or another for the past few years while I've talked with dozens of great engineers about this, I had to drive and nobody has been interested in teaming up to push forward or really shared the vision. Its _very_ encouraging to find some others pursuing similar goals. I'll sign up for matrix so we can take this discussion out of this poor hijacked request, but I would like want to respond to what's been said and also provide some context for how this came about.

I think it's natural to be skeptical about the value of more immersive user interfaces and also to question if a game engine would be reasonable vehicle for that. This is a _much_ larger discussion and beyond the scope of this request. I’ll just say that what started as fun short-term work assignment become a personal quest after learning about the achievements of Douglas Engelbart in the 1960's. These include making the first prototypes of computers with monitor/keyboard/mouse, the internet, hypertext, gui's, word processors, video conferencing,databases, emails, lots more. More importantly, learning about the philosophy that led those achievements and his opinion that this work is still in its infancy is what motivate this work for me now. YMMV

A few words on why I think a game engine is the path forward. There have been demo standalone 3d Wayland demo compositors and windows demos that cost money for a few years now. However, it only takes a few hours of being a disembodied observer, floating in a void full of abstractions to stop feeling like a human being. I’m interested in building tools to extend human capabilities and collaboration and severing the link between mind and body is probably a poor first step.

I spent about a month performing a self-induced torture routine that involved spent as many hours as I could tolerate in VR, trying to do work find solutions to why it sucked. To stay human, it helps to bring in 3d models of things like desks, a floor and other common objects and input devices. You’ll also quickly start to want realistic materials and lighting for the models. You’ll need physics to interact meaningfully with your world and networking to interact with other people. As your virtual space gets more complex, you’ll want a visual editor to design the space and UI paradims. You need a game engine.

So, back to technical stuff

@wmww re:Simula: Simula is forked from the Motorcar demo Forrest Reiling made for his masters thesis. I met with Forest last year to get his take on how to move beyond the demo, and he and I are occasionally in the same orbit. My summary of his advice is:

1. Don't use X11, he wasted lots of time on that rabbit hole and not the right tool for the job.
2. Don't fork Motorcar, its based on an outdated QT API and has features are interesting, but tangential to this goal.
3. Do use Wayland, its explicitly made for things like this.
   He was also aware of Simula and like you (and me) had doubts that Haskell is the right choice for this project. None of this is to intended disparage that project, the devs, or Haskell. I have much respect that they've been pushing forward, and although I suspect this is another rabbit hole, would be very happy to be wrong.

I think we're all seeing glimpses of the same vision from different angles. I've also gone down more than my fair share of what I think are dead-ends on this.

This includes, but is not limited to:

• Using all pre-existing 3d window manager solutions I could access, including but not limited to Virtual Desktop, VR Toolbox, Hololens, Android/Daydream
• A custom solution using Windows and Unity
• Using VNC as a conduit
• Using chrome remote desktop as a conduit
• Using embedded web browsers with Cloud apps and/or Cloud9 IDE
• Using X11 with many creative hacks including "eye candy" linux apps like Beryl or Compiz
• At the advice of software engineers who are much better than me, compiled and experimented with many pre-existing Wayland Compositors either for standalone use or as reference for a Godot plugin/gdnative app. These include weston, weston-lib, wlc, swc, sway, and others ( but notably, I did not try Mir :-)

I now believe that the primary difficulty is more conceptual than technological and that the biggest barrier is over-complicating the problem.

To quote a few sections from the Wayland display server documentation.
“Applications can allocate their own off-screen buffers and render their window contents directly, using hardware accelerated libraries like libGL. In the end, what’s needed is a way to present the resulting window surface for display, and a way to receive and arbitrate input among multiple clients. This is what Wayland provides, by piecing together the components already in the eco-system in a slightly different way.

“ The extra requirement for Wayland enabling is the EGL_WL_bind_wayland_display extension that lets the compositor create an EGLImage from a generic Wayland shared buffer. It's similar to the EGL_KHR_image_pixmap extension to create an EGLImage from an X pixmap. The extension has a setup step where you have to bind the EGL display to a Wayland display.

Then as the compositor receives generic Wayland buffers from the clients (typically when the client calls eglSwapBuffers), it will be able to pass the struct wl_buffer pointer to eglCreateImageKHR as the EGLClientBuffer argument and with EGL_WAYLAND_BUFFER_WL as the target. This will create an EGLImage, which can then be used by the compositor as a texture or passed to the modesetting code to use as an overlay plane.

This is a key point, though admittedly this is also the point where my motivation firmly collides with my lack of experience/knowledge about framebuffers, EGL, and C programming and Godot internals. My reading of that implies that the core rendering goal is being able to hand godot a EGLImage such that it that it can render it into a viewport texture. My understanding is that this same core issue exists whether you are running a single app as a wayland client, or if you are running a Wayland compositor or are otherwise nesting different clients.

If this makes sense, I’d argue that the core things a compositor is responsible for (drawing to screen and processing input) godot already does very robustly. Adding a compositor middleman not only leaves you with the same problem you started with (getting a texture into godot and passing input back) but also adds the baggage of the middleman. Any higher level composition of multiple surfaces can and should be handled inside the 2d or 3d Godot scenetree as separate viewports. Leveraging Godots workflows and visual editor is a large goal of doing this to begin with. Similarly, with Godot its trivially easy to redirect inputs to an arbitrary objects/viewports.

When I met with the author of Wayland, it was to confirm that this plan is rational and the simplest way to accomplish the goal. He agreed that is how Wayland is intended to be used and should be the right way to go (though its certainly possible that something about Godot I failed to describe/understand precludes this from working). This is why I mentioned just using the core server library that is baked somewhere into all the compositors I looked at. Godot is the compositor. That said I’ve not looked at Mir. I guess the question is if the value it provides is enough to justify being the stack?

If this is all true, I’m hoping this can be as simple a wayland server gdnative plugin to handle communications and a viewport-like node that can be dropped in and pointed at a binary on the system to launch and talk to. No core changes to engine whatsoever and any additional WM features can be developed as godot projects by users like me.

I noticed that the StreamPeer docs say it is ”an abstraction and base class for stream-based protocols (such as TCP or Unix Sockets). It provides an API for sending and receiving data through streams as raw data or strings.”

Given that the communication protocol is done over Unix file sockets, could StreamPeer maybe be used in conjunction with a Viewport Texture or similar for the rendering side?

Thanks, I hope to work with some of you soon!

[wmww]

I made a Matrix room for further discussion here: https://matrix.to/#/!rBmyrWnyssyCORKYgR:matrix.org. Anyone interested in helping or just lurking is welcome to join.

The vast majority of Wayland compositor complexity is not graphics compositing itself. Mostly its dealing with the Wayland protocol (input seats, XDG positioners, subsurfaces, oh my!) and handling low-level output (DRM/KMS + libinput if you want it). Mir (or a similar library) is absolutely worth integrating into the stack. I've tried building a compositor from scratch. I put 3 months into it, it would take about a year to get it right (little of that work is stuff Godot can help with). With Mir we should be able to get something running within a few weeks of work.

See you on Matrix!

Edit: fixed matrix room permissions and link

[studioschade]

hi sorry for the continued spam. I'm admittedly new to matrix but when I try to connect to that room it says I'm not invited. Does it have some permissions that need to be opened up?

[wmww]

Whoops, had the permissions wrong, sorry. The link above will now get anyone into the matrix room. \

[lboklin]

Hi, I'm one of the contributors to the Simula project mentioned. While it doesn't solve the problem of porting the Godot editor to the Wayland API, it appears pertinent to the discussion as it provides a way to run Wayland applications in Godot.

The idea is that Simula will partly be a drag-and-drop asset that can be integrated into any Godot project (for Linux) as a Wayland compositor, and also provide the basis for a fully featured VR DE. No Haskell knowledge is expected in order to use this asset in your project.

We would appreciate anyone willing to help us test it and provide feedback or otherwise contribute. It is in a roughly working state but there are things that need to be worked out, such as build process streamlining and UX improvements.

Code contributions are of course very welcome, but we do recognize that Haskellers are few and far between, so our expectations are set accordingly. It is however a wonderful language for those interested in learning, but I digress :)

More info available at https://github.com/SimulaVR/Simula including a basic video demo.

[katp32]

Low-level KMS/DRM support, as well as UDev input support would also open up compatibility with some other more obscure setups, in addition to Wayland. I hope it could be investigated further.

[lboklin]

@atoms118 This fork has a patch for raw keyboard input. Don't think the Godot devs would be interested in upstreaming it so there's no PR. Any feature not being merged into mainline that is useful for building a compositor could go into it.

[est31]

Great news: Wayland expert @ddevault has announced they will start work on a Wayland backend for godot.

[NilsIrl]

I just tested Godot on sway with Xwayland and I just want to say that it worked flawlessly. Really a pleasure to use.

[aaronfranke]

Should likely be placed on the 4.0 milestone. Depends on https://github.com/godotengine/godot/pull/27266 being merged for 4.0.

Then, see: https://github.com/godotengine/godot/pull/23425, https://github.com/godotengine/godot/pull/23426, https://github.com/godotengine/godot/pull/27463, and https://github.com/godotengine/godot/pull/27747

We should probably remove "pr welcome" since we already have five conflicting PRs... 😄

[akien-mga]

We should probably remove "pr welcome" since we already have five conflicting PRs... 😄

Indeed, that was a mistake. PRs will be welcome once we have done the OS/Display split in the master branch for 4.0.

[akien-mga]

PRs will be welcome once we have done the OS/Display split in the master branch for 4.0.

This was now implemented with #37317.

[akien-mga]

Superseded by https://github.com/godotengine/godot-proposals/issues/990. Please add comments there with relevant technical information from this thread that should be kept in mind when implementing Wayland support.