Wlroots: Support for checkpoint/resume (CRIU) of wayland apps?

[jamadazi]

This is a continuation of swaywm/sway#5587 as it is more appropriate to discuss here.

It would be amazing if sway and other wlroots-based compositors were able to natively support checkpoint and resume for GUI applications.

AFAIK, such functionality was never before supported by any project, because it is difficult to do under an X server. But now with Wayland, it is no longer necessary to rely on an external component we cannot control (the X server), so the compositor (which is entirely under our control) could implement whatever magic is necessary.

I am not familiar with the technical details of what this would entail (how to handle GPU rendering? the surfaces? the wayland protocol state?), but this is why I am creating this issue to start a discussion.

I know that this is uncharted territory, but if other people are interested, this could make for a truly unique desktop experience.

I read somewhere that there was an attempt to do this in Weston before, but it was never finished.

This would allow for so many interesting use cases, like session management (snapshotting specific running applications and restoring them after a reboot, or just at a later point), or save-states for games. It would effectively obsolete OS-level hibernation for me.

[AdrianVovk]

What do you mean checkpoint and resume? Being able to snapshot the entire state of an app perfectly? Or just the size/position of its windows

[emersion]

I don't think this is possible, and I don't think this is the compositor's responsibility.

[jamadazi]

The technology for checkpoint/resume of Linux processes (yes, snapshot the entire state of an app perfectly) already exists: https://criu.org

This is used on servers by container technologies like Docker and such, for live migration and other use cases. It will (almost) perfectly snapshot the state of a running process and then recreate it in the future, possibly even on another machine.

You can try it out for simple headless processes using the criu commandline utilities. For example, I managed to successfully snapshot and restore a ssh daemon.

The problem is that it cannot directly be used for GUI apps, because there is also state in the compositor (wayland) or the X server (X11), such as the open windows that are currently being displayed and the ongoing protocol connection/session. So it needs some assistance.

The compositor needs to know that the application is about to be suspended, so it can gracefully close the windows, put the protocols that it uses to communicate with the client in some good state, etc. Then on resume, it will need to know to reopen the windows and do any other preparation so that the process can start running seamlessly.

The CRIU website has some information about the limitations and design challenges involved. It is what inspired me to post this issue, because it seems like something that would actually be possible. See these links:

https://criu.org/X_applications
https://criu.org/Integration#Wayland.2FWeston (apparently there was an attempt to do this for wayland in weston)
https://criu.org/What_cannot_be_checkpointed

See this for inspiration for the kinds of use cases that this technology can be useful for:
https://criu.org/Usage_scenarios

[ammen99]

The technology for checkpoint/resume of Linux processes (yes, snapshot the entire state of an app perfectly) already exists: https://criu.org

This seems to be working for containers, but it doesn't seem to work for regular desktop applications, does it?

[jamadazi]

The technology for checkpoint/resume of Linux processes (yes, snapshot the entire state of an app perfectly) already exists: https://criu.org

This seems to be working for containers, but it doesn't seem to work for regular desktop applications, does it?

Correct. This is exactly why I am making this issue. For it to be made to work for desktop GUI applications, it would need integration with the compositor (for wayland apps) or X server (for X11 apps).

If we figure out how to implement what is necessary, wlroots-based compositors could become the first to support this (for wayland apps at least). It would be a truly novel user experience. ;)

[AdrianVovk]

@jamadazi Here's the trouble: I don't see this being feasible or stable in modern desktop distros. Here's why: An application's state is much more complicated than its own memory and its communication with the compositor. What about talking to other system services?

For example, let's create a hypothetical music player A. Since it's a music player, A will ask gnome-settings-daemon's media-keys plugin for control over the media keys (play/pause/skip/etc on the keyboard), so that it can respond to those signals. To do so, on startup it will have to create a proxy object to g-s-d via dbus and keep it in memory. Whenever the app is focused it will then call a function on the proxy object to request back control of the keys. Now, imagine the app gets snapshotted, the system reboots, and the app gets resumed. What happens when you focus the app? g-s-d is now likely running under a completely different dbus handle and PID. On some systems, the entire machine will have a new dbus id. The app will try to use its existing proxy and that will very likely crash

What about apps that try to obtain privileges with polkit? Let's say there's an app that sometimes needs access to a special device. To do this it would need to create a polkit action that chmod's the handle in /dev, and then whenever the app needs the device it'll ask polkit for permission. The user will type in the admin. password and the file will be chmoded. The app can now manipulate the file. Then the app is snapshotted, the system is rebooted, and the app is restored. The file is now reset to how it was before, but the app will continue thinking it has permission to manipulate it. Chances are that'll lead to a crash, or at least strange error dialogs.

This might work for isolated apps that don't integrate with the system at all, but I don't see it working on a fully-featured app in a fully-featured desktop environment

Edit: CRIU themselves bring up another good example. What about apps that talk to hardware? They can load a bunch of state into the hardware and then get checkpoint/reboot/restored. The hardware will no longer have the state the app will think it has.

[jamadazi]

OK, you bring up good points. Now that I think more about it, indeed far too many desktop apps would be disqualified (basically anything useful), especially since modern toolkits use GPU rendering, which means hardware state. It would be far too much work to implement the needed workarounds for every individual aspect of the system all at once.

Although there is still some hope for the distant future. The different components could be tackled one by one. For example, if I were to somehow make dbus handle C/R gracefully, many apps/users could benefit, as there is a lot of non-GUI software that uses dbus. Similarly with polkit. It would open up C/R to such software and result in fewer remaining things missing to support GUI apps. Eventually it might make sense to revisit this issue here.

I guess I will close this for now, as it indeed does not make sense with the state of things as of today. But I am still hopeful. At least now I know that if I want C/R on Linux to be better supported overall, I should direct my efforts towards things like dbus first, before wayland.