Gnome-shell-extension-gsconnect: XWayland Clipboard Proxy

[ofuangka]

The pull request https://github.com/andyholmes/gnome-shell-extension-gsconnect/pull/422 disabled Clipboard Sharing for Wayland in order to remove the GSConnect dependency on XWayland. This bug is intended to track a possible alternative implementation.

Steps to reproduce the behavior:

1. Log into a Wayland session
2. Open GSConnect Mobile Settings
3. Select a device
4. In the "Sharing" panel, Clipboard Sharing is not present

Expected behavior to be determined

System Details:

• GSConnect version: >20
  
  
  
  • Installed from: GitHub master branch
  
  
• GNOME/Shell version: 3.30
• Distro/Release: Arch

See the pull request linked above for additional details/discussion

[getzze]

It is not a perfect solution, but until clipboard is better handled in Wayland it would do: we could have a text area accessible from the gnome menu where we paste the text we want and it can be sent to the clipboard on the phone. At the same time, the phone clipboard is sent to desktop and the text area content is updated with it.
That would be a way to send text between devices.

[andyholmes]

I had two thoughts about how to do this. The first was just to run a mini XWayland process that used stdin/stdout pipes to communicate clipboard changes with the daemon. So in GSConnect there would just be a singleton with a signal and a few methods.

The second was something that occurred to me when I found out they built keypress simulation into gnome-shell's Clutter to make their new OSK work in Wayland. Since StClipboard also works in Wayland I thought it would be possible to export those both over DBus. That would kind of kill two birds with one stone, but the DBus interface would have to be always-on, and probably when Atspi works on Wayland we'd want to revert back anyways.

we could have a text area accessible from the gnome menu where we paste the text we want and it can be sent to the clipboard on the phone.

Probably this wouldn't be a bad idea anyways, since it would be a more accessible way to sync clipboard content on demand. I guess I just worry about UI clutter and we'd need a way to inject the StEntry since the device menu is built from a GMenu.

[getzze]

I like your second idea, although I don't know if dbus messaging will be fast enough for mouse and keyboard events.

[andyholmes]

I don't know if dbus messaging will be fast enough for mouse and keyboard events.

That's good point, although direct DBus connection is an option and still easier than writing some pipe protocol.

On the other hand, I just saw this MR although haven't had time to really look at it (https://gitlab.gnome.org/GNOME/mutter/merge_requests/320). I'm guessing because of how things like clipboards work in Wayland they don't have a choice but to build it into the compositor/mutter.

[ArtyomZorin]

I came across the new release of the Sway compositor for Wayland, and I saw that they included this library for interacting with the clipboard: https://github.com/bugaevc/wl-clipboard

Not sure if it'll work with Gnome, but it might be worth a look.

[ferdnyc]

@ArtyomZorin I looked at wl-clipboard a while ago. There are two main issues with using it in GSConnect:

• It's not actually a library, it's a pair of command line utilities to get and set the clipboard contents. In theory GSConnect could use those utilities as helper programs, as it does several other tools, but...
• It's entirely "pull"-based clipboard manipulation. IOW, it's meant as a way to manipulate the current clipboard, when the user wishes to do so. For clipboard updates coming from the Android side (when a selection is sent from the device by the KDE Connect app) that's fine, gsconnect could call wl-copy and pass it the clipboard data from KDE Connect. But GSConnect also needs to be able to _automatically_ pick up any changes in the current clipboard and/or primary selection, so they can be mirrored to the device. wl-clipboard didn't appear to be useful for that side of the functionality, since there's no way to know _when_ wl-paste should be called to collect the clipboard data.

[andyholmes]

Ugh, so I was looking into StClipboard since it seems to work in Wayland by way of some XWayland hackery, but unfortunately it doesn't have any signals like GtkClipboard. Dealing with this in GJS means polling in gnome-shell's main thread, which is what Clipboard Indicator does. Although a idle GSource with low priority probably shouldn't be intrusive, we then have to pass it over DBus as well.

Presumably GtkClipboard should actually work in Wayland (from what I can tell), however in Gtk/Gdk4 the clipboard seems to have been moved to Gdk. I might have a look into GtkClipboard, because I'd much rather this get fixed upstream before they stop doing releases of Gtk3, but I really have no idea what the plan is.

[ferdnyc]

I'm not sure if I mentioned this previously, apologies if I'm being redundant.

In my testing, CopyQ appeared to be fully-functional (or, at least, has the full functionality GSConnect would require in order to do clipboard sync, meaning selection/copy buffer capture and update) when it's running as an XWayland application. But...

1. It's a Qt application, so it's got a lot of heavy dependencies.
2. At least in my limited testing with the build in Fedora's repo, setting Qt to run in Wayland-native mode via the QT_QPA_PLATFORM=wayland environment variable breaks the necessary functionality (and, indeed, most functionality).

[andyholmes]

I had a look at the source of CopyQ, but it looks like it only really supports X11, and probably just works on XWayland because of GNOME using a little hidden GDK window.

[diegovasconcelos]

Look:
https://www.phoronix.com/scan.php?page=news_item&px=GNOME-3.34-Clipboard-Manager

[rugk]

This idea has not really been discussed yet:

we could have a text area accessible from the gnome menu where we paste the text we want and it can be sent to the clipboard on the phone.

IMHO, I very much like this as a workaround.
As it seems clipboard support is not coming soon, and I really, really miss it (it was one of the main reasons to use GSConnect), would not something like this be useful?

[andyholmes]

It's a less ideal solution, for the reason that it will require arbitrarily injecting a text entry into the menu. Currently, the whole device menu is exported over DBus using GMenu, which is very convenient and all done using pre-existing C code. We simply add and remove menu items and submenus in the service and they're picked up by the shell extension.

I have a branch somewhere with a partially working menu editor, which will be necessary if we ever want users to have the option of customizing the menu items. If we start injecting elements into the menu in this way, that will become a lot less feasible to do. It's also not very HIG, and complaints about that before was why I removed the circular battery graphic.

More generally, the problem with Wayland and KDE Connect is that it presumes access to desktop resources that often directly contradicts to the application isolation that Wayland presumes. I think the simplest way to fix it for now is just to run a small XWayland process and pipe the changes back and forth, I just haven't had time to do it.

[andyholmes]

I threw together a quick fix sitting in the xwayland-clipboard branch. If something goes wrong, and it might, it will probably go very wrong and could spin up your CPU and use enough memory to start swapping out. Here's a build for canaries testers:

[email protected]

[ferdnyc]

I'm still running X on my daily-driver (Nvidia card, binary kernel module), but I'll fire up my laptop w/ F30/Wayland later and give it a try. Been a while since I've visited a coal mine...

[ferdnyc]

Hmm. I installed the zipfile version you provided, @andyholmes, but something's being weird... I paired the phone to my laptop no problem, and everything looks normal (and enabled) from that end. But in the phone I'm only getting a tiny subset of plugins even AVAILABLE. And clipboard sync isn't one of them.

Everything is normal with the pairing to my desktop, still. Maybe I need to unpair/disconnect the other machine?

[ferdnyc]

Nope, same thing with the other computer unpaired. I've also restarted on both sides of the connection. I'll grab a debug log of the handshaking and pairing process when I get a chance.

[andyholmes]

Oops, sorry. There was a logic inversion mistake in there. Here's an updated ZIP:

[email protected]

[ferdnyc]

@andyholmes
So I gave the replacement zip a quick spin, between my laptop and my phone with Google Docs open.

• Text typed into GEdit can be copied on the laptop, then immediately pasted on the phone.
• Text copied in Google Docs on the phone can be pasted into LibreOffice Writer on the laptop, again no issues.
• If the text has any formatting applied to it, that's discarded and the plain text comes through, as expected.
• Copying non-text content, such as a selection area in the GIMP, results in an empty clipboard on the phone, also as expected.

The gjs process for xclipboard never even registered as far as CPU usage goes, and it takes up < 40MB of resident memory which is nothing significant.

All in all, looks good so far! Very cool.

[rugk]

it takes up < 40MB of resident memory

I'd consider 40MB for a simply clipboard proxy service pretty much… :thinking:

[andyholmes]

The gjs process for xclipboard never even registered as far as CPU usage goes, and it takes up < 40MB of resident memory which is nothing significant.

For me it was only about 9MB RSS, which I think is ~5 for the gjs engine and ~4 for the libraries, was yours really about 40?

[ferdnyc]

For me it was only about 9MB RSS, which I think is ~5 for the gjs engine and ~4 for the libraries, was yours really about 40?

Yeah, well... that's what top showed in the "RES" column, who knows what gets included in there? If I just run gjs and leave it sitting at the command line the process is already 24MB, however it measures.

ETA: Looks like top's "RES" column is all RAM allocation including shared memory, which in that same sitting-at-the-command-line gjs is already 18MB, so that probably accounts for the difference.

[andyholmes]

I've merged this in, since I haven't heard anyone complain. I'll spin up a RC sometime soon and see how that goes.

[terencode]

@andyholmes for GNOME 3.34 you would be able to use https://gitlab.gnome.org/GNOME/mutter/merge_requests/320

[ferdnyc]

@andyholmes for GNOME 3.34 you would be able to use https://gitlab.gnome.org/GNOME/mutter/merge_requests/320

Holy _crap_, that MR has a longer history than the city of Rome! :laughing: All resolved in the end, I guess. (Though that note from the initial comment, "but it may work on both x11 and wayland" — emphasis mine — is vaguely concerning. Maybe I'm just reading it too literally.)

[HEXcube]

I've merged this in, since I haven't heard anyone complain. I'll spin up a RC sometime soon and see how that goes.

I tried the new v25-rc1 prerelease on my Ubuntu 18.04 LTS with GNOME 3.28 Wayland session. And guess what, clipboard still doesn't work! :disappointed: When copying from desktop, I get a {} on phone and nothing otherwise.

[ferdnyc]

@HEXcube Hmm. I tested on 3.32.2... possibly 3.28 is missing some functionality.

Can you do this test, for starters? Open a terminal, and inside it run:

$ cd ~/.local/share/gnome-shell/extensions/[email protected]/service/components/
$ ./xclipboard

It'll just sit there doing nothing (possibly after a version warning message or two), but then after that, anytime you copy text it should be instantly output to the terminal where xclipboard is running. Then just hit Ctrl+c in the terminal when you're done.

For example:

$ ~/.local/.../[email protected]/service/components/xclipboard
Gjs-Message: 05:06:49.193: JS WARNING: [./xclipboard 5]: Requiring Gdk but it ha
s 2 versions available; use imports.gi.versions to pick one
anytime you copy text&gave the replacement zip a quick spin, inside it run:
```sh#Text typed into GEdit can be copied^C

(...As I went around this page and copied some selected text a few times.)

Reopening for possible 3.28 compatibility issues.

[ferdnyc]

Oops. For the email readers, I messed up my commands in the previous message. The correct commands would be:

$ cd ~/.local/share/gnome-shell/extensions/[email protected]/service/components/
$ ./xclipboard

(Forgot the all-important ./!) I've edited my original comment accordingly.

[andyholmes]

Here's an updated Zip with a little more corner-cases covered:

[email protected]

[HEXcube]

Can you do this test, for starters? Open a terminal, and inside it run:

$ cd ~/.local/share/gnome-shell/extensions/[email protected]/service/components/
$ ./xclipboard

It'll just sit there doing nothing (possibly after a version warning message or two), but then after that, anytime you copy text it should be instantly output to the terminal where xclipboard is running. Then just hit Ctrl+c in the terminal when you're done.

This is what I got on v25-rc1:

./xclipboard
Gtk-Message: 16:17:48.052: Failed to load module "appmenu-gtk-module"
Gtk-Message: 16:17:48.054: Failed to load module "appmenu-gtk-module"
dointerminaloutputL(...As I went around this page and copied some selected text a few times.)

[HEXcube]

With the new v25-rc2:

~/.local/share/gnome-shell/extensions/[email protected]/service/components/xclipboard
Gtk-Message: 16:32:18.035: Failed to load module "appmenu-gtk-module"
Gtk-Message: 16:32:18.036: Failed to load module "appmenu-gtk-module"
email re output to the terMxclipboard is running. Then just hit Ctrl+c in the terminal when you're done.Qsit there doing nothing (possibly after a version warning message or two), but th

[HEXcube]

Here's an updated Zip with a little more corner-cases covered:

[email protected]

When I tried this zip, got this output:

~/.local/share/gnome-shell/extensions/[email protected]/service/components/xclipboard
Gtk-Message: 18:05:39.682: Failed to load module "appmenu-gtk-module"
Gtk-Message: 18:05:39.683: Failed to load module "appmenu-gtk-module"
 output to th� (possibly after a version warning message or two), but then after that, anytime you copy text it should be instantly output to the terminal where xclipboard is running. Then just hit Ctrl+c in the terminal when you're doneUt'll just sit there doing nothing (possibly after a version warning message or two),

[andyholmes]

The xclipboard script seems to be working correctly. Are you sure you have clipboard sync enabled in the settings?

[HEXcube]

The xclipboard script seems to be working correctly. Are you sure you have clipboard sync enabled in the settings?

Yes, it's enabled on both phone and GSconnect.

[andyholmes]

Does generating a support log provide any information as to what's happening?

[HEXcube]

Does generating a support log provide any information as to what's happening?

Got this:
gsconnect.9F643Z.log

Output of xclipboard:

~/.local/share/gnome-shell/extensions/[email protected]/service/components/xclipboard
Gtk-Message: 18:30:53.829: Failed to load module "appmenu-gtk-module"
Gtk-Message: 18:30:53.831: Failed to load module "appmenu-gtk-module"
`It'll just sit there doing nothing (possibly after a version warning message or two), but then a4 output to the terminal where xclipboard is running.9 Then just hit Ctrl+c in the terminal when you're done.

[andyholmes]

From your log it seems like the xclipboard script is failing to start for some reason. Are there any messages in journalctl or GNOME Logs about this happening? It should be right around the time this message is printed:

[session uid=1000 pid=23016] Successfully activated service 'org.gnome.Shell.Extensions.GSConnect'

[HEXcube]

I got this from GNOME Logs:

WARNING: XClipboard not running

It wasn't displayed immediately after GSconnect activation though. I couldn't find any other references to xclipboard, even after trying multiple copy paste attempts on my notebook and connected phone.

[HEXcube]

I don't know if it's related, but gnome-terminal-server outputs this message :confused: :

Display does not support owner-change; copy/paste will be broken!

[andyholmes]

Display does not support owner-change; copy/paste will be broken!

This is because in GDK's Wayland backend the function gdk_display_supports_selection_notification () always returns false. This shouldn't be related to the problem though.

I'm guessing the error is not showing up in the log because it's a debug message, which won't be logged until Generate Support Log is activated after GSConnect starts. Give this Zip a shot and see if there are any errors during around GSConnect startup:

[email protected]

[HEXcube]

After testing this new zip, I got some error messages on log. These are some relevant parts of the log:

XClipboard log messages.log
GSconnect log messages.log

[HEXcube]

XClipboard log messages.log
GSconnect log messages.log

GitHub isn't showing preview for these files, so I thought I'll paste them here directly:

17:31:38 gjs: WARNING: XClipboard not running
17:31:38 gjs: WARNING: XClipboard not running
17:31:38 gjs: WARNING: XClipboard not running
17:31:13 gjs: WARNING: XClipboard Proxy: 
17:31:13 gjs: WARNING: XClipboard Proxy: @/home/user/.local/share/gnome-shell/extensions/[email protected]/service/components/xclipboard:90:1
17:31:13 gjs: WARNING: XClipboard Proxy: readContent@/home/user/.local/share/gnome-shell/extensions/[email protected]/service/components/xclipboard:76:9
17:31:13 gjs: WARNING: XClipboard Proxy: (gjs:3060): Gjs-WARNING **: 17:31:13.486: JS ERROR: Error: Expected type utf8 for Argument 'text' but got type 'object'
17:31:13 gjs: WARNING: XClipboard Proxy: 
17:05:02 gjs: WARNING: XClipboard Proxy: Gtk-Message: 17:05:02.519: Failed to load module "appmenu-gtk-module"
17:05:02 gjs: WARNING: XClipboard Proxy: Gtk-Message: 17:05:02.513: Failed to load module "appmenu-gtk-module"
11:25:34 gjs: WARNING: XClipboard not running

18:43:47 gnome-shell: [pixel-saver]: Finding XID for window [email protected] failed
17:31:56 dbus-daemon: ('Handling file: ', 'file:///home/user/.local/share/gnome-shell/extensions/[email protected]/stylesheet.css')
17:31:13 gjs: WARNING: XClipboard Proxy: @/home/user/.local/share/gnome-shell/extensions/[email protected]/service/components/xclipboard:90:1
17:26:06 dbus-daemon: ('Handling file: ', 'file:///home/user/.local/share/gnome-shell/extensions/[email protected]')
17:05:02 gnome-shell: JS ERROR: /org/gnome/Shell/Extensions/GSConnect/Device/349dcdcd550cddea: TypeError: this.device.action_group is null
_init@/home/user/.local/share/gnome-shell/extensions/[email protected]/shell/device.js:51:9
Menu@/home/user/.local/share/gnome-shell/extensions/[email protected]/shell/device.js:187:25
_init@/home/user/.local/share/gnome-shell/extensions/[email protected]/shell/device.js:237:20
_Base.prototype._construct@resource:///org/gnome/gjs/modules/_legacy.js:18:5
Class.prototype._construct/newClass@resource:///org/gnome/gjs/modules/_legacy.js:114:32
_onDeviceAdded@/home/user/.local/share/gnome-shell/extensions/[email protected]/extension.js:235:29
_onInterfacesAdded@/home/user/.local/share/gnome-shell/extensions/[email protected]/shell/remote.js:201:13
17:05:02 dbus-daemon: [session uid=1000 pid=4081] Successfully activated service 'org.gnome.Shell.Extensions.GSConnect'

[andyholmes]

Hi, sorry there haven't been any updates on this issue - it's still targetted for v25.

@HEXcube So far I think you're the only tester having problems, and you seem to be the only one on 3.28. Could you please try this ZIP here:

[email protected]

In particular these lines here:

https://github.com/andyholmes/gnome-shell-extension-gsconnect/blob/49364e555d001081e0cf10c6c635fd92351b0686/src/service/components/clipboard.js#L86-L90

https://github.com/andyholmes/gnome-shell-extension-gsconnect/blob/49364e555d001081e0cf10c6c635fd92351b0686/src/service/components/xclipboard#L69-L73

I believe these lines are the proper way to account for +/- 3.28 behaviour, but I haven't had a copy of 3.28 running in awhile. If the clipboard still doesn't work, think this may be the spot causing problems and might need your help with a little debugging.

[andyholmes]

I'm marking this as closed since the feature was released in v25. Please open a new issue if problems still occur with v25.

[HEXcube]

@andyholmes This bug is now fixed! :smiley: I tested the latest update of GSconnect from GNOME Extensions site. I'm running GSConnect v26 now on Ubuntu 18.04 LTS with GNOME 3.28 Wayland session and clipboard sync works smoothly. Thanks a lot! :kissing_heart: