Gnome-shell-extension-gsconnect: Multimedia controls to simulate media key press

[OlegWock]

First of all, thanks for awesome extension, it's my main reason to update to 18.04 :)

I have one proposal, but I don't know is it extension area. Small info: I wrote small Chrome extension to switch tracks at SoundCloud by pressing media keys at keyboard, it's pretty useful. But when I try to control multimedia from phone, it says that media players doesn't launched. Can extension simulate (maybe have option to toggle/disable such mode) media key press even if there is no media players launched?

Thank you

[andyholmes]

I've spoken to the upstream developers about this, and they don't seem interested in implementing this in their project. It might be possible to have the MPRIS plugin translate commands into media keys, but I'll have to do some research on how they work and whether they're compatible with Wayland. A fake MPRIS player will also have to be exposed so that the phone thinks there is media player active when there isn't, so this might take some time, if it's possible.

[andyholmes]

I'm going to mark this an enhancement and keep it around, but it's a lower priority and I still haven't figured out how I might do it. I just want to let you know I don't forget about these older ones.

[ferdnyc]

You know, this may not actually require anything other than the features already present in GSConnect.

By pure, _impossible_ coincidence, just yesterday I happened to have noticed something posted at Linux Uprising which serendipitously fits with both this request, _and_ the work @OlegWock has already done.

Meet otommod/browser-mpris2, written up by Linux Uprising as:
"Add YouTube Player Controls To Your Linux Desktop With browser-mpris2 (Chrome Extension)".

It's just what the name implies: A browser extension that turns an embedded video player on the browser canvas into an MPRIS2-controllable media player.

Their major caveat: "currently only supports YouTube". So, not immediately useful.

Ah, but! ...BUT! Since you _already have_ code for controlling SoundCloud's player... and in the form of a Chrome extension, no less...

You see where I'm going with this, right?

Stick your SoundCloud-controller chocolate into browser-mpris2's peanut butter, and you hopefully end up with SoundCloud's player controllable via MPRIS2, same as YouTube. At that point, GSConnect/KDE Connect should _already_ support remote controlling it, same as any other MPRIS2-compatible app.

Seems worth a try.

[andyholmes]

Gave this a quick, lazy try, but couldn't get the extension to work myself. I think if this can work, it's probably a better fix, but who knows. There _is_ a pseudo-plugin in kdeconnect-android that re-uses the mousepad plugin for a presentation remote; maybe they'll consider doing the same with media keys.

Will try a bit harder later with the browser extension, and try and bring it up in the KDE Connect development channel.

[ferdnyc]

There _is_ a pseudo-plugin in kdeconnect-android that re-uses the mousepad plugin for a presentation remote; maybe they'll consider doing the same with media keys.

...And given the limitations we've since encountered in trying to support _that_ plugin via the extended keyboard library, who knows how well media-key passing would really work even if it was added.

(Luckily the media-player remote control in GSConnect / KDE Connect doesn't rely on media key bindings at all — using MPRIS2 means that compatible media players attach themselves to the session DBus, allowing for two-way communication with GSConnect so that we can query the state of the player and send it control commands. GSConnect in turn relays that information (in both directions) along its link with the Android app.

Ironically, all that turns out to be a lot _easier_ to do (reliably) than faking some keyboard input to deliver keypresses. Mostly because the keyboard input support for X and/or Wayland is built atop a teetering pile of incrementally-enhanced technologies that ultimately stretches back something like 30 years.)

[mtijink]

You could try plasma-browser-integration (https://community.kde.org/Plasma/Browser_Integration), which integrates the browser music/video controls with MPRIS. KDE Connect/GSConnect can use that to control your music!

Despite its name, it doesn't really require KDE plasma.

[ferdnyc]

@mtijink wrote:

You could try plasma-browser-integration (https://community.kde.org/Plasma/Browser_Integration), which integrates the browser music/video controls with MPRIS. KDE Connect/GSConnect can use that to control your music!

Despite its name, it doesn't really require KDE plasma.

Huh! I just tried that in Google Chrome (unstable 74.0.3717.0) on Fedora 29 running under Gnome Shell.

It initially complained about the missing native host binary, so I installed that with sudo dnf plasma-browser-integration (along with the only two dependencies I didn't already have installed that it pulled in, kf5-krunner and kf5-threadweaver). Not sure if it was really necessary (I know krunner is for the browser-tab integration into KDE search, which I don't need), but it was painless enough and stopped its complaining. After that, and a browser restart, it kicked into gear.

• MPRIS2 controls are exported for an HTML5 video embedded on a personal web page.
• They're exported for YouTube videos.
• And they are, indeed, exported when playing music on SoundCloud.

As such, they are passed to the KDE Connect app automatically by GSConnect, and playback can be controlled from a paired Android device (or from Gnome Shell's own MPRIS2 integration at the top of its notification dropdown).

Impressive!

[ferdnyc]

Given the above, I wonder if this request is even really worth keeping open? I've been using plasma-browser-integration to control YouTube videos and other HTML5 multimedia content with MPRIS2 → GSConnect → KDE Connect since I installed it (so, the past 6 months already), and it's so seamless I usually forget that it's not just _part of_ the normal GSConnect experience.

Given the KDE Connect team's reluctance to support this on their end (and honestly, given that plasma-b-i exists, I can't blame them), and given that our own MPRIS2 support works far better than keypress emulation anyway, recommending that GSConnect users just install plasma-b-i if they want to control browser media might be the best approach.

[andyholmes]

Probably okay to close, even chrome comes with an MPRIS adapter now I think.