Element-web: Push-to-talk in video/voice conferencing

[BloodyIron]

Whether it's Freeswitch or Jitsi (AFAIK Freeswitch is on the way out), having Push-To-Talk for channel chat would be seriously appreciated! Right now if you have a channel with a whole lot of people wanting to talk, it can turn into a really big mess.

Also, this PTT feature would need to work when the "app" doesn't have focus. I'm thinking this may not work unless the "desktop" version is used, but I'm not 100% up to speed with browser features like this.

It might also be a good idea to get a keyboard bind to toggle self-muting of your own microphone. Let's say, if your fat cat knocks a glass of water all over your desk, oh noes!

[t3chguy]

Also, this PTT feature would need to work when the "app" doesn't have focus. I'm thinking this may not work unless the "desktop" version is used, but I'm not 100% up to speed with browser features like this.

yes this would not be possible in-browser, Discord has the same issue

[BloodyIron]

@t3chguy yeah, I wasn't sure if that particular detail was "solved" by browser devs since I last looked at it. ;)

[t3chguy]

as far as I'm aware its done by intent, so web applications can't behave like keyloggers

[BloodyIron]

@t3chguy that... is actually completely reasonable! I hadn't thought of that. Please don't keylog me Facebook! ;D

[turt2live]

ftr this is listed in the voip metabug: https://github.com/vector-im/riot-web/issues/3025#issuecomment-287929855

[BloodyIron]

@turt2live when I found that thread, I wasn't sure if the PTT facet would get lost in the size of that thread. :( So, I opened this one.

Along the lines of the reason why we have individual issues, instead of just one big meta issue for every single thing people want. ;P

[turt2live]

I'm not declaring a duplicate, just providing a reference so people can find it more easily.

[BloodyIron]

@turt2live you monster! :O

[josephtocci]

Nextel PTT (Always online, push notifications with audio, Example: Zello)
Remember this? I was too young to have a phone, but my parents each had one and the PTT was far above everything else at the time. There were no smartphones at the time. It was faster than calling, and usually the message got through. These days texting won out, partially because if you put your phone down and came back to it you could just read it. PTT you might miss. The other reason texting won out is because you don't necessarily want the person you are talking too IRL to hear what the person on the phone said. There are pros and cons to this method. There is a decent following on Zello which does this. In fact I know someone who uses Zello with all his camping buddies.

Mumble PTT (Must be running, no push notifications for audio, login/join/accept call to use and hear PTT)
This one is more what people will do now. Especially gamers. Even my friend on Zello would probably use this instead. The way he uses Zello is to schedule an hour with everybody, and everybody goes online at the same time and talks. There is no reason to use the always on Zello if you are just going to chatroom with voice. Basically you just mute the microphone, and have a keyboard button to hold down to unmute. (Headphone button on phone) If someone is trying to be quiet IRL and can't get around it, he can plug his headphones in to hear, and type in the room instead of talking. Optionally a text-to-speech thing would be cool, but that is a separate subject.

My thoughts
I think the Mumble type PTT is what people actually seem to need. You set up a room for the game, and everyone logs on and plays the game. (Not necessarily games but games is the common example) Text is king unless you are doing something with your hands, if you are doing something with your hands, set up a call. It's not complicated.

Forward
So I recommend a keyboard button for desktop and headphone button for phone would be excellent shortcuts to temporarily unmute the microphone while it is held down for both native calls and jitsi conferencing. I would recommend Riot to certain people if it had that.

[t3chguy]

headphone button for phone

/me points at vector-im/riot-ios and vector-im/riot-android

[josephtocci]

Android issue created:
https://github.com/vector-im/riot-android/issues/1905

iPhone issue created:
https://github.com/vector-im/riot-ios/issues/1736

[josephtocci]

TL/DR, turn the camera on when you press the unmute button, unless you want an audio only conference call. Would reduce load

You could even make the same button to unmute the microphone to turn on the camera. I don't necessarily want my camera on all the time, but when I'm talking I want it on. So have the camera on but don't send anything to matrix unless I hold down the PTT button to unmute the microphone. You could make this the default for conference calls and not for 1-on-1. Does that solve the infrastructure problem? Maybe sometime in the future add the Nextel PTT, it is awesome, but if you say it's too much for now then ok.

It reduces load, and you can have an audio PTT only/no camera conference option for gamers, and a full conference call option for people that have 20 eyes to look at 20 different camera feeds for their friends. lol. Probably the best option considering infrastructure, and nothing has to change server side because load should go down even if you add a healthy amount of users.

[BloodyIron]

So any word when we can see push to talk for riot?

[BloodyIron]

Where is this on the radar? I know it's listed p3, but I'm not sure how that falls in terms of timeline of being looked at.

[t3chguy]

This would have to be implemented by Jitsi. (as Jitsi is what we use for Conferencing)

[BloodyIron]

Any chance the Riot team can co-ordinate that with Jitsi?

[BloodyIron]

Well, hoping I can appeal to them : https://github.com/jitsi/jitsi-meet/issues/3464

If anyone wants to help, please post her and in the linked jitsi thread. This really _needs_ to get implemented. Not being able to change the shortcut is pretty unacceptable.

[anoadragon453]

Just a heads up that I'm making major progress with this, and hope to have basic PTT functionality implemented very soon :)

[anoadragon453]

Got a very good chunk done tonight. Getting the functionality working required some extra PRs in other projects, but now I have everything I need to see it through.

Just working out the last fiddly bits, but I will do so tomorrow :)

[anoadragon453]

So I've managed to get proper functionality working! You can see a short and silly demo here: https://matrix.to/#/!DdJkzRliezrwpNebLk:matrix.org/$1541185902690rkqXW:amorgan.xyz

There's currently two problems however:

1. You're currently unable to edit the shortcut
2. The shortcut is registered when you add a Jitsi widget, not when you join in to the call, meaning it might not register in all cases

I plan to fix both of those (hopefully today) by integrating into Riot's settings, which I should be able to use to just register the shortcut on start-up and/or when the functionality is enabled.

Accomplishing all this required some PRs in an extra repo, notably https://github.com/WilixLead/iohook, and as such we're currently waiting for those to make it to master, which one already has.

So yes, functionality is nearing close, exciting!

[BloodyIron]

@anoadragon453 wait... you can link to Riot URLs, didn't even know that...

As for your work, woot! Awesome!

Does this work when the app is minimized/not in focus?

Also, for the shortcut change, are you going to push that upstream to jitsi as per your comment in : https://github.com/jitsi/jitsi-meet/issues/3464 ?

Woot! Thanks for your hard work @anoadragon453 ! :DD

[anoadragon453]

Thank you!

Does this work when the app is minimized/not in focus?

Yep!

Also, for the shortcut change, are you going to push that upstream to jitsi as per your comment in : jitsi/jitsi-meet#3464 ?

So turns out the jitsi people were right and we can just use their iframe API to trigger the change. They do, unhelpfully, only have a function for toggling audio, so for muting we first have to query whether the audio is already muted, and if not then toggle.

So essentially what I'm doing here is useful for electron apps that want to embed Jitsi and have PTT functionality to control it, but I'm not actually changing anything in the jitsi project itself.

[BloodyIron]

@anoadragon453 okay but I was also asking about the changing of the PTT key binding, my understanding is that needs to be added upstream to jitsi, is that not the case?

[anoadragon453]

Ahh, well since we're controlling it from Riot the keybinding actually happens in Riot. We aren't sending a "space" to Jitsi, we're sending a command to toggle mic audio from their API, so making the existing keybinding changeable in jitsi was not something that was necessary to accomplish it in Riot.

However, while jitsi people have said their shortcut code isn't very extensible, I assume someone could probably hack in a new item into the menu to change the shortcut.

Not promising anything on that end, but I am learning a lot about how a process like that may go with this project, so perhaps one day in the future?

In the meantime, use Jitsi through Riot and everything will be wonderful :D

[BloodyIron]

@anoadragon453 I primarily care about riot, I just didn't realise we could change the keybind from the riot end. I like where this is going! Yay! :D

[anoadragon453]

If you want an update, I'm currently getting close to implementing enabling the functionality and changing the shortcut in settings.

How to record what shortcut the user is currently pressing is still a bit of an unknown for me. I might try to make it work with iohook, or perhaps instead use a separate library as the user will have Riot in focus anyways. The only issue with the latter is iohook seems to use a different encoding for keys than most other libraries, so Shift - A in one lib may be Ctrl - 7 in iohook.

Anyways, I'll try stuff out and report back on how it goes!

[anoadragon453]

Alright! So after the work of tonight we have a nearly complete spot in Settings for push to talk now. It has a enable/disable toggle, a display of what the current shortcut is, and a button to set that shortcut. One thing that still needs to be fixed on the enabling/disabling front is enabling the shortcut on startup if you've enabled the option before already. Not entirely sure where to do this yet, but I'll likely find a spot soon.

In addition, I still need to figure out a way to record what shortcut keys are currently being pressed and then assign that to activate push-to-talk, but with the way I've set things up it should be as simple as dropping in a library/function into the right spot.

Some additional things that may be useful (but possibly not in scope of this PR) are:

• Setting Jitsi audio to mute-by-default when push-to-talk is on
• Having a separate settings menu where you can customize all the shortcuts in Riot
• Ability to set whether the shortcut should get captured by Riot or continue to pass-through to the OS while still toggling PTT

But other than that, things work as expected!

[ara4n]

(one thought: please make sure that the iohook stuff is only enabled if the user has explicitly turned on PTT. and please apply all possible paranoia to minimise the risk of a JS exploit in Riot somehow giving an attacker a route to turning the app into a systemwise keylogger through the magic of iohook...)

[anoadragon453]

@ara4n Very good point! Selective enablement can certainly be arranged. I'll need to look in to how far a JS exploit in electron could go. If it would allow for IPC calls then that would not be fun.

[ara4n]

i haven't looked into how iohook works at all, but i'm assuming that you're configuring it from JS to say "oh hai, please tell me whenever the user hits alt-space no matter which app is in the foreground". at which point a JS XSS could presumably say "oh hai, i'm interested in all the other keycodes too". So i'm wondering the native code of iohook has some way of being hardcoded to prevent that from ever happening (e.g. only ever letting you register a single keycode to intercept).

(This said, a JS XSS in electron is very bad news anyway, given there may be exploits which can escalate into running native code too, like the https://electronjs.org/blog/chromium-rce-vulnerability trainwreck. But we should still try to safeguard a trivial XSS from being able to break outside the web env via native extensions like iohook...)

[anoadragon453]

Seeing as I've thrown a few PRs at iohook already, I could easily add one that limits how many shortcuts you can set at once, perhaps in the package.json? Is that modifiable from electron?

[BloodyIron]

Oh, also, any chance we can get Mouse button support for PTT? :)

[anoadragon453]

We're definitely aiming for it!

[anoadragon453]

Alright, so push to talk in Settings is fully functional now. However, while it saves and registers the shortcut, iohook won't actually pick up on it because it listens for keycode instead of rawcode in keyboard events. I've created an issue to fix this and if not given much fanfare will create a PR for it soon as well. A release for iohook still hasn't gone out since my other PRs, so hopefully this one will make it in before a new release does go out.

But yes, with that we will have keyboard shortcut recording (already done) and then the ability to listen for the recorded shortcut! Unfortunately not that many key combinations are available at the moment, as well as mouse support currently not out, but I'll try to get a PR for all of this up and running and then we can have people with different OS' and setups try things and give feedback.

Finally, this currently only works on Dimension, but I'm adding support for Scalar, Riot's default integration manager, as soon as everything else is working. I've already got the dev env set up :)

[skylord123]

@anoadragon453 I love you. Keep it up.

[anoadragon453]

@skylord123 Thanks!

Alright, I think I've finally figured out a nice solution to the keymapping problem. This whole time I've been trying to record items using a library like Mousetrap so that we can have a nice visual of the current keys that are being pressed. The problem with this is that Mousetrap doesn't have a nice display for many items. Nevertheless, as it had a common mapping with iohook (the rawcode property), I went ahead and implemented it, and life was pretty good.

However, the fact that we couldn't properly display all combinations of keys being pressed still bugged me. If you pressed Enter, it would be ' ' as being the current shortcut, and while one could do a lot of regex to make this nicer, it's not an elegant solution.

Cue KeyboardEvent, which is a thing that exists, and actually generalizes to all OS' and international keyboards (as best as it can)! While this is great for display, it unfortunately doesn't have any codes that are in common with what iohook is expecting.

Then I realized, well we could just listen with both at the same time? And since this option is device-specific, if the rawcode of, say, Enter ends up being 65383 and 87 on Windows, that's alright as you'd be setting it per-device.

So, the solution is now, when setting a shortcut, use iohook and this fancy in-browser event listener to both display and get the OS/keyboard-specific keycodes for any given shortcut, even supporting fancy ones like Ctrl+ K + Enter + Backspace (for some reason), which wasn't possible with the Mousetrap setup.

As for security, this would require a function allowing iohook to listen to all keyboard shortcuts. Fortunately, I think we can limit this to working only when Riot is in-focus, cutting it off if the user decides to go out of focus. This is controlled in the main Electron process, and thus shouldn't be override-able as a result of an Electron XSS to the best of my knowledge (but someone please correct me if otherwise).

In the process of making Mousetrap work, I also submitted another PR to iohook, but now it looks like that functionality won't be needed after all. Oh well, hope it's useful to someone :)

[anoadragon453]

So the combination of KeyboardEvent and iohook was exactly what was needed, and we can now represent pretty much all keyboard combinations now (mouse to likely following in a separate PR :)

Still some bugs to work out, as well as getting this working on Scalar, but as a treat here's a video of it in action! The Missing Translation is due to, well, a translation for this new text not existing yet. And yes, visuals to hopefully be improved before the final version :)

https://streamable.com/ekqf5

[BloodyIron]

lol mrtestmanman nice! :D @anoadragon453

Thanks for your help!

[anoadragon453]

So after a short weekend break, Push to Talk is now completely working! I've done the following today:

• Made PTT initialize when a Jitsi widget is shown, and disabled when the widget disappears.

This is all instead of doing so only when the PTT option is enabled, or when Riot starts, which are both unnecessary. Of course, the option will still enable/disable mid-call

• Stopped recording a new shortcut when Riot loses focus.

There is justified concern of the shortcut recording feature, which records all keypresses when setting a shortcut to see which keys you're pressing. This feature is now always disabled when Riot is not in focus, even in the event of an XSS.

• Stop Riot from crashing when a secondary window was opened.

This one was important, obviously. Riot no longer crashes when you try to open a window, such as when sending a file or clicking on the taskbar icon :)

The final step is just to get the setup working with Scalar, which should happen shortly after I get a working Scalar setup going locally (about 80% of the way there, but need a few more pieces). After that will be the PR phase :)

[BloodyIron]

@anoadragon453

How does this impact those that want the option to have the mic wide open too? Is that just the behaviour when PTT is disabled in settings?

[anoadragon453]

If PTT is disabled, it will be the same functionality is it is now.

[anoadragon453]

Alright! Most of the day was spent on getting Scalar up and running, but now it finally is, hooray!

So will just need to hack on Scalar a little bit tomorrow and everything should be good to go!

[anoadragon453]

Ok, got Scalar working really nicely, and PRs for everything are now live!

• https://github.com/vector-im/riot-web/pull/7709
• https://github.com/matrix-org/matrix-react-sdk/pull/2280

In addition, I managed to make the following work while polishing everything up:

• i18n support, so no more 'missing translation' text
• Any key (in any combination) can be used on the keyboard, not just a limited subset
• Distinction between similar keys like Shift and RightShift

Mouse buttons might work (testing wanted!), but I'm envisioning we'll need to do some UI work for that (little picture of a mouse, maybe?).

Look forward to seeing these land on the /develop branch!

[BloodyIron]

Can we add to the future to-do list the Riot icon have a mute symbol over it when PTT is muted or something? Although, not sure how that would work for multiple room jitsi chat... uhhhh?

Also, when we get the audible cue for PTT mute/unmute sound, can we make that two sounds, and user-customizable and/or pre-defined in the Riot config.json? (for those of us who built it ourselves :D )

Also, thanks a tonne for this! Yay! Can't wait to use it! :DDD

[anoadragon453]

Can we add to the future to-do list the Riot icon have a mute symbol over it when PTT is muted or something?

Sure

Also, when we get the audible cue for PTT mute/unmute sound, can we make that two sounds, and user-customizable and/or pre-defined in the Riot config.json? (for those of us who built it ourselves :D )

That might be part of a big redactor which allows to customize all the different sounds of Riot, but I'll see how difficult it is to just add custom sounds for PTT for now.

Also, thanks a tonne for this! Yay!

yw!

[anoadragon453]

Mainlining has been momentarily stuck behind the concern that our build machines currently run MacOS (which can compile electron builds for Linux/MacOS/Windows), but compiling native node modules (of which iohook is) requires actually building on Linx/MacOS/Windows boxes. We can obviously accomplish this via VMs, but the infrastructure needs to be modified first.

In the meantime, as iohook provides pre-compiled binaries, the decision is to just use these until we have our own systems set up. The binaries are downloaded automatically during the typically npm install phase, so the actual build process will not change at all, unless you want to compile iohook on your own (instructions for which should be provided soon).

Other than that, the issues with the Settings UI have been fixed, and here is the current implementation :)

[BloodyIron]

@anoadragon453 any particular advantage to having iohook be part of the build flow vs binaries?

[jryans]

any particular advantage to having iohook be part of the build flow vs binaries?

I would assume the main advantages are around trust and future maintenance. When you rely on pre-compiled dependencies, you are trusting the dependency author to build them without anything malicious added in. On the maintenance side, they may also stop providing them for new Node / Electron versions. So in the long run, it's best to build from source, especially for security focused software like Riot.

[anoadragon453]

Pretty much this.

[BloodyIron]

right! any chance iohook will get baked into the main repo here then? (for those of us looking to do our own builds of riot? ;P )

[anoadragon453]

Shouldn't need to, the process is just running npm install like you normally do, but then going into node_modules/iohook and running npm run build.

[anoadragon453]

Scalar has just released a new build which includes the PTT patches, which means one no longer needs to run their own integrations manager to get this all working, they can just use the default in Riot.

Should make testing the above PRs much easier for everyone involved.

[BloodyIron]

@anoadragon453 I'm probably going to forget that in the future, would that be documented anywhere? So npm install pulls the iohook source too? And the build second step is on the local files?

As for integrations manager, was that necessary for jitsi before or what? I thought that part might still be required?

[anoadragon453]

@BloodyIron Haha, we'll definitely provide instructions in our repo, but if you like the build instructions are also mentioned in iohook's documentation.

As for integrations manager, was that necessary for jitsi before or what? I thought that part might still be required?

An integrations manager is required, yes. But now you can just use the one Riot uses by default which has the new Jitsi patches integrated. The latest dimension is also supported as well.

[BloodyIron]

oh btw, did you test PTT with screen sharing at all? does the behaviour change there at all?

So Riot has Jitsi baked in now? (next build release?) so it won't need to load the "widget" when starting/stopping chat, or what? How do you tell it what Jitsi user to use? (self-hosted)

Dimension, the integration server? not quite seeing why you mention it's supported, am I missing something?

[anoadragon453]

@BloodyIron It should work regardless of screen sharing mode enabled or not.

So Riot has Jitsi baked in now? (next build release?) so it won't need to load the "widget" when starting/stopping chat, or what? How do you tell it what Jitsi user to use? (self-hosted)

Mmm, no not really baked in yet, although @Half-Shot has expressed some interest in working on this, and it would be fairly trivial to actually get some of the Riot UI to react to what's happening in the Jitsi call. For the moment the widget is still required though.

How do you tell it what Jitsi user to use?

You mean Jitsi server? (Username can just be set by double clicking on your square in the top right) For the server, go into Riot's config.json and you can set the Jitsi URL using the integrations_jitsi_widget_url key. The default is https://scalar.vector.im/api/widgets/jitsi.html.

Dimension, the integration server? not quite seeing why you mention it's supported, am I missing something?

Some people use Dimension as it includes some more features than Scalar. So for those people, PTT will also work :)

[BloodyIron]

Hah yeah I meant to put Jitsi Server, not User. I just wasn't sure if these changes meant Jitsi server declaration changed in some way, as you mentioned [not needing] an integration server by default?

I haven't quite gotten to the point of identifying which integration server I'll use, but I suspect it may be Dimension. Haven't vetted. Good to know this isn't a constraint! :D

Any idea how soon before this hard work of yours could hit public release? :)

[anoadragon453]

So while the PR for this has been out for a while, getting it merged has been blocked on getting a Windows VM up and running and being able to automate builds through it. While something that can likely be done in a few days, the team's time is pretty strapped right now and would rather be spending what time we have on Riot's redesign than on wrangling Window's many awful command line environments.

That said, we have made progress on this this week. We've set up a stripped-down Windows running headless on our build server. We've got an ssh server on it and have it hooked up to our Jenkins infrastructure. At the moment the only blocker has been getting a suitable bash environment for running Riot's setup scripts or just sucking it up and converting them to batch. (We're using git-bash (MINGW32) to execute things, but running stuff in there triggered from a native Windows cmd or Powershell doesn't really work too well, any advice on a less hacky solution would be appreciated).

So that's our current progress - tantalizingly close, and just working on it where we have a spare 30m or so. So if anyone does any advice for getting a proper bash shell that can be ssh'd into and doesn't involve Windows Subsystem Linux, that'd be highly appreciated!

Otherwise we'll probably get it all working in the next few days anyways :) Watch this space!

[JoshStrobl]

@anoadragon453 That's fantastic to hear and thanks for the update. Really looking forward to being able to test out PTT :+1:

[BloodyIron]

@anoadragon453 cygwin?

[anoadragon453]

We ended up just throwing it on a faster machine and installed WSL.

But now we're hitting this bug and builds are timing out :/

[anoadragon453]

Build time outs fixed by reducing VirtualBox CPUs to 1 which still complete builds in a reasonable amount of time.

Slowly but surely!

[BloodyIron]

@anoadragon453 how on earth does limiting the CPU to 1 solve build timeouts? That's weird man! :O

But glad to hear progress! :DDD

[anoadragon453]

@anoadragon453 how on earth does limiting the CPU to 1 solve build timeouts? That's weird man! :O

windozzzzz

[anoadragon453]

Huzzah! Hot on the heels of our FOSDEM stuff we finally got Windows to play nice! We only hit numerous npm bugs (switched to yarn) and a bug in the ecosystem but all is good now and we have working Windows builds from Windows!

At this point I just need to update and rebase my PTT PR and then, pending review, you should expect to see it in the client shortly!

[anoadragon453]

Hey, thanks for asking. Currently progress is blocked on https://github.com/wilix-team/iohook/issues/151, aka getting the underlying native node module working on Windows with Electron v4. Everything is good to go on the MacOS/Linux side.

I've tried a couple times to get to the bottom of it but to no avail (C++ isn't my main language). If anyone is familiar with C++ and its build chain if they'd like to take a shot at fixing that it would be highly appreciated!

[BloodyIron]

Any chance the MacOS/Linux users can get this first? ;D

[anoadragon453]

Was thinking about it but last thing we want to do is split our codebase on a platform-by-platform basis :)

[alien2003]

Seems like it is fixed

[anoadragon453]

@alien2003 Indeed thanks, though I'm getting another bug on our Windows dev box. Going to remake the folder and see how it does.

[alien2003]

Hype... Finally we will ditch Discord

[BloodyIron]

Pssst any news on this? ;)))

[anoadragon453]

Well I spent about 6 hours on this just now and have hit a brick wall on compiling iohook on Windows again. Running yarn run build aka cmake-js spits out There is no Visual C++ compiler installed. Install Visual C++ Build Toolset or Visual Studio. It does seem to be 100% installed though, and the message persists even after reboots.

If anyone can figure out how to build iohook on Windows atm please let me know. I've left build instructions here: https://github.com/anoadragon453/riot-web/blob/anoa/jitsi_ptt/docs/native_node_modules.md#compiling-iohook

Here are my notes for the current session for installing from a fresh Windows install:

Turn off windows defender and firewall etc
Install chocolately: get npm, yarn, git, python2, visual C++ Build Tools 2017
Open Powershell
Clone iohook
yarn
yarn run build
Error :(

Now, even if we do get it building we'll likely run into this showstopper: https://github.com/wilix-team/iohook/issues/167

I think tomorrow I'll just change the PRs to not do PTT if on Windows because it's been a blocker for far too long and everytime something on Windows is fixed something else comes up again :')

[BloodyIron]

@anoadragon453 maybe your PATH value was missing something?

[skylord123]

@anoadragon453 definitely sounds like a path issue of some sort.

Check out "Setting up your command-line environment" under this link:
https://devblogs.microsoft.com/cppblog/finding-the-visual-c-compiler-tools-in-visual-studio-2017/

I use windows the most (sadly) and so I am a bit sad to hear about the showstoppers. I'm fine with forgetting about Windows for now as long as it isn't put off for a very long time (of course that isn't entirely in our control if the upstream issues aren't resolved).

Thanks for all the time you have spent on this. I definitely appreciate it :)

[BloodyIron]

I think it's in everyone's best interests if we get this feature sorted out for everyone. If not for this feature, at least for streamlining the build process for other things too.

[W4rH0rse]

PTT is absolutely critical to alot of gamers who want to move away from Discord, and most of them are on Windows. I'd love to use Riot as a replacement for Discord, but until PTT is supported I cant.

[TaylorEverlund]

PTT is absolutely critical to alot of gamers who want to move away from Discord, and most of them are on Windows. I'd love to use Riot as a replacement for Discord, but until PTT is supported I cant.

Ditto, I have friends who've made the move from TS to Discord and now want to move to riot but PTT is holding them back. We've programmed many of mouth breathing food chewers to use PTT and without it would be a nightmare. I wish I could help in some way with the coding on this but with so many other side projects I just don't have the time. I wish y'all the best of luck and for a speedy resolution.

[ImGunter]

Any updates on where we are with this feature?

[anoadragon453]

It's currently been blocked behind other tasks, but as Matrix 1.0 has just released and all the work that has gone with that, I intend to work on this feature very soon.

[ImGunter]

Yay that's great to hear i can't wait for that feature to come out please keep us updated and thank you for your great work!

[anoadragon453]

Thanks, I will :)

[alien2003]

Not directly related but seems like a good addition to Jitsi PTT feature
https://github.com/Johni0702/mumble-web

[ImGunter]

Now that we have reactions and editable messages is push to talk next on the road map?

[anoadragon453]

Not directly related but seems like a good addition to Jitsi PTT feature
https://github.com/Johni0702/mumble-web

Oh, that already has instructions for being an integration manager widget, interesting!

As long as it has support for HTML5 cross-document messaging so that we can give it commands from the web interface it should be quite easy to integrate.

[anoadragon453]

Now that we have reactions and editable messages is push to talk next on the road map?

It's getting close :)

[BloodyIron]

I'm switching to riot AWAY from Mumble :( Reliable tech, but their devs have dragged their heels (and effectively halted) for well over 7 years now.

[ImGunter]

Same bloody mumble is such a great program with great potential but the devs don’t seem to have the time anymore.

[alien2003]

Oh, that already has instructions for being an integration manager widget, interesting!

Yes. I tried it for my Minecraft server room, works well but it's not persistent widget by default. The only way to make it persistent is to set it to "Jitsi" type and it breaks widget in mobile apps

[BloodyIron]

Any word on this?

[anoadragon453]

Sort of blocked behind not working on Windows. I don't think we want to release a feature that only supports a subset of our supported platforms. Windows may work now though, I haven't checked in a few months :thinking:

[BloodyIron]

@anoadragon453 is it the supporting library is limited on Windows, or what? I know this is going to be a FR that won't go away, and I'm not talking about my stubbornness, I'm talking about everyone else's too XD

[anoadragon453]

FR?

It's good ol' iohook.

[BloodyIron]

Feature Request.

[anoadragon453]

Feature Request.

ahh

[remram44]

Please be considerate, there are dozens of people watching this issue and this repository to be notified of progress, and you're spamming all of them every time you add an unrelated comment 😉 If you push the devs to unsubscribe from this issue you'll never get your feature...

[BloodyIron]

The last update was Jul 22nd, I would not call this "spam"... -.-

[skylord123]

@remram44 It's not spam. It was a valid question and answer and wasn't off topic. People care about this issue and want to continually know what is holding it up and check if the issue has been resolved.

I thought the question and answer was helpful as someone subscribed to this issue.

edit: also the dev working on this issue was one of the people you said was "spamming" so I don't think we need to worry about other devs unsubscribing and not solving the issue.

[Mhowser]

Any update to this?

[anoadragon453]

Any update to this?

We still don't have the build infrastructure to produce Windows builds on a Windows box, but I imagine this will change very shortly as we have a number of native node modules coming to Riot Desktop (namely enabling Push-To-Talk and searching in encrypted rooms).

So be on the lookup for that (very hopefully) soon!

Edit: In the meantime I'll update my PR to latest riot develop.

[anoadragon453]

Some progress! https://github.com/vector-im/riot-web/pull/10369#issuecomment-549865992

We want to get it into the codebase so the PRs stop bitrotting, but people who want to enable it will still need to insert their own native node module in.

That said, things are still moving on upgrading the build infra. Integrating seshat into Riot Desktop for instance has made the priority for native node module support even higher.

[alien2003]

@anoadragon453 That's better than nothing but still not enough to make average Discord users move to Riot :( It will be hard to convince Discord users to install Riot if there is no ready-to-use build with PTT stuff included

[anoadragon453]

@anoadragon453 That's better than nothing but still not enough to make average Discord users move to Riot :( It will be hard to convince Discord users to install Riot if there is no ready-to-use build with PTT stuff included

Of course. This is just a stopgap solution to stop the code from bitrotting. Once the build infrastructure is up then all users will have it by default.

[BloodyIron]

So, it's coming up to nearly two years for this feature request. And my understanding Windows is the last mile.

Can we _please_ get this finalised and pushed into mainline before year's end? (2019)

[Mhowser]

Is there a way we could build a client with this PR to test it?

[samjco]

PTT rooms, please!! This would be greatly appreciated.

[samjco]

Just curious will any of these libraries work?

https://github.com/mnovicio/ptt-freeswitch-ui
https://github.com/Merve40/ptt
https://github.com/QVDev/GunPushToTalk
https://github.com/Theofilos-Chamalis/QR-PTT-PushToTalk

Can the mumble server audio server be tied in somehow and integrated into Riot?
https://www.mumble.info/

So here is my dirty hack.. Install mumble-server.
Hack the mumble web client here: https://github.com/Johni0702/mumble-web
which attaches to mumble. Use iframes with Riot that uses the modified mumber web client.
As a user is added to Riot, auto add/register them to the mumble server. When an audio room is made in Riot, add a room in mumble and point the iframe to it. Yeah, this hack is As dirty as it gets :)

[samjco]

Wait! did I just see this towards the bottom!!!

https://github.com/Theofilos-Chamalis/mumble-web

More notable web themes for mumble:
https://github.com/xPoke/MetroMumble/releases

[samjco]

Widget for matrix
https://git.aventer.biz/AVENTER/js-matrix-mumble-web/src/commit/55fb04eb92bc7e934eba99ed381fd714bff62d0e

For iOS:
https://github.com/mumble-voip/mumble-iphoneos

For OSX desktop kit
https://github.com/mumble-voip/mumblekit

https://medium.com/@RiotChat/riot-im-web-0-12-7c4ea84b180a
Honestly, *if Riot has Jitsi integrated, why not just use the video feature, but cut out the video and make the Push-to-talk button off of the mute button?*
Like: https://jitsi.org/news/audio-only-mode-in-jitsi-meet/
and here: https://community.jitsi.org/t/audio-only-mode/21359/2

[anoadragon453]

Honestly, if Riot has Jitsi integrated, why not just use the video feature, but cut out the video and make the Push-to-talk button off of the mute button?

That's what this PR does :)

But it's bitrotted and I haven't found time to update it :(

But hopefully sometime soon.

[samjco]

@anoadragon453 Did you see this?
Like: https://jitsi.org/news/audio-only-mode-in-jitsi-meet/
and here: https://community.jitsi.org/t/audio-only-mode/21359/2

[anoadragon453]

@samjco I'm aware of this mode. To be clear, the scope of this PR is just to have a global shortcut to toggle your microphone in a jitsi call, regardless of whether Riot is in focus, as well as tools to allow you to modify that shortcut. There's no audio-only UI here (yet), though this probably would unlock that work.

[Per0x]

Honestly, *if Riot has Jitsi integrated, why not just use the video feature, but cut out the video and make the Push-to-talk button off of the mute button?*

I don't recommend jitsi for this type of use. Beyond 3 users, Jitsi doesn't work very well, whereas a voice chat should work well with more than 10 people.

[samjco]

@Per0x Jitsi itself has an audio-only-mode:
https://jitsi.org/news/audio-only-mode-in-jitsi-meet/

We make a dummy meeting (in background) in a Riot #room based on unexposed domain and random characters and Just hide the UI. Anyone accessing the room would be auto-logged in.

As for private rooms, the domain + random is exposed.

Just thinking out loud here.

[BloodyIron]

What's the max # of users Riot devs have tested in a single Jitsi meeting? What's the upper limit?

[samjco]

@BloodyIron depends on the server. some say 10 other 30-40
https://community.jitsi.org/t/jitsi-meet-max-call-connect-on-one-conference/20519

[BloodyIron]

I'm more meaning hypothetically where computing/networking resources are not the constraint. At what point would Jitsi crawl to a halt?

[turt2live]

What's the max # of users Riot devs have tested in a single Jitsi meeting? What's the upper limit?

We regularly have ~20-24 people on a single call with no major issues. Earlier this week we got about 20-30 people onto a single call to load test it (though this time the machines trying to send the video had the problem, not the server to my knowledge).

[BloodyIron]

What's the max # of users Riot devs have tested in a single Jitsi meeting? What's the upper limit?

We regularly have ~20-24 people on a single call with no major issues. Earlier this week we got about 20-30 people onto a single call to load test it (though this time the machines trying to send the video had the problem, not the server to my knowledge).

How much bandwidth did the 20-30 person call use inbound and outbound? And was that video or audio-only?

[turt2live]

It was video and audio. I don't have the server's numbers, but from memory one of the machines responsible for trying to push ~6 streams was hitting about 20mbps upload and about 18mbps download (I know this because it was my machine :p)

[BloodyIron]

It was video and audio. I don't have the server's numbers, but from memory one of the machines responsible for trying to push ~6 streams was hitting about 20mbps upload and about 18mbps download (I know this because it was my machine :p)

6 streams? of webcams or what? How does 20-30 people turn into 6 streams? I'm not quite following you. Trying to plan capacity here.

Sounds like 3mbps per "stream"? (whatever each stream is, I don't yet know from what you are referring here).

[turt2live]

The load test wasn't 20-30 physical people. It was 20-30 video streams, mostly webcams. Mine just happens to try and send 1080p video over the internet, but Jitsi will downscale as needed for bandwidth iirc.

If you're planning for load, it's generally going to be best to ask the Jitsi crew specifically and not us. We're a very specific use case and might use Jitsi differently than you (mostly because we're a giant node for everyone who uses the default settings). The server isn't that large though from what I remember.

[Mathias-g]

Any news on the push to talk feature? We are thinking about implementing mumble when the MSC for room types gets implemented, and make a "mumble room type", but if push to talk is getting implemented in jitsi meet and the 1 to 1 calls then we won't have to.

(As an aside, it would also be nice to have webcam disabled by default)

[fti7]

Mumble Integration would be very nice :-) i think it would be the superior alternative for the "Voice Channels" Feature.
PTT also needs good Voice Activation etc. to be a real alternative.
This Issue is 2 Years old... dont think they will ever do it ;-)

[anoadragon453]

Heh, I was just saying today that I should get back to this. But a lot of the code has bit-rotted, so it'll take some work.

Most of the effort in this feature was getting the native node module for listening to global keystrokes working with the js package for site-scoped keystrokes. That will be useful for jitsi or any other voice activation software.

Tbh, jitsi eats all the resources on my PC so I'm happy to have mumble as a potential alternative for voice channels :)

[BloodyIron]

Still a feature desired there @anoadragon453 ;) I just figured I'd stop bugging you about it... for a little bit ;P

I for one am switching away from mumble, and would prefer that PTT not be tied to it, and that any sort of Mumble interfacing be created as another request, as this is originally about PTT.

[Mathias-g]

Can you link to the PR? @anoadragon453

If you don't mind it would be great to chat a bit about it with you so we can help out pushing it across the finishing line.

[anoadragon453]

@Mathias-g

There's two PRs against two separate repositories:

• https://github.com/vector-im/riot-web/pull/10369
• https://github.com/matrix-org/matrix-react-sdk/pull/2280

I believe pretty much everything from the riot-web PR (which mostly dealt with Electron stuff) needs to now be ported to the Riot Desktop repo: https://github.com/vector-im/riot-

However, the biggest hurdle with this PR was getting iohook, the native node module used to listen to keystrokes system-wide, to build on Windows. However, https://www.electronjs.org/docs/api/global-shortcut, while insufficient when the PR was initially created, now naively looks like it could suit our uscase. And if so, it makes the implementation much, much simpler.

Edit: Hrm, actually looking back at the edit history of global shortcut, the only new method if registerAll, which isn't particularly useful here, so there may've been another reason why it couldn't be used :thinking:

Edit 2: Ah yeah, globalShortcut doesn't tell you when you've released the keyboard shortcut, i.e when to mute the mic again.

[BloodyIron]

keep at it @anoadragon453 ! :D your work is appreciated!

[BloodyIron]

Can we please get this issue tied to a milestone so it doesn't keep getting lost? :(

[t3chguy]

That wouldn't mean anything, the team doesn't use github milestones due to them being very limited.

[Mhowser]

How about pinning this issue?

[BloodyIron]

Would pinning help? I just really want this put in, and the majority of the work is already done, isn't it? :/

[anoadragon453]

the majority of the work is already done, isn't it?

The code has bitrotted so it'll need some freshening up. It was also forked off the riot-web repo, but should not go against element-desktop.

The build process for iohook will also need to be redesigned to align with the native node module stuff that has been paved by the seshat integration.

So it's not a quick job, probably a week or so worth of work, but not impossible. The concepts should still hold.

[samjco]

So there is webrtc https://webrtc.github.io/samples/

Or something simple would to just add Pragli
Here an example of it being using in slack. https://pragli.com/blog/how-to-use-pragli-with-slack/

[t3chguy]

Right, Element (Riot) and Jitsi both use WebRTC.

[BloodyIron]

Any word on this? :/

[anoadragon453]

Not really, just sitting in my backlog unfortunately.