Ferdi: Do not hibernate services during calls

[kris7t]

Is your feature request related to a problem? Please describe.
Last time I had a call in Microsoft Teams, which I set to hibernate automatically, because I don't use it that often, mid-call the service was hibernated (I activated another service to reference at some other messages during the call).

Describe the solution you'd like
Ferdi should detect ongoing calls (perhaps display this as a badge on the service icon) and forego service hibernation in the meanwhile. This could also be a basis of push-to-talk support (#720): when there is a call in a service, global push-to-talk event should be directed to it.

On the implementation side, this would need:

1. Changes to ServiceModel to store ongoing call state (and microphone mute state) transiently (there is no need to save this to the Ferdi server, as call don't persist over app restarts).
2. An API for recipes to tell Ferdi about ongoing calls, and optionally receive push-to-talk events.
3. Update the hibernation logic to take calls into account.
4. Update the service icons view to display a badge on the service with an ongoing call, to show that the call was successfully detected.
5. Support in the recipes to detect ongoing calls, and optionally effect push-to-talk (either directly, or by generating click events on the mute/unmute button).

The current week (and maybe the next) is a bit busy for me, but sometime later I could get coding on this, if you like the design.

Describe alternatives you've considered
Currently, I disable Teams instead of setting it to hibernate when I don't need it. Thus, it only gets unloaded when I manually set it so. However, this is not ideal, as I have to remember to re-disable it when the calls are over, and I can't just quickly click on it to see the latest updates (and then let it be automatically hibernated again).

[mahadevans87]

Sure that sounds pretty reasonable. Please go ahead! You have addressed 2 issues here -

1. Prevent hibernation of services during an on going call - I feel the recipes should notify the Ferdi app in someway or the other about the states they wish to communicate - Now there could be a lot of different states a recipe might want to notify Ferdi (not just 'ongoing-call' for e.g). So please try to generalize your implementation where a recipe could notify Ferdi of any important state that it wishes to update.

1. I see the PTT feature as the reverse of the 1st feature where Ferdi might want to send a particular recipe an event from its process. This could be 'push-to-talk' or 'mute' (Let's say that I want a universal mute button) or something else. It should be upto the individual recipe to handle each state that Ferdi emits and take actions accordingly.

Please let me know if that makes sense.

[vantezzen]

@mahadevans87 Sounds like a great idea, we'd need to add a custom method to https://github.com/getferdi/ferdi/blob/develop/src/webview/lib/RecipeWebview.js similar to the "setBadge" method.

In recipes this could look something like:
`JavaScript // webview.js module.exports = Ferdi => { someEventHandlingMethod.on('call-start', () => { Ferdi.setAllowHibernation(false); }); someEventHandlingMethod.on('call-end', () => { Ferdi.setAllowHibernation(true); }); };

[kris7t]

@mahadevans87 Cool ideas! Both 1 and 2 sounds very reasonable and useful.

I see two ways to make this configurable: either Ferdi has some built-in list of state informations (let's call these _flags_, for instance) that recipes may need to communicate, as well as a built-in list of events to send to services (let's call these, I don't know, _commands_), or these could be defined by recipes. I'm leaning more toward the build-in approach, because then we can have proper display (e.g., service icon badges) for flags and proper localizations for flag and command names.

Another area where having a flag might be useful, besides class, would be file uploads and other long-running operations. They Ferdi could show that such an operation is in progress, avoid interrupting it by hibernation, and maybe even display the current progress %.

Another area for commands would be for taking/rejecting a call, or jumping to the next unread message among a view of many conversations.

So, on the recipe API side, we'd need a way for recipes to

• set flags, possibly with other data (e.g., Ferdi.setFlag('in-call', {muted: false})),
• unset them (e.g., Ferdi.unsetFlag('in-call'))
• declare what command they may be able to receive (e.g., Ferdi.enableCommand('push-to-talk'), Ferdi.disableCommand('push-to-talk'))
• act on commands (e.g., Ferdi.onCommand('push-to-talk', (payload) => { /* ... */ }))

Then the ServiceModel can expose the currently set flags, and the currently enabled commands. So the hibernation status of the service can depend on the (non-)existence of certain flags, and when the user instructs Ferdi to send a global command (e.g., press the global push-to-talk key), it can be delivered to the service that has declared it wants to act on the command. (Local hotkeys, that interact with the active service, are also a possibility, but less interesting, since the web applications running inside Ferdi can already implement them on their own.)

On the UI side, we need at minimum a way to issue commands (e.g., right-click menu on service icon, key binding) and display active flags (e.g., service icon badge, tooltip text). Customizations, such as setting which flags should disable hibernation, or which global hotkey corresponds to which command, are conceivable, but probably not required for a minimum working example.

This starts seeming like a moderately big task (especially because my React skills are rusty, and my Mobx and Electron skills are yet to be acquired), but it also sounds fun. 🤞

Also, better names for flags and commands are sorely wanted. 😄

[vantezzen]

The idea with flags and commands sounds good but I am unsure if these will really ever be used in reality:
Looking at our recipe PRs, in my experience 90% of those contributing recipes are first-time contributors who only have minimum of knowledge about how recipes work. Currently, this is ok because we try to make creating recipes as easy as possible with custom scripts and step-by-step guides.

Those contributors would however probably eather skip implementing flags as they don't want to invest the time into learning when to set which flags and how commands work (making it useless to have such a powerful concept implemented in the first place) or they will stop creating recipes as they find it too complicated (which is also not something we want).

We'll just have to decide if the complexity and time-intesity of implementing your concept is worth it with the knowledge that only a handful of recipes will ever add support for it.

Instead implementing standalone methods like "setPreventHibernation" or something like "handlePushToTalk" would restrict what can be done but would also be a lot easier to learn for contributors as they would only need to learn how to call a single function properly instead of having to learn the whole concept of flags and when and how to set them.

[kris7t]

@vantezzen Yeah, fair point, recipe complexity should not be disregarded.

I still don't like setPreventHibernation, as it is not very semantic, I'd prefer setInCall instead, then (so Ferdi can show an "in-call" badge, instead of a .

Maybe a middle ground between @mahadevans87's fully generic idea (and my elaboration with flags/commands thereof): have a generic way of communicating between Ferdi and recipes (something slightly higher level than Electron IPC), and have stuff like setInCall / setPreventHibernation and handlePushToTalk as even higher-level wrappers on top of that. So we get the benefits of a robust communication mechanism, but also of a comfortable recipe API.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Please check if this issue is still relevant and please close it if it's not. This will make sure that our open issues are actually of use and reduce the list of obsolete issues. Thank you for your contributions.