Fenix: [Bug] Fenix hijack the audio output of other apps on the device

[karlcow]

Steps to reproduce

1. Starts an app on the device with an audio output (podcast player for example)
2. Go to a site with videos
3. Play a video

Expected behavior

Both sounds can be heard.

Actual behavior

The sound of the app is muted.

Device information

Browser / Version: Firefox Fenix (Preview) 1.4.1
Operating System: OnePlus 6 (Android 9) - 1080 x 2280 pixels, 19:9 ratio (~402 ppi pixel density)

and

https://github.com/webcompat/web-bugs/issues/40062
https://github.com/webcompat/web-bugs/issues/40119

[colintheshots]

I'm pretty sure this is the desired audio focus behavior. It's also what Firefox Nightly and Chrome both do.

[yaomtc]

The main issue I see is with sites like Twitter, which starts playing (muted) videos as you scroll down the feed. While these videos are muted, there is still a media notification that not only results in stopping music playback, but also adds a large notification at the top with media controls before I've even touched the video.

[mars-f]

I observed this problem when scrolling down a page with a video ad that played on a 10 second loop. The ad played silently but my podcast player audio would get killed. If I restarted the podcast audio it would get killed again in 10 seconds. Viewing the same page in Chrome did not interrupt the player audio.

[everett1992]

I'm pretty sure this is the desired audio focus behavior. It's also what Firefox Nightly and Chrome both do.

Chrome does not do this, and it's one of the first things I noticed when switching to Firefox (preview and mobile both have this behavior). Chrome will not request audio focus if the media is muted, you can play a muted video without hijacking audio focus. When you unmute the video then chrome will request focus.

Here is an example video, I can repro on firefox and preview but not chrome: https://www.reddit.com/r/aww/comments/f53esr/we_adopted_a_senior_doggo_and_he_loves_sleeping/

I think this code explains the behavior. The media service doesn't check if the media has audio or is muted before requesting audio focus. I checked MediaState.Playing and it's List<Media> but neither track if the media is muted, so I don't see a quick fix.

[BranescuMihai]

Thanks @everett1992, I looked over and you're right, we don't seem to track the muted state other than having a setter in the GeckoMediaController.
I think we should expose a volume or isMuted field in the MediaElement Metadata, pass it along to GeckoMedia Metadata, and then do a check in the piece of code you pointed out.

I've filled a Bugzilla bug about this.

[sblatz]

I'll keep an eye on this. Thank you for your investigation @BranescuMihai!

Discussed during AC triage, @pocmo said this GV fix would be very helpful in solving this 😄

[owlishDeveloper]

Hi! 👋 FYI - we were looking at the Bugzilla bug mentioned above, and it looks like the volume is already supported in the API. You will see the conversation about that in the bug. I closed the bug as Resolved-Incomplete for now, but feel free to re-open and add more information, or maybe you'll decide to follow some other path in solving this issue - anyways, just letting you know!

[everett1992]

I'll do some investigating tomorrow, but the linked apis onVolumeChange and
setVolume may not be applicable. We need to know if a media element has
volume when the element is played or paused. We need to know the state, not
the state change.

On Thu, Mar 26, 2020, 10:14 PM owlishDeveloper [e-mail]
wrote:

Hi! 👋 FYI - we were looking at the Bugzilla bug mentioned above
https://bugzilla.mozilla.org/show_bug.cgi?id=1616249, and it looks like
the volume is already supported in the API. You will see the conversation
about that in the bug. I closed the bug as Resolved-Incomplete for now,
but feel free to re-open and add more information, or maybe you'll decide
to follow some other path in solving this issue - anyways, just letting you
know!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mozilla-mobile/fenix/issues/6146#issuecomment-604819277,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE6WMZE4UL6MXXKP4VAQ3LRJQY47ANCNFSM4JCW3EVA
.

[everett1992]

I took a crack at a pull request https://github.com/mozilla-mobile/android-components/pull/6432

[sblatz]

@pocmo, @vesta0 what should be the behavior here? Sounds like we do have a volume API for this, but how should the notification be handled when there's no audio playing?

[kbrosnan]

Part of the concern here is that the notification has purposes besides the primary ability to control media playback. It provides a way to alert the user that the website may be consuming data. It allows the app to stay active while consuming media.

[everett1992]

Part of the concern here is that the notification has purposes besides the primary ability to control media playback. It provides a way to alert the user that the website may be consuming data. It allows the app to stay active while consuming media.

I'm happy to refactor the pull request so it doesn't affect the notification behavior.

However, my impression was that the notification is created to _allow_ the app to consume data in the background. Won't the OS restrict data use background apps?

If the media cannot be heard (muted) and cannot be seen (in the background) do we want it to consume data?

[sblatz]

Sounds like Sebastian is on top of reviewing this PR and will make the call here on the notification visibility 😄

Thanks for your work on this @everett1992

[PandoraDay]

The main issue I see is with sites like Twitter, which starts playing (muted) videos as you scroll down the feed. While these videos are muted, there is still a media notification that not only results in stopping music playback, but also adds a large notification at the top with media controls before I've even touched the video.

It happens the same to me. This Audio Notification can't be hide once the video has been played, not even changing the current tab or opening another application. The way I do to get rid of that notification is closing Firefox Preview for good and do a clean boot of the application, avoiding to open a tab with audio.

Unrooted Nexus 6P with Android 8.1.0 and Firefox Preview 4.2.0

[pocmo]

We landed a patch in Android Components and this change will soon show up in Fenix Nightly (Note that we are currently having issues with Nightly updates through Google Play, so it may take a bit longer until it is available).

With this change we will:

• No longer start the media service for muted media. This means that we are not showing a notification and also not requesting audio focus (which means we are not stopping audio of other apps). In addition to that we will not keep the app from getting destroyed.
• Once media becomes unmuted we will start to show the service and do everything above. We will continue to do so if the user mutes the media manually after that.

So in a nutshell this means you can scroll though a Twitter feed with muted media and we will not show a notification or pause other apps until you explicitly unmute a media element in that feed.

[AndiAJ]

Hi @pocmo , I've just checked this on my Nightly debug Build #21040820 39.0.20200410130104, 464e3038d GV: 77.0a1-20200409095500 AS: 0.55.0 from Monday 4/13 using a Google Pixel 3a (Android 10) on the following web sites:

✔️ Corsolinux & Codepen
► Video

❌ NBC
• After the user swipes a bit down, a media notification gets displayed even if the video doesn't have any sound, Soundcloud app playback is paused.

► Video

❓ GQ
• After the page loads a media notification gets displayed. (Soundcloud is paused)
• If you swipe down until you reach the featured video and wait for it to load the previously displayed notification disappears.
• If the video is unmuted, the video playback notification is displayed again.
• If you play the song from the Soundcloud app (the GQ video will be paused) and afterwards mute the video and resume, no notification will be displayed.

► Video

✔️ Twitter
• No notification is displayed while the muted videos play (The Soundcloud app continues the playback)
• After the videos are unmuted the notification gets displayed and Soundcloud is paused.
• If you pause the Twitter video, mute it and play it again no notification is displayed.
• If you swipe away from the unmuted video the notification will disappear.

► Video

Please review and share your thoughts. ☺️
I'll remove the QA needed label until further notice.

[pocmo]

Looking at nbcnews I see that the media is not muted (even though there's nothing you can hear), which causes us to show the media notification.

[brampitoyo]

@AndiAJ Yes. You have the right idea. Our audio notification should behave identically to the mute control on Firefox desktop tabs.

If the video autoplays, first check whether it’s muted or not. If it’s muted, no need for notification. If it’s not muted, then show the notification.

If the video plays on manual user trigger, the behaviour is just the same as above. Only show notification if the video is not muted.

[AmyYLee]

@brampitoyo is UX feedback still needed here? If not, feel free to remove UX-feedback tag.

[bunnybooboo]

Related to this bug? https://support.mozilla.org/en-US/questions/1277926