Xamarin.forms: [F100] MediaPlayer View

[hartez]

Rationale

Forms does not currently have a simple cross-platform way to play audio and video without creating a custom renderer.

Implementation

This will probably require a MediaSource class similar to the ImageSource class, with the appropriate subclasses (FileMediaSource, StreamMediaSource, UriMediaSource and their respective type converters).

The actual MediaPlayerView should have a bindable property Source of MediaSource. The renderers for each platform should use native media players.

The MediaPlayerView should have a boolean property ControlsVisible; when set to false, the transport/playback controls of the native player should be hidden.

The view should have a property PlaybackState with possible enum values Unknown, Paused, and Playing.

The view should expose events for
PlayRequested
PauseRequested
PlaybackStateChanged

Renderers should respond to PlayRequested by attempting to play the source media in the native player. They should respond to PauseRequested by attempting to pause the media. Changes in native playback state should raise PlaybackStateChanged in the MediaPlayerView.

The MediaPlayerView should include SendPlay() and SendPause() methods for raising the PlayRequested and PauseRequested events.

The initial version of this control will not support background playback for audio.

Backward Compatibility

This is a brand new View, so there should not be any backward compatibility issues.

Difficulty: Very

In addition to playing audio/video, this control also needs to gracefully support incoming call interruption, system-wide audio requests, volume controls (software and hardware), and keeping the screen active during playback. All of these interactions are different on each platform.

[adrianknight89]

Android's stock video view isn't great, especially on older APIs and devices. I'm not sure if it's a good idea for Xamarin to create a cross-platform video player unless you're willing to put some serious effort into making a professional-grade control. I believe Google is now using ExoPlayer for YouTube and possibly other apps as well.

https://www.youtube.com/watch?v=6VjF638VObA
https://google.github.io/ExoPlayer/

If you decide to go forward with a video view, can you leverage ExoPlayer for Android? Also, @martijn00 and others have contributed to ExoPlayerXamarin and XamarinMediaManager which can be utilized instead.

[adamped]

Would also like your input here @martijn00 :) Is it wise to make this part of the XF framework? Because I would be leaning to a no on this one. It's incredibly complex, and are you sure the team is up to committing to this level of work/maintenance?

[jassmith]

I agree that there is complexity here and the urgency to implement this is minimal. If such a thing were to be designed properly and done, fine. It is rather unlikely we will implement this ourselves.

[andreinitescu]

A video and audio playback control is a must.
My suggestion is to have something like how Windows XAML has, a MediaElement control
See https://msdn.microsoft.com/en-us/library/system.windows.controls.mediaelement(v=vs.110).aspx

It doesn't have any UI beside actual video rendering. No volume controls, no seek bar, nada.
It does have methods like Play() Pause() Stop() and some events to notify when playback changed.

We need to define exactly what to support.
Some quick search reveals people already done this by their own: https://github.com/inthehand/InTheHand.Forms

[hartez]

The UWP media element _does_ have UI other than the video. You can hide the transport controls, but they are available.

[andreinitescu]

@hartez Correct. I was referring to older XAML platforms (WPF, WindowsPhone, Silverlight). I guess they added it in UWP because of the unified platform idea which means unified look and feel too

[hartez]

So here's _my_ thinking, as of right now: if vanilla, out-of-the-box XF supports a media player element, it will be a basic one which utilizes the built-in native media players. Like, very, very basic. For advanced scenarios, bells-and-whistles, etc, developers will need to look at plugins (like the ones Adrian mentioned upthread) or write custom renderers.

That's the scenario I'm currently targeting.

[andreinitescu]

@hartez I think that's what I said too.
Can you please update the description of the ticket to reflect what you said?
I suggest to just have a clear list of events, properties, and methods.
I think the things you mentioned in Difficulty: Very are not needed, nor the idea of using ExoPlayer.

[hartez]

The stuff under "Difficulty" is still relevant; at the very least, the control has to play nice with system audio events. For example, when a call comes in, the native player needs to respond by ducking or pausing; that change in state has to be reported up to the Forms control.

[charlesroddie]

Is this thread a request to make https://github.com/martijn00/XamarinMediaManager official, or to copy over code from that project, or to write a new forms media player from scratch?

[adrianknight89]

I agree with @andreinitescu that we should have a cross-platform media player (whose playback controls can be hidden when needed). Whether it will be basic or advanced is another issue, and on a personal level, I don't care if this control is part of XF or a third-party plugin.

It looks like @martijn00 doesn't have the time to commit to working on XamarinMediaManager which has NOT seen a new release in 7 months.

My concern with a basic control is that it becomes yet another impediment among others for XF developers to create a serious production app. Video is quite hot on mobile now, and things like live streaming are becoming prevalent across many apps. My recommendation for the team would be hiring Martin to work on XMM. See here.

@davidortinau Would love to hear your opinion.

[andreinitescu]

@adrianknight89 In the first version of this control, I'd not put any playback controls at all, just how WPF and Silverlight works. My suggestion is to align with UWP MediaElement:

Events: MediaOpened, MediaEnded, MediaFailed, CurrentStateChanged, BufferingProgressChanged
Properties: Uri Source, TimeSpan Position, double Volume
Methods: Play(), Pause()

This will be enough to enable cross platform media playback. Adding your own playback controls will be easy, and you can give any look and feel you want. In next iterations of this control, add more features including builtin playback controls.

[davidortinau]

@adrianknight89 I'm personally using @martijn00's project. I say we support @martijn00 over there.

I would like to see this in Xamarin.Forms down the road.

As for how basic or advanced, what's your definition there @adrianknight89 ? What 2 or 3 features would you say are "must have" that you are concerned would be cut if we delivered a v1 basic?

[adrianknight89]

I agree with you on supporting @martijn00. XamarinMediaManager could be _the_ video player all of us leverage similar to how FFImageLoading has become quite popular.

What I consider basic is the minimum cross-platform functionality that people will need (e.g. play, pause, stop, repeat/reset, and volume).

For me, [1] an advanced scenario is HTTP Live Streaming (HLS). As far as I know, Android's MediaPlayer does not do this at all or properly (see here) whereas Google's developed ExoPlayer to include this feature among many other benefits:

https://academy.realm.io/posts/360andev-effie-barak-switching-exoplayer-better-video-android/

MediaPlayer has other disadvantages: it is a black box, written natively, impossible to debug or figure out exactly what exceptions are happening. MediaPlayer is also installed locally on your phone, therefore there could be differences in implementation and the way it works on different OS’s and phones. We cannot control or guarantee exactly which version will be used. Many of the states can be buggy, and it is hard to debug them. MediaPlayer sends out weird exceptions codes, and we could not see for certain what crashes are happening.

ExoPlayer solves all the aforementioned problems. It is very extensible, but it has a steeper implementation curve. It is open source, written in Java, easy to debug, and easy to read. It is built on top of MediaCodec, which is more low level, and it handles HLS correctly. It supports background playing, subtitles, variable playback speed, and variable resolutions.

As mentioned above, ExoPlayer has a steep learning curve even for Android developers despite the documentation available today. Xamarin could make it easier for us to use it.

Other (semi) advanced features would be [2] subtitles, annotations, playback speed, and fullscreen (though to some, the last one should be part of basic functionality). Another thing is, [3] portrait only apps should be able to show video on landscape while all other pages on the stack do not trigger SizeChanged events. [4] I'd also like to see the ability to stop playback when earbuds are unplugged and when you get out of car (presumably after bluetooth connection is dropped). [5] Aspect ratio/mode should be handled properly (i.e. let us choose between various modes). [6] Finally, just like Spotify/YouTube does it, there should be a way for us to let users choose output device when there are multiple sources available simultaneously (phone speakers, portable home devices, car speakers, etc.)

For my personal use case, I'd like to see in v1 everything in basic as well as [3], [4], and [5]. The other features can come in later.

EDIT: We should also test the media player to ensure it can be embedded in a ViewCell and recycled properly with minimal performance degradation.

[charlesroddie]

@andreinitescu
Events: MediaOpened, MediaEnded, MediaFailed, CurrentStateChanged, BufferingProgressChanged
Properties: Uri Source, TimeSpan Position, double Volume
Methods: Play(), Pause()

I agree with this list as the essentials, minus CurrentStateChanged and BufferingProgressChanged.

I would also suggest that these should all be instance properties on a media View. (Or an audio object if that is separate.)

MediaManager has had the most development, but has the above as static methods, e.g. CrossMediaManager.Current.Play. InTheHand.Forms is also worth looking at.

[PaulVrugt]

We're using inthehand.forms for media playback. I think the implementation would be perfect for a basic video player in XF. It is a quite basic control, but does exactly what you want for basic playback. It has events, handles aspect ratio correctly. Has an option for showing/hiding default controls and supports custom http headers for a http source, which come in handy if you are streaming video from an api.

[peterfoot]

Hi, I'm the author of InTheHand.Forms and I modelled the MediaElement control on the UWP equivalent. Since then UWP MediaPlayerElement has been released which has a slightly different API. I still think the InTheHand.Forms.MediaPlayer stands up as a decent basic control which wraps the native players. I have looked into (but not started) creating renderers for macOS and WPF as these are now Xamarin Forms targets. I'd be happy to submit the control to Xamarin.Forms.

[davidortinau]

@peterfoot I spoke with @jassmith and we would like to proceed with your proposal. iOS, Android, and UWP are the primary needs. We would prefer to match the UWP API closely.

Let us know if you would like to discuss anything in depth, or just shoot us the PR when you're ready for review. This is great! Thanks.

[davidbuckleyni]

What is the status of this has this now been created in 4 as I need to play an audio file to confirm scanning has happened correctly. Even a system sound of the phone for example beep ?.

[jrahma]

This should have an option:

1. Play Single Sound Only
2. Play Queued Sound Files in sequence
3. Play Video
4. Display Thumbnail for Video files (before start and after finished)
5. Option for Videos to Play in full screen
6. Option to keep playing sound or video even if the app is closed (though the device player)

If we are talking about recorder then there are more..

Thanks...... :)

[jrahma]

Don't forget also the Save option in case the media was played from http and user wants to save it locally in his gallery

This will be a nice feature to avoid IO Streams coding

[KodeKreachor]

Streaming video from Azure (similar to Azure Media Player) would be supreme, being able to stream from manifest urls. Currently AMP in WebView keeps loading an aesthetically unpleasant image in the background:

[PaulVrugt]

@samhouts What's the holdup for this feature? The PR has been there for ages. An update about the status would be appreciated

[jrahma]

I hope it will play sounds and videos from device as well as URL

[davidbuckleyni]

From.what they told me that is the plan.

On Fri, 18 Oct 2019, 10:23 Jassim Rahma, [e-mail] wrote:

I hope it will play sounds and videos from device as well as URL

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/xamarin/Xamarin.Forms/issues/1692?email_source=notifications&email_token=ABK34HK473B6XKULV7YLTQDQPF6ATA5CNFSM4EN2GYN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBTRD2Q#issuecomment-543625706,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABK34HIVK4OJA4X2GGT2TRLQPF6ATANCNFSM4EN2GYNQ
.

[Phenek]

Hey everyone, first that is awesome! Thanks a lot.
I would like to ask to those who work on this PR, if Renderers should be enough to perform (HLS) lecture with DRM Content (FairPlay, Widevine, Microsoft PlayReady, ..) ,
in your opinion?

@peterfoot 👍