Monogame: Question: Interest in VLC-backed videoplayer lib integration?

[mfkl]

Hello,

This is a question, not an issue.

I'm maintaining a C# wrapper for libvlc and looking into integration with game engines. Unity.Android is currently working but that's about it for now.

Is there community interest in having a libvlc based VideoPlayer implementation integrated with MonoGame?

Cheers,

mfkl

[Jjagg]

Oh yeah, that'd be awesome! Thanks you for the proposal! :)

Video player support is terrible in MonoGame atm, so users are pretty much forced to use a third-party solution if they want to play video. It's not supported at all on mobile and Linux and the Windows implementation has some issues. Not sure about the implementation on Mac or closed platforms (consoles). So yeah, IMO a good maintained video player implementation would be very very welcome.

cc @tomspilman @cra0zy @dellis1972
Also cc @mrhelmut for maybe some practical insights

[MichaelDePiazzi]

I would most definitely be interested in this!

I spent some time trying to get the Windows DirectX version working well (it's based on SharpDX / Media Foundation API). But it leaks memory and I was not able to solve it. It also requires Windows Media Player to be installed, which not all users have (or are willing to install).

In fact, music playback for Windows DirectX is also pretty janky (it's also based on Media Foundation). Could VLC also potentially be used for that? I would love to see a better backend for this also.

[harry-cpp]

I could see 2 problems with using vlc for a video player:

1. License
2. Size of the library

[mfkl]

Thanks for the comments guys.

@cra0zy good points.

License

I'm a license n00b. libvlc/libvlcsharp are LGPL while MonoGame is MS-PL.

if you distribute any portion of the software in its source code form, you may do so only under the Ms-PL by including a complete copy of this license with your distribution. If you distribute any portion of the software in its compiled or object code form, you may only do so under any other license that complies with the Ms-PL

Not sure if that causes an issue, but I'll look into it.

Size of the library

libvlcsharp is small (280 KB) but the native platform specific libvlc builds are indeed somewhat heavy. The size can be potentially reduced (on desktop only currently) by using https://github.com/mfkl/libvlc-nuget/blob/master/cherry-picking.md

[mrhelmut]

While we're at it, I'd just like to note that it would be cool if the VideoPlayer tries to load the library only if it's being used. We should avoid native dependencies to be required if they are not used.

What would be the benefit of libvlc over other libraries like libtheora?

[jbkempf]

@cra0zy @mfkl libVLC, libVLCsharp are LGPL. As long as MonoGame uses it as a shared library, it should be OK.

@mrhelmut libtheora is a codec library. It's very far from being a full player. libVLC gives you a simple player abstraction that is ABI/API stable, and plays everything.

@cra0zy size can be big. Depending on the number of plugins/codecs you put. But as VLC is modular, you can remove a lot of those, if needed.

@MichaelDePiazzi music is doable, sure. But it depends if you need low-latency playback or not.

[harry-cpp]

Okey, so we could use it for DesktopGL and WindowsDX backends. Does anyone have any rough size estimate if we include just 1 video codec?

[jbkempf]

@mfkl also, if you do, please use the OpenGL/DX callbacks API.

[mfkl]

While we're at it, I'd just like to note that it would be cool if the VideoPlayer tries to load the library only if it's being used.

That's the current behavior for most platforms, i.e. call Core.Initialize() to load libvlc before using the player.

please use the OpenGL/DX callbacks API.

Yup https://github.com/videolan/libvlcsharp/pull/11

[mfkl]

Does anyone have any rough size estimate if we include just 1 video codec?

The core libvlc dlls (libvlc.dll, libvlccore.dll) are small (~ 3MB). What gets heavy is the 300+ plugins dlls and you can remove a lot of those if your need is specific enough.

You can see sizes here: https://www.nuget.org/profiles/videolan (note: most packages handle multiple CPU architectures, so take it into account when checking sizes).

[mrhelmut]

Wow, shipping with so many dlls doesn't sound like a plan, what's the minimal set of dependencies? Can they be built statically with libvlc?
I would also be curious about the CPU and memory usage for 1080p playback.

[mfkl]

shipping with so many dlls doesn't sound like a plan

You don't actually need to ship all of them, they're just included in the nuget. Most are really light and they're all hidden away from the user. The libvlc core handle loading/unloading of those dlls, as needed.

what's the minimal set of dependencies?

What's the minimal set of features you need?

libvlc is a multimedia toolbox which can do lots of different things, such as...

Network browsing for distant filesystems (SMB, FTP, SFTP, NFS...).
HDMI passthrough for Audio HD codecs, like E-AC3, TrueHD or DTS-HD.
Stream to distant renderers, like Chromecast.
360 video and 3D audio playback with viewpoint change.
Support for Ambisonics audio and more than 8 audio channels.
Subtitles size modification live.
Hardware decoding and display on all platforms.
DVD playback and menu navigation.

It really depends on what you want to do.

Can they be built statically with libvlc?

That's currently the case for iOS, macOS and Android. Don't think that's currently possible for Windows.

I would also be curious about the CPU and memory usage for 1080p playback.

For the VLC side you can try using VLC desktop :) With MonoGame/libvlc, I'll let you know when things are in place 👍

[mrhelmut]

We only need to decode frames, we have no need of everything else. We also need it be as generic/compatible as possible without tweaking hundreds of parameters.

For example, hardware acceleration is cool and all, but if it's an opt-in option that doesn't gracefully fallback to software on incompatible hardware, then it's going to be complicated to define what's the good set of parameters for MonoGame.

Also, from what I understand (tell me if I'm wrong), libvlc instantiate its own GL/DX context to handle the rendering, which may cause incompatibilities with some hardware.

Please note that I'm not trying to go against libvlc, it's a good proposition and potentially an efficient one (that would allow to support multiple codec at once). But I'm trying to get the relevancy of depending on a swiss army knife while we need only one of its features (that any other dedicated library would do as well).

[willmotil]

Vlc for the monogame video / audio support would be amazing.

I used the vlc player since forever. The player alone is powerful reliable and has all kinds of ability's the current video player in monogame barely works to be honest and is basically unreliable.

[mfkl]

what's the minimal set of dependencies?

Here's a quick experiment with some info https://github.com/mfkl/LibVLC.Windows.Light

We also need it be as generic/compatible as possible without tweaking hundreds of parameters.|

I get it. You don't need to tweak hundreds of parameters though, most configuration tweaks (for Windows) can be achieved with a few lines in the csproj https://github.com/mfkl/libvlc-nuget/blob/master/cherry-picking.md such as

 <ItemGroup>
    <VlcWindowsX64IncludeFiles Include="libvlc.dll;libvlccore.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\video_output\libdirect3d11_plugin.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\video_output\libdirect3d9_plugin.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\video_output\libdirectdraw_plugin.dll" />    
    <VlcWindowsX64IncludeFiles Include="plugins\access\libfilesystem_plugin.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\audio_output\libdirectsound_plugin.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\audio_output\libmmdevice_plugin.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\audio_output\libwasapi_plugin.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\audio_output\libwaveout_plugin.dll" />
    <VlcWindowsX64IncludeFiles Include="plugins\codec\%2A" />
    <VlcWindowsX64IncludeFiles Include="plugins\demux\%2A" />
</ItemGroup>

You can provide a default config such as the above if you like, but ultimately it should be up to the user to make that decision IMHO. Because they will want to play HLS, m3u playlists and display subtitles at some point :-) But that repo should give you a better idea of the situation, I hope.

hardware acceleration is cool and all, but if it's an opt-in option that doesn't gracefully fallback to software on incompatible hardware

It can be turned on and off easily.

libvlc instantiate its own GL/DX context to handle the rendering

The libvlc API for this is still being refined as we speak, so let me get back to you on this. But my understanding is that libvlc does not instantiate its own rendering context.

Please note that I'm not trying to go against libvlc

No worries, that wasn't my impression :+1:

[mfkl]

I'm trying to get the relevancy of depending

Talking about this with a friend, I'm realizing only now that I may have expressed myself incorrectly.

The intended purpose of this discussion isn't to argue that MonoGame should take a dependency on a potential LibVLCSharp.MonoGame package and on LibVLC packages (which are platform specific anyway). I don't think that's a good idea, as my guess is most MonoGame users do not need video capabilities (but correct me if I'm wrong).

The initial point of this issue was to gauge for community/MonoGame devs interest about a potential, independent LibVLCSharp.MonoGame package. Sorry if that wasn't clear initially.

[Jjagg]

Ah, that changes things. Considering the bad state of the built in video player I think it would be valuable for users to have such a package available. Also its capabilities far exceed what MG's built-in video player should do (if we ever get it fixed up 😅), so it would be valuable either way for people requiring more than what MG offers by default.

To actually gauge community interest I think a forum post would be more effective than this issue. No worries about the misunderstanding, this is a place for discussion and the discussion in this thread is still relevant/useful :)

[MichaelDePiazzi]

The initial point of this issue was to gauge for community/MonoGame devs interest about a potential, independent LibVLCSharp.MonoGame package. Sorry if that wasn't clear initially.

This wasn't entirely clear to me initially. But an independent package would work even better for me! So yes, I am definitely still interested. And as mentioned, I am interested in both video and music playback capabilities (I do not required low-latency playback). I would really like to ditch my game's current dependency on Windows Media Player as it causes a few support issues.

[SeriousMartin]

I'm also highly interested! What would be next steps?

[mfkl]

I'm also highly interested!

Good to know.

What would be next steps?

For now, I have to wait a bit that the libvlc API stabilizes (as it's very new) as d3d support is added (in addition of existing opengl). I will ping this issue when work is started so interested parties are aware.

[willmotil]

Has there been any updates on this ?

[mfkl]

I've got a video playing in a Direct3D texture displaying in a WinForms app using SharpDX here https://code.videolan.org/mfkl/LibVLCSharp/blob/d3d-texture/Samples/LibVLCSharp.D3DCallbacks.Texture/Program.cs.

Currently working on Unity3D integration.

The libvlc API has yet to change, meaning that the opengl and d3d APIs will merge into one, but that doesn't prevent us from starting integration, as the final API will be very similar to the current ones, and we can just swap it then.

[harry-cpp]

Closing as duplicate of #3983