Webtorrent-desktop: Support more audio and video codecs

[MathieuDebit]

What version of WebTorrent Desktop? (See the 'About WebTorrent' menu)
Version 0.20.0 (0.99.3)

What operating system and version?
Windows, MacOS and Linux

Support more codecs

I'd like to discuss about ways to support more audio and video codecs in Webtorrent Desktop. Considering the choosen technology (Electron/Javascript), there are different solutions available (FFMPEG.js, WebAssembly, plugins, etc), but I suppose the more relevant comes from Butter Project.

Butter Desktop currently supports native decoding for a lot of audio and video codecs (see supporting codecs). I tried to dive into the code, I'm not sure about it but they seem to use video.js with a set of addons and custom implementations.
In the end, Butter Desktop is an app built upon similar web technologies, but supports all the audio and video codecs needed.

What do you think about it? Do you think we could be helped by the current Butter Desktop codebase to support more codecs in Webtorrent Desktop?

Thanks a lot 🙏

[feross]

@MathieuDebit Any ideas how they support the AC3 audio codec? Are they running ffmpeg manually? Or did they bundle the codec into the chromium content module?

[MathieuDebit]

@feross I'm not really sure as the codebase goes beyond my skills, but Butter is still using NW.js, and it seems that there is a way to enable prioritary codecs by compiling sources with FFMPEG:

• nwBuilder withFFmpeg: true: https://github.com/butterproject/butter-desktop/blob/caed8c45e90f471587225d5e3744bc006e1537e3/gulpfile.js#L115-L125
• "Nearly impossible to understand how to compile ffmpeg": https://github.com/nwjs/nw.js/issues/4595
• "enabling for ac3 and hevc codecs support": https://github.com/nwjs/nw.js/issues/5653
• "Enable Proprietary Codecs" on NW.js: http://docs.nwjs.io/en/latest/For%20Developers/Enable%20Proprietary%20Codecs/

[MathieuDebit]

@feross as you said here https://github.com/webtorrent/webtorrent-desktop/issues/350#issuecomment-240903825, I guess the best solution would be to build Electron with a custom FFmpeg, did you already explore this approach?

• Here is an idea of how to implement it: https://github.com/electron/libchromiumcontent/issues/198#issuecomment-211352345
• It seems that there is no problems with seeking: https://github.com/electron/libchromiumcontent/issues/198#issuecomment-367489559
• We should also think about legal issues by adding proprietary codecs: https://github.com/electron/electron/issues/9534#issuecomment-303098715

Sources:

• https://discuss.atom.io/t/information-about-bundled-ffmpeg/28456
• https://github.com/electron/libchromiumcontent/issues/198
• http://pietropassarelli.com/ffmpeg-electron.html
• https://github.com/electron/electron/issues/9534

[feross]

did you already explore this approach?

Yep, we did at one point in time but abandoned it because it complicated the build process too much and the legal situation was unclear. I think things have gotten better on both fronts, so happy to explore this in future versions of WD.

If anyone else has the interest/skill to tackle this before then, go for it!

[MathieuDebit]

So the idea is to replace the ffmpeg library in Electron with our own one.

Thankfully projects like ffbinaries or ffmpeg-static allow us to get the latest binaries for all platforms.

But I don't know how to tell Electron to use these binaries instead... 🚶

Also it could change the build process, will we need a CI or something like electron-builder?

[MathieuDebit]

Sorry if all of this seems obvious for some of you, but as I'm still learning in that field, I prefer write everything down to be sure the most noob of us will be able to follow 😛

So here is two approaches I've found to support more codecs in Electron with FFmpeg:

• build libchromiumcontent with a custom libffmpeg: this solution seems to be the best, but is a power-user feature and means that the WD build process will be way more complicated (here, simply use FFmpeg binaries in Node via ffbinaries or ffmpeg-static (example) is not enough, since we don't only want to access FFmpeg methods, but use it in the <video> tag).

• Convert on-the-fly videos through a local http server like in this app called StreamStudio: this solution doesn't change the build process, and is apparently not that CPU consuming, but will surely complicate the codebase.

I'm digging into the first solution, which is the cleanest, but surely the hardest path 🚣‍♀️

[feross]

@MathieuDebit Thanks for sharing your progress. At one point, I believe the Electron team was producing builds of libchromiumcontent which included a much larger list of ffmpeg codecs enabled. You might want to see if there's something already compiled that you can use.

[paxter]

Any update here? Without (E-)AC3 support webtorrent is useless. :/

[MathieuDebit]

Hi @paxter, unfortunately I wasn't able to build Electron with a custom FFmpeg version, because custom builds of Electron and libchromiumcontent are not easy (at least for me 🤷‍♂️) and Electron team does not share their build process...

[dsernst]

Missing codec support can be a bit annoying because using an external player is a little shaky if the file is still downloading.

Right now the error message looks like:

1) As a user, it would be helpful if the error showed the name of the unsupported codec, so I have more information if it's the same type I'm running across often, or more of a long-tail problem.

2) Would it be helpful to start tracking unsuccessful play attempts across different codecs in telemetry logs, to prioritize formats? Looks like https://webtorrent.io/desktop/telemetry is behind a password, though.