Hls.js: HEVC support

[saPhil]

Is there any word on HEVC support coming to hls.js? Without it my project is basically dead in the water, so it would be nice to at least get some sort of feedback on whether it can happen. Looking through I see that there are issues in the project going as far back as 2016 that ask questions about HEVC support, but I can't find any official statement about it.

https://github.com/video-dev/hls.js/issues/619
https://github.com/video-dev/hls.js/issues/1097
https://github.com/video-dev/hls.js/issues/1470
https://github.com/video-dev/hls.js/issues/2209

1728

[Vladislavik]

Waiting it too

[SalimF]

me too 👦

[robwalch]

Me three ~but we haven't gotten any contributions~ If you'd like support, please provide more information here like:

• Which platform and browsers do you expect this to work on? Do they support HEVC via MSE?
• List of test streams with HEVC renditions like Apple's reference streams: https://developer.apple.com/streaming/examples/advanced-stream-hevc.html
• Use cases (4k HDR, Alpha) https://developer.apple.com/videos/play/wwdc2019/506/

If this is priority for your company please dedicate resources to adding support via a PR. It's not on the roadmap for 2020 at JW Player, but that could change. If it does I'll let you know with a PR or breakdown of what needs to change in the player. I assume our mp4 remuxer assumes h264 AVC and just isn't up to snuff yet. Before we embark on this though it needs to be clear which devices or browsers will be able to playback HEVC, and whether or not the browser can even handle multi-codec manifests or not.

[mangui]

an old fork with PoC HEVC support
https://github.com/xlazom00/hls.js/commit/db6a87552e6b07ec36eb952b2eb700fb000149ca

[andrewaggb]

I've been looking for hevc as well. As far as I can tell the bitmovin player (paid) may be one of the only ones with support for it from a dash/hls point of view.

Here's an example
https://bitmovin.com/demos/stream-test?format=hls&manifest=https%3A%2F%2Fbitmovin-a.akamaihd.net%2Fcontent%2Fdataset%2Fmulti-codec%2Fhevc%2Fstream_fmp4.m3u8

For my project I wrote an mp4 encoder (it's in c# though based off specs and ffmpeg sources) and I transmit the video already packetized as mp4 over websockets, so it's not helpful here, other than to say practically speaking the bulk of the work is parsing the width/height/codec parameters and whatnot and creating the mp4 header. You also need a valid hev1 mimeType to initialize the media source extensions. This guy has a pretty good link which shows whether your browser supports different codes (or thinks it does).
https://cconcolato.github.io/media-mime-support/

For actual testing, I've been using the new edge with the hevc codec purchased in the windows store (it's a buck or two) and the http://chromium.woolyss.com/ browser (free) with all codecs enabled. The old version of edge with the codec from the store works as well.

I work in IP surveillance and several large vendors are now shipping a webassembly version of ffmpeg, transmitting video over websockets, and decoding to canvas using the webassembly ffmpeg. I'm tempted to try that at some point but I don't have a huge dev team like these guys do so I'm not sure if/when I'll get around to it. They are sending 4MP camera feeds that seem to playback just fine using javascript ffmpeg, so it's apparently fast enough.

[SalimF]

They are sending 4MP camera feeds that seem to playback just fine using javascript ffmpeg

Does the data transmitted as H265 codec and encoded by ffmpeg.js or it have other layer of encoding (e.g make it browser compatible )
Since ffmpeg.jsonly support H.264

[andrewaggb]

I'm pretty sure they compiled their own version of ffmpeg for
javascript/wasm with the codecs they need. Basically my understanding is
that you make a c file to expose/brdige the libavcodec functions you need
in/out of javascript that's also compiled with it and then you use a web
worker to do the decoding.

[andrewaggb]

In case it's helpful for people who maybe don't understand the difference - if you don't have to use hls + mpegts and can use mpeg dash + mp4/m4s that should work with edge/safari/etc. Media source extensions native format is mp4/m4s. If you have the choice between hls and dash and just want h265 as an option, dash is easier. hls requires converting mpegts to mp4/m4s and parsing all the hevc headers in javascript (which is not a low effort project). In dash it's already in that format and presumably some other tool or library has done the conversion (eg ffmpeg).

[SalimF]

^^Hi I read article were they able to compile ffmpeg.js to support all codec required with WebAssembly , they didn't mention h265 but since ffmpeg.js works as stand alone ffmpeg binary file that will solves h265 encoding support on the browser
https://itnext.io/build-ffmpeg-webassembly-version-ffmpeg-js-part-1-preparation-ed12bf4c8fac

[robwalch]

We're not going to add ffmpeg to hls.js.

Here's an example
https://bitmovin.com/demos/stream-test?format=hls&manifest=https%3A%2F%2Fbitmovin-a.akamaihd.net%2Fcontent%2Fdataset%2Fmulti-codec%2Fhevc%2Fstream_fmp4.m3u8

That does not work in Chrome. Chrome does not support hevc via MSE.

It works in Safari, and you don't need hls.js in Safari to play HLS files. You can play hevc HLS steams in any video player using video.src = when the browser supports the file format and codec natively.

To contribute to the conversation please answer these questions https://github.com/video-dev/hls.js/issues/2451#issuecomment-562340852

• Which platform and browsers do you expect this to work on? Do they support HEVC via MSE?
• List of test streams with HEVC renditions like Apple's reference streams: https://developer.apple.com/streaming/examples/advanced-stream-hevc.html
• Use cases (4k HDR, Alpha) https://developer.apple.com/videos/play/wwdc2019/506/

[robwalch]

I have confirmed that hls.js can play the stream mentioned above (HEVC renditions over fmp4) https://bitmovin-a.akamaihd.net/content/dataset/multi-codec/hevc/stream_fmp4.m3u8 in Safari. So the support is there, just not over TS (haven't seen a single example), and it's not something we actively test.

Apple's reference stream https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_adv_example_hevc/master.m3u8 ~still presents issues, so we can do better.~ is supported as of v1.0.0-beta.3.

[robwalch]

HEVC fmp4 is supported in hls.js in Safari as of v1.0.0-beta.3 with #3358

I have not seen any evidence that HEVC in TS is supported in Safari and thus am not planning to support it in hls.js. Test streams provided here and this one I just produced with HEVC in TS do not render video in Safari with src= or in hls.js https://playertest.longtailvideo.com/hevc-mpeg-ts/stream.m3u8