Peertube: Segmented video architecture to support live streaming and long videos

[scanlime]

Hi, I'm quite interested in solutions to #970 and #151. Both of these issues have at their root that PeerTube currently requires each video to be a single media file.

For background: There are multiple possible solutions to this problem, including very common ones (HTTP Live Streaming or DASH) and more experimental (github/Novage/p2p-media-loader). The most typical solution would be to use a growing .m3u8 playlist full of very short mp4 files which optionally grows during a broadcast. This same playlist format can be used for non-streaming (video on demand) content to provide fast loading and seeking even on very long streams.

I'm opening this issue to ask the PeerTube developers whether it's realistic to expect a solution to these issues in the current architecture. If so, this may be a good place to discuss a migration plan, which may involve a way for the protocol and servers to support both segmented and whole video files. If not, it would be good to know why not, and we can perhaps move on to other projects that still try to integrate at the ActivityPub level, and this may be a good place to plan for protocol interoperability.

[scanlime]

In #1253, Chocobozzz says the video format is stable. I assume this means Peertube wouldn't consider replacing either current format (mp4 over HTTP, and the same but wrapped in WebTorrent) with an alternative segmented format (HLS, DASH, etc).

Replacing the format entirely would be a compelling option in some ways, since it leads to a straightforward server without any duplication of the videos. It burdens the client with remuxing in the event users want to download the video as a single file.

Another option might be to include HLS (.m3u8) as well as playlist-of-torrent formats alongside the current monolithic video files. This would either require duplicate storage space, or on-the-fly remuxing.

Another alternative is that videos could have multiple types of backend storage, with the missing formats generated on-the-fly until they've been requested frequently enough to cache.

Would it be at all possible to switch entirely to a new segmented format, or are we definitely stuck supporting multiple formats?

[scanlime]

Another format option: Segmented MP4 wrapped in DASH. We can use a DASH MPD file to describe all available resolutions, and we can use multiple files if necessary during a live-stream. After live-streaming, or for single-file uploads, we'd remux into a segmented MP4 that can be used as-is or the individual segments can be referenced from the DASH MPD.

Mp4box is a tool to remux mp4 files using segments,
https://gpac.wp.imt.fr/2011/02/02/mp4box-fragmentation-segmentation-splitting-and-interleaving/

[scanlime]

Looks like ffmpeg can produce fragmented moov atoms on its own too. Not sure if we'd need to create the manifest ourselves in that case,

https://stackoverflow.com/a/9734251

[Serkan-devel]

I thought I head something like this but under the term, stream rebroadcasting.
On a talk about webrtc and the phoenix web framework, they mentioned that there will be something like this in the future

[Chocobozzz]

It was in 2016 so technically we are in the future :p

There is https://github.com/Novage/p2p-media-loader that is also interesting

[Serkan-devel]

are there any roadblocks to implement it?

[scanlime]

my biggest question here is how we provide segmented videos over
activitypub while maintaining compatibility with whatever other software
(including other versions of peertube) that we want compatibility with.

-micah

On Tue, Dec 18, 2018 at 8:14 AM Serkan-devel [e-mail]
wrote:

are there any roadblocks to implement it?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Chocobozzz/PeerTube/issues/1237#issuecomment-448276547,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA4YpBy1Kgw4Q7yrtykzHHFBc_1aGzEcks5u6RRogaJpZM4XWiGH
.

[Serkan-devel]

that might require many messages sent for a video

[Chocobozzz]

@scanlime Adding the DASH/HLS playlist URLs in the AP project, while continuing to provide the torrent file.

@SerCom-KC I'll try to investigate how we could integrate this library in PeerTube this year. But p2p-media-loader still lacks support of checking segment integrity and I don't know yet how we'll be able to provide multiple URLs for a single segment (redundancy system/CDN). (edit by rigelk: Chocobozzz made a PR adding segment integrity and it has been merged into p2p-media-loader)

[Chocobozzz]

WIP to add HLS support in our player (in addition to webtorrent): https://github.com/Chocobozzz/PeerTube/pull/1566. It uses p2p-media-loader to share segments with WebRTC. Then I'll try to work on the server side, to generate the playlists/segments.

HLS transcoding will be opt-in, because we'll need to have the mp4 raw files + mpeg segments (therefore multiply the videos storage by 2).

If our HLS player works well, we can imagine adding support of livestream (https://github.com/Chocobozzz/PeerTube/issues/151) in the future in it (livestream is not be possible/easy with webtorrent).

Dash support for videojs is not implemented in p2p-media-loader yet.

[mrlika]

mpeg segments

It is better to use fmp4 segments in HLS because they do not require transmuxing. MPEG-TS segments are transmuxed into fmp4 segments on the fly to be added for playback. And you can have all the segments in one fmp4 file. In HLS manifest you should specify byte ranges of all the segments including initialization segment.
In this way, you can have both raw mp4 and HLS manifest for it. But in this case HLS is overhead because raw fmp4 file is natively supported almost everywhere. HLS is good when you have multiple qualities of the same video - few fmp4 files with fragments of the same duration but of different qualities.

See example:

fMP4 file: https://storage.googleapis.com/wvtemp/rkuroiwa/hls_single_segment/sintel_single_segment_hls.mp4

HLS manifest for it: https://storage.googleapis.com/wvtemp/rkuroiwa/hls_single_segment/sintel_1080p_single_segment.m3u8

[mrlika]

I think single fMP4 is the same thing that is progressively downloaded and played by WebTorrent library

[Chocobozzz]

@mrlika Thanks for your feedback.

I considered fragmented mp4 so we don't have to generate 2 different files for every resolution, but unfortunately the library we use to stream webtorrent files does not support them: https://github.com/jhiesey/videostream/issues/20

[jimmywarting]

(livestream is not be possible/easy with webtorrent).

Would mutable torrent and https://github.com/webtorrent/webtorrent/issues/886 solve live streaming by using webtorrent?

[Chocobozzz]

Would mutable torrent and webtorrent/webtorrent#886 solve live streaming by using webtorrent?

I don't really know, but I don't think we can find a cleaner way to do live streaming than using a well known protocol (HLS) with a custom loader that do P2P for some segments (p2p-media-loader).

[Chocobozzz]

I considered fragmented mp4 so we don't have to generate 2 different files for every resolution, but unfortunately the library we use to stream webtorrent files does not support them: jhiesey/videostream#20

Or maybe I can try to generate a fragmented mp4 file that also contains the moov atom at the beginning of the file :thinking:

EDIT: does not work

[Chocobozzz]

Implemented in https://github.com/Chocobozzz/PeerTube/pull/1566

We now use a fragmented mp4 instead multiple ts segments. But we still have 2 different files per resolution: 1 for HLS and 1 for WebTorrent. One day, if we are able to stream in the browser fragmented mp4 files (coming from webtorrent) we'll use only 1 file.

[Chocobozzz]

Demo: https://peertube2.cpy.re/videos/watch/4b997fc0-e106-42d9-bff9-ae9d64902bbb?mode=p2p-media-loader

[scanlime]

Cool! Loads very fast on Firefox on laptop here. Not sure if mobile is supported yet, I tried on iOS and it was a black screen.

[scanlime]

Also, is there any chance we could have a demo of a longer video? I have a lot of videos that are 5 or 10 hours long, and it'd be great to see something like that load instantly.

[Chocobozzz]

@scanlime Should work on Android. I'll see why iOS does not work.

Also, is there any chance we could have a demo of a longer video? I have a lot of videos that are 5 or 10 hours long, and it'd be great to see something like that load instantly.

No sorry, but please try it :)

[ncryptid]

Sorry for commenting on a closed issue, but I was wondering how adding fragmented videos affects live streaming. Is that still something in the "maybe, but unlikely/difficult to implement" pile of requested features or is it now something that could conceivably be done?

[tcitworld]

@ncryptid Please see https://github.com/Chocobozzz/PeerTube/issues/151#issuecomment-459955744