Peertube: Max resolution and max length / size for video and audio uploading

[kimsible]

It's really a big issue for moderation.

Actually we only can fix a daily and max stockage quota per user.

I wonder how can we implement easily :

• A max resolution and max length or size for each video uploading (maybe per extension)
• A max length or size for each audio uploading (maybe per extension)

Would you have an idea to create this limitation without implementing a full-feature, maybe with ffmpeg ?

[rigelk]

Could you please detail the moderation use case and how urgent the need is?

More precisely:

• how is unbridled video resolution currently problematic for you?
• wouldn't bitrate be a more fitting metric to consider (instead of resolution)? would it also apply to audio in your use case?
• how is unbridled video/audio duration currently problematic for you?
• what is the use case for different limits per extension?
• what is the use case for different limits between audio/video?

[kimsible]

I simply need to not block all the threads if lots of videos / audio are being sent / transcoded at the same time according to the limitations of a server CPU / RAM.

how is unbridled video resolution currently problematic for you?

how is unbridled video/audio duration currently problematic for you?

I thought about 2K and 4K resolutions and duration lower than 1 hour for videos and 4~5 hours for audio for a Youtube Standard HD format / FLAC audio as metrics understandable for most of video makers but maybe it's not the right approach.

what is the use case for different limits per extension?

what is the use case for different limits between audio/video?

Actually only for audio, we sugguest audio live replay instead of video live replay; we want to encourage audio compression for audio with long duration, especially for lowless compression WAV or FLAC, but can be applied for all audio, again it may not be the right approach.

wouldn't bitrate be a more fitting metric to consider (instead of resolution)? would it also apply to audio in your use case?

Yes, bitrate could be the solution for both audio and video. Do you think about setting the maximum bitrate in ffmpeg ?

[rigelk]

I simply need to not block all the threads if lots of videos / audio are being sent / transcoded at the same time according to the limitations of a server CPU / RAM.

I see. This is moderation in its broadest sense then: making use of resources in moderation - I'm re-labeling the issue accordingly.

we sugguest _audio live replay_ instead of _video live replay_

What are they? If they correspond to a standard, I'm unfamiliar with both terms.

we want to encourage audio compression for audio with long duration

I see, but complexity-wise, even long uncompressed audio streams are significantly lower on CPU than short video transcodes. It might not be worth the trouble.

Do you think about setting the maximum bitrate in ffmpeg ?

We rely on ffmpeg, so there really is no other way. Note that our target resolutions already limit the target bitrate. Here we are talking about limiting the input bitrate, and throwing errors at the user if it doesn't match the input limit:

Accepting/refusing a file in PeerTube is done very simply right now: either you access the upload page or not, and either your file has the correct extension or not. End of story. Here we need ffprobe or its equivalent in client-side js to test the file for its bitrate/resolution/etc. From a UX perspective, having the test client-side is the best, since otherwise you have to wait for the file to be fully transferred for the test to happen server-side and potentially face an error that will erase the metadata you began to write for the video, since we let users edit it while uploading the video. But having it client-side also means a non-negligible bundle size increase. Offloading this bundle in a plugin would be lighter and preserve our UX.

As an example of bundle size increase, the lightest client-side video metadata extraction tool available is https://www.npmjs.com/package/mediainfo.js, and it still adds 5.06 MB to the bundle size.

[kimsible]

Thanks @rigelk for these precious advices.

we sugguest audio live replay instead of video live replay

What are they? If they correspond to a standard, I'm unfamiliar with both terms.

Just very long audio or video (from 1 hour to several hours), for example : video game live streaming of Twitch / Youtube.

having it client-side also means a non-negligible bundle size increase. Offloading this bundle in a plugin would be lighter and preserve our UX.

As an example of bundle size increase, the lightest client-side video metadata extraction tool available is https://www.npmjs.com/package/mediainfo.js, and it still adds 5.06 MB to the bundle size.

Can we hook the file uploading form and also add an alert-info on that page ?

[kimsible]

From https://docs.joinpeertube.org/#/api-plugins, I only found this hook filter:api.video.upload.accept.result but server side only. :-(

[rigelk]

@kimsible indeed, it is going to require an additional client hook :pensive:

[kimsible]

@kimsible indeed, it is going to require an additional client hook pensive

Are the peertubeHelpers avalaible in the register plugin also allow to display custom alert messages (info, warning, error) ?

It would be fine to display a custom alert above the Upload Form before and after browsing the file.

[rigelk]

@kimsible no, but you can inject a div that uses bootstrap classes.

[scanlime]

I've had my server running with a change that enables multiple transcodes in parallel, and this seems to get around one of the symptoms you're seeing: that an especially large upload will block smaller ones for a long time.

[kimsible]

I created this plugin : https://github.com/kimsible/peertube-plugin-upload-limits
It checks defined limits both on client side and server side.

The only problem is that we can't hook YouTube and Torrents imports for the moment, any idea @rigelk ?

[kimsible]

By the way, developing something for URLs imports is too much efforts though youtube-dl relies on a non-standard extracting method, it should be constantly updated... (but I don't think it would even resolve broken youtube-dl API as Google or other closed service changed their methods without any informations if the extracting method is not an official API with credentials).

I will suggest to disable it with upload limits and maybe thinking about creating limits also for Torrent imports as it is a standard.

If you have any comments or requests for this plugin, don't hesitate to ping me.

[rigelk]

@kimsible I am adding the required hooks for both import types. The server-side hooks will allow to check files after they have been downloaded, this way you should be able to keep both import features enabled. They will be present by the next release, 2.2.

[kimsible]

Thanks a lot @rigelk it's a great news :D ! I just updated the plugin to lazy load MediaInfo (with wasm file) and marked only if needed.

[kimsible]

@rigelk thanks for adding the hook in the PR, I'll test it soon.

Also 2 things more :

• I've re-implemented in the plugin, creating and injecting PeerTube Toast, would it be easy to add a client peertubeHelpers to the plugins system allowing to send toasts ?
• Is there a way to translate error message server side and displayed in an error toast after the hook filter:api.video.upload.accept.result ?

[rigelk]

@kimsible beware, the hooks are still not under their definitive form in the PR.

would it be easy to add a client peertubeHelpers to the plugins system allowing to send toasts

Probably. You are welcome to submit a PR :slightly_smiling_face:

EDIT: translating messages server-side is not yet doable.

[kimsible]

After several tests, the best way to display enough upload instructions without ruining the form view is to display a custom bootstrap modal.

@rigelk I'm gonna submit a PR for the custom modal as the notifier before.

[kimsible]

The plugin is now ready for PeerTube v2.2 with all new implements (modal and markdown upload instructions, import URL and Torrent filters) and still compatible with PeerTube v2.1 without limitations on imports and with a simple alert for upload instructions.

https://github.com/kimsible/peertube-plugin-upload-limits