Youtubeexplode: DownloadMediaStreamAsync taking too long

[Kriste00]

When using DownloadMediaStreamAsync to download even a short video it will use up to a minute to return.
PS: It worked fine a few days ago.

[Tyrrrz]

Looks like an issue with ratebypass

[Kriste00]

ratebypass?

[Tyrrrz]

Only concerns adaptive streams, youtube-dl is having the same issue

[Tyrrrz]

The issue is that Youtube normally serves media streams at the same speed that you play them so that 5 minute video will take around 5 minute to download. There is a parameter that lets you bypass this limitation, called ratebypass, which YoutubeExplode and other libraries were forcefully adding to the request, but since recently it stopped working, most likely because parameter integrity is now enforced in the signature.

[Hrxn]

So, youtube-dl was using ratebypass as well?

[Kriste00]

Any workarounds?

[Tyrrrz]

@Hrxn yes.
@Kriste00 none, except not using adaptive streams.

[Kriste00]

How to not use adaptive streams?

[Tyrrrz]

By using muxed streams

[Hrxn]

@Tyrrrz Ah, good to know.

By using muxed streams

Well, is this really a viable alternative? Because they are limited to 1280x720, not sure if that is really acceptable.

But, on the bright side, the download still works as expected. A bit slow maybe, but you can always queue up your desired links and let it do the work in the background.

Or am I misunderstanding something here?

[..] most likely because parameter integrity is now enforced in the signature.

That means that the requests are still all completely correct, right?
And I hope that is also true for the returned data, i.e. the media streams. They are still intact, and the integrity is still guaranteed, or not?

[Tyrrrz]

You are correct on all accounts

[Kriste00]

Is higher videoquality = better audio quality?

[SlowLogicBoy]

@Kriste00 no,
Audio streams come separated from video stream, unless you are talking about mixed streams.

[Kriste00]

I mean Mixed streams

[Tyrrrz]

Yeah, generally higher videoquality = better audio quality for muxed streams

[Tyrrrz]

From what I can tell, parameter set is now somehow enforced by signature because changing values will result in a forbidden error. That means that to get adaptive streams with rate bypass, we'd need it to already be present when requested from get_video_info or find a way to re-calculate signature and make it valid. Or maybe use partial streams or something.

[Hrxn]

@Tyrrrz Thank you for your answer, really appreciated!

[Giggitybyte]

Any news on this?

[Tyrrrz]

Well, when you open a video in browser, it plays sequential partial streams, but they are nowhere to be found in original manifest or dash manifest (which in turn is almost never available too). It makes me think that they are somehow set from the JS code.

[Tyrrrz]

One possibility that I haven't tried yet is to download the stream in multiple parts from multiple connections, not sure if that will work though.

[shikhir-arora]

I'm not using YouTubeExplode here, but that doesn't really work. It scales a bit and certainly helps with large files because the DASH throttle doesn't kick in as fast (but we're still talking several minutes with many connections to download an hour video, better than the hour it could take but not suitable for real-time). With youtube-dl: https://github.com/GieselaDev/Giesela/commit/222d41796d0318b0fd61472c7703751d0577d85e#commitcomment-27136405

Mixcloud has been doing this for a long time @Tyrrrz -- similar to what you described. That's why efficient Mixcloud downloaders use an approach by proxy and serving multiple client connections.

[Tyrrrz]

So good news, I guess. I quickly threw together a segmented downloader and it managed to download a 3:50 video in less than a second, compared to ~2 minutes from before.

[Kriste00]

Yay

[Tyrrrz]

You can try the segmented downloader on the segmented-download branch with the demo projects and see how it works for you. It's still work in progress and if you have comments, please post them in #89.

[Kriste00]

Could you make a myget beta build for this.

[Giggitybyte]

I was just about to make a new issue requesting this ^
Automatic MyGet builds for this repo would be very helpful.

[Tyrrrz]

I could set up MyGet or just use AppVeyor's nuget feed, but usually I can publish a pre-release fast enough for this to not be necessary.
Speaking of which, latest changes are available in new alpha release, which should be indexed within 10 minutes from this post: https://www.nuget.org/packages/YoutubeExplode/4.1.0-alpha3

EDIT: ok, maybe more than 10 minutes. NuGet seems slow but it will be there eventually.

[Giggitybyte]

4 hours later :(

[Tyrrrz]

Well that's weird

[Tyrrrz]

At least it definitely uploaded

Apparently they're having issues again

[Tyrrrz]

Anyway, I investigated how the new downloader works on some videos and it was still throttling in some cases so I made some improvements to it and published alpha4. When nuget finally revives itself, please test it on videos of all sizes.

[Tyrrrz]

Oh, it actually got published: https://www.nuget.org/packages/YoutubeExplode/4.1.0-alpha4. Curiously, alpha3 is still returning 404 for some reason. 😆

[Giggitybyte]

I've given alpha4 a light test and it seems it downloads audio streams at full speed.
I've tried out around 15 videos with lengths varying from a few seconds to a half hour and they all seem to be downloading just as fast as the muxed streams.

I'll give some longer videos a try later, but right now everything seems to be working smoothly 👍