Newpipe: Shuffle limited to first n number of videos

[DeadManIV]

• [x] I carefully read the contribution guidelines and agree to them.
• [x] I checked if the issue/feature exists in the latest version.

The shuffle in background play of playlists currently doesn't shuffle the entire playlist. It shuffles only the first 50(? I'm not actually sure how many) videos. YouTube itself has the same problem. For reference my playlist is: https://www.youtube.com/playlist?list=FL3hm1ymIqH41ZKDnr1TW5Iw

This website fixes the issue: http://youtube-playlist-randomizer.valami.info/playlist3.php?pl=FL3hm1ymIqH41ZKDnr1TW5Iw

Perhaps NewPipe could use a similar solution for shuffling videos?

P.S. Thanks for all your work, NewPipe is an amazing app!

[theScrabi]

Yes we should take care about this. @karyogamy can you confirm this?

[karyogamy]

Currently, the shuffling is done every time a new partial playlist comes in, so that it doesn't use too much bandwidth to download the entire playlist.

FYI, on Youtube, the partial playlist size is 100 items and is about 25kb. From a good connection, this request takes about 500ms. So we can see this may can add up (time-wise, as the request is sequential) if the user wants to open a long playlist.

Perhaps we can create an toggle to let user choose if they want to download the full playlist when starting to play. This should be an enhancement though, not a bug, as Youtube is doing the same as well.

[theScrabi]

Yes this might be good.
Does Youtube tell how mch elements are in a Playlist before downloading it?

[DeadManIV]

I believe YouTube has changed how their shuffle works, because I'm getting videos from all over my playlist now when I hit shuffle.
I was wondering if this feature was still planned for NewPipe?

[Avelina9X]

This is still very much an issue. Any chance on getting on update on this? It is very annoying to listen to the same 100 songs when they are part of a much much larger playlist.

[sven-hoek]

For anyone interested: As a workaround, you can scroll down to the end of the playlist and start playing one of the last songs on background ("Start here when backgrounded"). After that, you can switch to the background player and activate shuffle. It should then shuffle the whole list.
It would be cool, though, if it were possible to shuffle the whole list without scrolling down each time.

[EitvidasPuras]

This is still an issue. I wonder what makes it so difficult to fix. It's really annoying to have to scroll all the way to the end everytime so that the whole playlist would shuffle :/

[PeterHindes]

This might be worked arround by shuffeling an index ahead of time. It could be much longer than most playlists as it would just be integers. Might be a bit of an inefficent way to implement it, but its probably faster than fetching all the videos.
So we have an array of maybe 10,000 int's and shuffle them randomly and then check if the next number in the array is an index of a song in the playlist fetched from youtube. If not we check the next number untill we find a song.

[PeterHindes]

Looks like, having checked out #1943 this is likeley not a useable solution though, as the problem lies in the loader for the background player, not the shuffle function.

[PeterHindes]

Having looked at a playlist with a large number of videos such as this one. Its clear that youtube has a video count attribute exposed. This might be leveraged to initalize blank spots in the playlist untill they are loaded in.

[XiangRongLin]

Following the idea of @PeterHindes of intializing an array of the playlist size and shuffling that, I found a way to get the video at a specific index in a playlist.

Using this playlist as example:
https://www.youtube.com/playlist?list=PLgfrzIaFR9HfrMfEHHA75y6ieaeyBxuTu

You can get a specific video by using the embedded player and adding index=NUMBER to the url, like:
https://www.youtube.com/embed/?listType=playlist&list=PLgfrzIaFR9HfrMfEHHA75y6ieaeyBxuTu&index=600
Note that the index starts at 0, while the video number in the playlist starts at 1.

The normal video url can be found with following css query, where the href attribut is the contains the video id: link[rel*="canonical"]

If I understand the extractor correcly the YoutubeStreamLinkHandlerFactory would need to be changed here, so that it fetches the webpage and parses the html to extract the id.
But this seems like bad design since all linkhandler only parse the url.