Video-hub-app: Corrupt/Difficult preview extraction needs some optimisation

[cal2195]

Just leaving this issue here to discuss future improvements - I'm having terrible trouble getting the hub to function well without the corrupt video checking function!

Part of this is due to low timeout (I confirmed when running with --debug), which is killing the same clips everytime, and as the extraction queue is always in the default order, it never manages to get anywhere.

Some of it could be due to badly encoded videos, but I feel the app should be able to cope with such things, even if just marking it as "Ignore this one" or "Try again harder later". Even just "Do it until it's finished" was a better solution currently as I cannot add any new videos as it never gets to them.

For now, I've just sorted the extraction list by date added, so it does the new ones first, but this will need a proper solution.

💭

[whyboris]

Thank you -- as I commented while removing your original method for checking for corrupt videos I'm not very happy about my current implementation: https://github.com/whyboris/Video-Hub-App/pull/269

I welcome all suggestions 🙇

[whyboris]

I have another idea -- the source material matters! If it's 4K (or larger) it will take longer to extract screenshots:

• Input resolution matters too

So rather than doing the computation solely on the _output_ resolution, I will take _input_ resolution into account as well!

[cal2195]

Haha, oops! I had assumed this was already taken into account! 😅

Good idea regardless! 👍

[whyboris]

With #325 I started taking source video height as input when calculating the timeout duration. At the moment set proportionately longer when above 720p 🤷‍♀️

I would like to eventually figure out a better way to handle broken videos. It really sucks when a large folder of broken videos in the beginning will stall the extraction for long enough that users will never get to extracting the non-corrupt videos that come after in extraction order.

We'll need to handle that problem in another PR ... somehow 😅

[whyboris]

I had a user tell me that out of 85 videos, on first scan 80 screenshots were extracted, on 2nd scan more got extracted, but only by 4th scan all 85 were extracted. This was _after_ we emailed back and forth and I requested he not use the computer while extractions happen (best scenario).

Originally, the same user said as many as 40% of his videos never got their screenshots extracted 😰

I _really_ need to improve the extraction timeouts and possibly change how extraction process works.

Most-importantly I can just extend the maximum extraction time 👍 -- I'll do that for sure, at least by 30% across the board.

A possibly-simple solution is to do a probabilistic extraction: randomizing which video gets its screenshots extracted next will mean that problematic videos won't gunk up the extraction queue every time at the same spot (imagine having a few at the beginning -- always frustrating).

I'll see if it's easy enough to randomize the order right before the batch extraction method is called 👍

[whyboris]

363 will close the issue by doing the following:

The first scan is not changed, but the re-scanning ("Add missing thumbnails") will do this:

• Allow for 50% more time per screenshot (if needed)
• Randomize the order of extracting screenshots (so as not to get stuck on any corrupt files 'every time')

[whyboris]

Please feel free to create a new issue if needed (so it's higher on the list of issues) rather than continue this one -- just refer to this one if needed 🙇