Aerial: [Feature Request] Ability to limit cache size

[Chewieez]

I think it would be very helpful to have an option to limit the available size for caching. I just saw that after just a couple weeks of being installed, my cache folder was 12gigs and my coworkers was 14 gigs.

We both currently have smaller SSDs in our machines and also have VMs installed so we don't have a large amount of free disk space. I'd love the ability to set the cache to never grow to more than a few gigs.

Thank you!

[glouel]

Hi @Chewieez

Yep, that's a bit of a reccuring issue I need to take care of. Right now Aerial will download everything by default, and with 100 videos and 4K HDR files that are huge, that's not a good default.

My idea for a "sensible" default was to only enable tvOS13 videos (and disable the older ones by default). This should limit somewhat the size.

Having a limit is complicated with the way it works now. Right now, Aerial will stream and persist anything that's selected (and everything is selected by default). That can lead to 30 GB caches. If I put a limit, then it's a question of what do I do when the limit is reached. Do I stop downloading videos and only use what's selected (might be a bit weird for people, what if you want to manually download one, should I deny you ? etc), or do I do a rolling cache ala Apple TV (what most people seem to want, I'm a bit reluctant on that one).

So it's complicated, I'm still not sure which way to go ! It's definitely an issue (not a huge one, but it's getting worse every year with new videos). I'm open to suggestions on the topic !

[puttin]

related https://github.com/JohnCoates/Aerial/issues/190

[adib]

+1 on this – also maybe move the video folder into a separate Caches folder (NSCachesDirectory instead of the general _Application Support_ folder.

[glouel]

+1 on this

This is the very next thing on my list after finishing the inclusion of Weather

– also maybe move the video folder into a separate Caches folder (NSCachesDirectory instead of the general _Application Support_ folder.

So, this was the default for Aerial long ago. And I had so many people complaining that "Aerial deleted my videos" and "keeps using by bandwidth" while they were using some sort of cache cleaning tool to reclaim space that I really don't want to go back there. Catalina install also wiped the cache which was unexpected, and not nice ! So, nope, sorry, really don't want to go there again. Cache limitation will have to do.

[adib]

@glouel "People" complaining that caches are deleted after they're using cache-removal tool is really user-problem (i.e. PEBKAC).

OTOH these videos shouldn't get backed up by Time Machine and should "give way" to "more important" files instead of hogging the precious main drive.

[glouel]

While I agree in principle, it's kinda my problem when I have to spend multiple hours doing support on my own, as the %age of people understanding how macOS caches work is probably slim. I'll see about bringing it as an option when the limit comes around.

[prathmeshranaut]

+1 My cache is 33 GB right now. Would love to have it set to a limit say 5 GB and then have a checkbox, which says

• [ ] Do not download new videos once cache is full.
• [ ] Keep downloading new videos and clear old videos in cache.

1st option would just stop downloading videos once the cache is full for the user which is pretty self explanatory.
2nd option would work like LRU cache.

[glouel]

@prathmeshranaut that's next on my list right now ! The new cache thing will have a lot of impacts everywhere so I'm currently trying to assess the changes I need to make beside the UI (both the cache tab, and the video tab where you set your playlist will have to change).

Regarding the rolling cache, I'm still not exactly sure how I'll have it work. Something LRU like may not make a lot of sense considering the playlist is random, if you see what I mean. The way Apple TV does it is, replace the cache (or a portion of it, I'm not sure) every X period (for example a week).

Then there's a big question around how I replace the cache. Do I stream new videos ? I've done everything to hide the streaming as it's quite unreliable because the servers aren't configured to handle it properly. So let's say someone want to disable streaming and have a rolling cache... do I just download in the background without asking ? I know for many internet is ubiquitous but there are still some metered connections out there (and macOS is pretty bad at telling you when you are doing something like being tethered to your phone).

So still a few headaches around the rolling cache. I'll give more updates when I'm getting close to release a beta.

[deceze]

Perhaps putting an expiry date on items in the cache makes the most sense, perhaps as a configurable option like "cache videos for n days/weeks/forever". While the cache isn't full, randomly pick any video, in the cache or not. If the cache is full, only pick videos from the cache. Let cached videos expire after the chosen time span, freeing up the cache, allowing new videos in.

The result should be a slowly rolling, random cache. The more bandwidth you want to conserve, the more conservative you can be with your expiry date setting.

[CraigCottingham]

If I deselect the "Cache Aerials as they play" checkbox, can I safely delete the contents of the cache folder? Do I need to log out/log back in or restart for Aerial to stop looking for videos in the cache folder?

[glouel]

Perhaps putting an expiry date on items in the cache makes the most sense, perhaps as a configurable option like "cache videos for _n_ days/weeks/forever". While the cache isn't full, randomly pick any video, in the cache or not. If the cache is full, only pick videos from the cache. Let cached videos expire after the chosen time span, freeing up the cache, allowing new videos in.

The result should be a slowly rolling, random cache. The more bandwidth you want to conserve, the more conservative you can be with your expiry date setting.

I like that, mark the download date of an item so I can slowly purge in the background. Doesn't solve everything but that's a good compromise I think for the rolling cache !

If I deselect the "Cache Aerials as they play" checkbox, can I safely delete the contents of the cache folder? Do I need to log out/log back in or restart for Aerial to stop looking for videos in the cache folder?

You want to go full streaming and never cache anything that's right ? I wouldn't recommend doing that but if that's what you want yes, no need to restart or anything, you can wipe the videos and Aerial will not cache again.

But you should really not do that, it's a lot of bandwitdh, and the streaming is unreliable (no resume if a connexion hang, which happens).

[CraigCottingham]

But you should really not do that, it's a lot of bandwitdh

I work from home, when my screensaver is up _nobody_ is using any bandwidth to speak of, and I'm fortunate enough to have a 1 gig fiber Internet connection. I'm more bothered by the 33 GB of otherwise-unusable SSD space. I appreciate your concern regardless. 😄

If the rolling cache gets implemented, I may revisit turning it back on.

[glouel]

Just be aware that Aerial may "hang", you'll see it get stuck in the middle of a video if streaming fails. You can still exit normally so it's no issue.

[prathmeshranaut]

Just be aware that Aerial may "hang", you'll see it get stuck in the middle of a video if streaming fails. You can still exit normally so it's no issue.

Happens sometime to me. But I still like seeing the paused Aerial view that I don't mind.

I work from home, when my screensaver is up nobody is using any bandwidth to speak of, and I'm fortunate enough to have a 1 gig fiber Internet connection. I'm more bothered by the 33 GB of otherwise-unusable SSD space. I appreciate your concern regardless. 😄

+100

[glouel]

@prathmeshranaut et all, I created a new issue to track the road to 2.0 which will include limit as one of the core new features, check here : https://github.com/JohnCoates/Aerial/issues/1006

I tried to explain where I'm going, feel free to discuss that there. It's going to take some fundamental changes that will go beyond just the limit but hopefully it will end up in a better place :)

[unbeatable-101]

This has been implemented with the 2.0 release, I suggest closing the issue.

[glouel]

Completely right thanks for reminding me !