Jsdelivr: @latest max-age is way too long

[luiscastro193]

The local cache for latest version lasts 7 days. 1 hour would be perfectly ok, even 1 day would be understandable but 1 week is too much in my opinion.

[MartinKolarik]

Please describe a use case where this is a real problem. Otherwise, it's really just your opinion.

[luiscastro193]

Well I realised about this using pdfjs-dist. The worker file isn't downloaded until you actually start using the library, so it wasn't in cache. The main file was in cache with a previous version, though. So when I tried to test it both versions didn't match and it threw me an error.

Also, it will often be the case than you and another person in your team have different versions without noticing; causing hard to debug inconsistencies.

[MartinKolarik]

This is indeed a possible issue (and one we've previously discussed) but it's a tradeoff between ease of use, speed, and "correctness". If this is a problem in your case, you may either:

• use a specific version and only update it manually,
• add a query parameter of your choice to shorten the cache time, .e.g, file.js?t={{date}} to force a refresh every day.

I believe our current config is best for majority of the projects. Btw, the version mismatch problem is best solved by something like this as it's not an issue specific to jsDelivr, but I understand it may not be possible if you don't control the project.

[luiscastro193]

The query parameter idea would indeed solve the issue. Unfortunately, I usually work with SPAs which don’t allow this approach. I will stick to unpkg then.

Maybe a specific query parameter for this could be a solution? Or a specific tag similar to “latest”.

[MartinKolarik]

We could maybe introduce a new tag indeed, something like @dev which would use redirects with short TTL to specific versions.

[SukkaW]

I think the best practise is to use specific version of code, especially for the open-sourced library, just as mozilla/pdfjs. Any tags, like @dev mentioned by @MartinKolarik, will cause abusement, at least higher server load to jsDelivr cluster.

[ckedwards]

Another option would be to use 302s and redirect @latest to @1.0.1 (or whatever the latest version is). This also solves potential caching issues when the url is not pinned to an exact version such as @1. In this case the cache time could even be increased to something like a year for the resolved urls since package versions shouldn't be republished.

For reference, this is what unpkg does. They also cache the redirect for 1 minute in the browser and 10 minutes at the CDN.
https://unpkg.com/#cache-behavior

[SukkaW]

@ckedwards That sounds promising, but unlike unpkg uses Cloudflare only, jsDelivr uses 4 CDN Providers and maybe some of them doesn't support cache 302.

Also, even if 302 is cacheable at CDN side, for browser side it is much slower than a direct request, which might cause performance impact. jsDelivr supports load files from GitHub repo also makes it hard to track version range on GitHub.

Even unpkg suggests to use specific version at its Introduction of Cache Behavior;

If you want users to be able to use the latest version when you cut a new release, the best policy is to put the version number in the URL directly in your installation instructions. This will also load more quickly because we won't have to resolve the latest version and redirect them.

Still, the best practice is to use a specific version which is immutable, so that it can be safely cached both in CDN and browser.

When using the version range in your production, the breaking changes of lib might ruin your product

[SukkaW]

But I do agree use 302 to redirect a version range to specific version in development is not a bad idea. Maybe cdn.jsdelivr.net/dev/npm or cdn.jsdelivr.net/npm/dev/ can be introduced in jsDelivr and serves the files using the unpkg way.

[MartinKolarik]

@ckedwards @SukkaW 302s are not an issue for the CDNs but are for the browsers - some don't cache them at all or cache them for a shorter time than instructed. When unpkg started doing redirects we looked into that in more detail and the result was that redirects are very bad for performance and not suitable for a production endpoint.

/dev/npm with redirects would be almost the same as https://github.com/jsdelivr/jsdelivr/issues/18121#issuecomment-446165932 and I think that might be the way to go.

[SukkaW]

@MartinKolarik Also, if the specific version is given in dev endpoint, then 301 should be delivered rather than 302.

[ckedwards]

@MartinKolarik https://github.com/jsdelivr/jsdelivr/issues/18121#issuecomment-523643224

redirects are very bad for performance

It seems like how bad depends on how fast the redirect is, but a redirect will most assuredly add latency. If you care about performance, pinning to an exact version is the way to go.

jsdelivr could mitigate some performance issues based off a reasonable caching algorithm for version ranges.
1) If the redirect is cached, return the cached result.
2) If the redirect is not cached, get the latest version that is fully downloaded and ready to go on jsdelivr and return that version. Cache the resolved version for X time.
Additionally, asynchronously check to see if a newer version exists that have not been downloaded and isn't in the process of being downloaded. If a newer version exists within the provided version range, queue a download for that latest version of the requested package.

I don't see any reason why jsdelivr needs to guarantee the latest version with in the version range when a version range is used. Just that it will go and get the latest version eventually, in a reasonable amount of time.

I am not arguing that exact versions shouldn't be used in production (I think there are multiple compelling reasons why exact versions should be used), I was just offering suggestions for what might be best option for jsdelivr when version ranges are used.

[yisar]

I think at least give me a way to clear the cache. Seven days is too long. After we released the new version, we couldn't get the latest file without modifying the version number. That's so bad.

[MartinKolarik]

@132yse https://github.com/jsdelivr/jsdelivr#purge-cache

[yisar]

@MartinKolarik Sent, please check.（from clicli）

[wellington1993]

I think a week is good and PageSpeed ​​say is not: # 18174

But thinking in that discussion ... Maybe a URL parameter to not cache?

I checked and the time interval beetween pdf.js releases lauches is more than 7 days:
https://github.com/mozilla/pdf.js/releases

I suggest increasing the time for @ latest requests cache to: 14 days

And add a way to not cache some requests.
If not save cache request local are poor for server. I suggest a parameter to reduce the cache time to less for development purposes.

What do you think?

Thanks!