Jsdelivr: Purge API use to update @latest and @[version] URLs

[jhildenbiddle]

First, thanks to the project maintainers for their efforts. Very much appreciated.

I'd like to understand the recommended approach for ensuring end users receive the @latest or most up-to-date @[major] version from jsDelivr after a new version has been published to NPM.

I recently switched from upkg.com to jsDelivr as my preferred CDN for all OSS libraries. One issue that I've run into since the switch is that jsDelivr takes significantly more time to update URLs that include “@latest” or "@[version]" after a new version has been published to NPM. I understand that it takes time for changes propagate through the CDN, but by comparison unpkg.com consistently returns the most up-to-date version using similar URLs within seconds or minutes of a new version being published.

I have requested and received access to the purge API mentioned in the docs, but I want to make sure I'm using it properly and that this is the correct tool to accomplish my goal.

Questions:

1. Does the 7 day max-age header returned by jsDelivr mean end users will not see the most up-to-date version of a file until 7 days after a new version has been published? Apologies in advance for what is probably a "Caching 101" question, but I'm still not clear on how this header plays into my issue. Seven days sounds like a long time to cache URLs intended to provide up-to-date files, and it sounds like others have concerns about the duration as well (see #18121).

2. Will the purge API update all @latest and @[version] URLs so that they point to the most recent version?

3. If the purge API will update URLs, is the expectation that devs will call to this API after publishing a new version to NPM to ensure that the CDN is up-to-date as soon as possible?

4. How long does a cache purge take?

5. Does the purge API allow purging a complete package, or do I need to specify individual files per the instructions provided with the response to my email?

Thanks in advance for any assistance you can offer. Looking forward to resolving this issue.

[MartinKolarik]

Hi @jhildenbiddle,

1. The max-age, s-maxage setting we use for @latest cause 2 things:
   
   
   
   • The CDN caches files for 12 hours so that's the maximum time for new users to start getting the updated file if no manual purge is done.
   
   
   • Browsers of individual users may cache the file for up to 7 days, so yes, it may take up to 7 days for all users to receive the updated version. Most of the time this is not really an issue. If you need to be sure a specific/latest version is used immediately, you can use an url with the exact version included.
   
   

The approach taken by unpkg almost completely eliminates client side caching because your browser makes a request for each file every minute. If there wasn't a new version, the file content isn't transferred again, but there's still the cost of making all those requests, and it can have a significant impact on the performance of your website.

1. The purge API updates exactly the URL you send it. CDN caches are usually like key/value data stores and URL serves as the key.

2. Yes and no. The expectation is there'll simply be some delay for links using @latest. However, what you described is the primary use case for the purge API - there are a few projects that do it because they need immediate updates.

3. As soon as you get a response from our API, it should be purged everywhere.

4. Only individual files right now.

[jhildenbiddle]

Thanks for the quick response, @MartinKolarik.

Your explanation of max-age and s-maxage are very helpful. Perhaps this information is available elsewhere and I just missed it, but it may help others to make this information available under the Publishing packages or Purge cache sections of the README. CDN-savvy devs will know to look at the response headers, but those unfamiliar with the nuances of caching (✋) may not. Just my $0.02.

A few follow-up questions:

1. Since authentication is not required for the purge API, how is permission granted? My guess is that the purge API works on any file that is part of a package in which an email address that has been granted purge API access is listed as an author. Same goes for GitHub?

2. Does the use of the purge API force jsDelivr to retrieve the latest version information from npm, or do I need to hold off on purging immediately after publishing to NPM to give jsDelivr time to detect the new version on its own?

3. To update the @latest and @[version] URLs, is it correct to assume that I should purge the following URLs given a release history of 1.0, 1.1, and a new release of 1.1.1?

   • https://purge.jsdelivr.net/npm/my_lib@latest/file.js
   • https://purge.jsdelivr.net/npm/my_lib@1/file.js
   • https://purge.jsdelivr.net/npm/[email protected]/file.js

4. Would the maintainers of jsDelivr have an issue with a library to help automate this process? The goal would be to make it easier to purge an entire package:

   const jsDelivrPurge = require("jsDelivrPurge");
   
   // Purge @latest and latest @version URLs
   jsDelivrPurge("https://cdn.jsdelivr.net/npm/package").then(result => {
    // ...
   });
   
   // Purge single @version file URL
   jsDelivrPurge("https://cdn.jsdelivr.net/npm/package@version/file").then(result => {
    // ...
   });
   
   // Purge multiple @version file URLs
   jsDelivrPurge([
    "https://cdn.jsdelivr.net/npm/package@version/file_1",
    "https://cdn.jsdelivr.net/npm/package@version/file_2",
    "https://cdn.jsdelivr.net/npm/package@version/file_3"
   ]).then(result => {
    // ...
   });
   

Thanks again, @MartinKolarik!

[MartinKolarik]

Your explanation of max-age and s-maxage are very helpful. Perhaps this information is available elsewhere and I just missed it, but it may help others to make this information available under the Publishing packages or Purge cache sections of the README.

Indeed it's probably one of the things we should add to docs.

1. @jimaek responded via email.

2. Our backend servers cache npm version info for 60 seconds so if there was a request that made it to the origin servers (not just CDN) and you made a purge request shortly after that, the purge may indeed not work. The chance of this happening is very low though.

3. Possibly also https://purge.jsdelivr.net/npm/my_lib/file.js

4. We are planning to improve this API and that includes easier purging of relevant files but there are packages with many files (hundreds or even thousands) so we don't want to encourage blindly sending purge requests for every file until we can implement it effectively.

[jhildenbiddle]

Got it. That's a wrap!

Thanks again for the quick replies and thorough answers, Dmitriy. The effort really is very much appreciated.

Best to you and keep up the great work. jsDelivr is an awesome service.

[timaschew]

off topic: who is Dmitriy? 🤔

[jimaek]

That would be me :)