Jsdelivr: New back-end question - upgrading projects

[Jerska]

Your new back-end is live and its features seem really great (and way simpler to handle for you guys)! Congratz!
For previous projects with a file describing their app, are you still hosting the files or are you keeping a map to reroute?

Basically, my question is : for projects which used jsdelivr.com/project/1/..., will updates pushed to npm or github still be handled or do we need to change to /npm/project@1 for updates to be caught?

[jimaek]

All old files are frozen and will no longer be updated. Nothing will break but it also means no auto-updates.
People using version aliasing need to switch to the new format.

In special cases where maintaining old URLs is critical we can find a way to do so.

[Jerska]

The main issue we'll have here was that, at Algolia, we're relying on this auto-updating feature to roll out releases to our users.
It was a really awesome feature to have and we've used it in basically any usage section of our libs. Two examples:

• autocomplete.js
• instantsearch.js

Would it be possible to reuse the old app manifests you have to reroute those old links to /npm or /gh?

[Jerska]

Since those links are embedded in our user websites, we obviously won't be able to change those links for them. :)

[jimaek]

@MartinKolarik Can we somehow combine our legacy files with npm?

1. Alias cdn.jsdelivr.net/algoliasearch to the npm endpoint
2. Pre-generate a cache for these files to make our backend think it served the legacy files as npm packages
3. Make required changes to match the file structure

[MartinKolarik]

In case npm packages are available for all versions I think just aliasing /algoliasearch/$version/$file to /npm/algoliasearch@version/dist/$file could work.

[MartinKolarik]

I'm not sure I understand why is it so important that people receive updates when using the old links though. @Jerskouille ?

[Jerska]

I'd be happy to expand @MartinKolarik !

There were a few advantages on always giving a /MAJOR/ link to our users:

1. Bugfixes: We've had the case a few times where we needed to roll-out a bug fix to everybody. Our algoliasearch package is our JavaScript API client and contains a whole lot of logic to handle connection to our servers and retrying in case of network issues.
   It took us a few iterations to get there with something we're really happy with, and being able to deploy it to every single user instead of manually asking them to upgrade was awesome.

2. Consistency of the docs : instead of having one doc per version, we can now simply have one doc per major version. So that when one of our users wants to add something, he/she just needs to find the doc about the lib and doesn't have to worry about selecting the right version. You won't find something in the docs that won't work with your current setup.
   We try to make most of what we create backward-compatible, instantsearch.js which was created in 2015 is still at its v1, but has changed a great deal since the beginning.

3. Why is it important for us with the old links? We have no way of making them change it. An email that we'd send asking them to change would require a small amount of work, but on a lot of people's end. Also, it would probably not be read by everybody, some of our users had a dev integrating us but are not devs themselves, etc.

[Jerska]

I do understand this might be involving quite some work on your side, and that's definitely not the goal here. However, you would hence be saving us a great deal of work trying to get our users to migrate. :)

[MartinKolarik]

@Jerskouille thanks for the additional details. I understand the benefits but you can still use version aliasing with the new URLs so basically the only reason for this is users using old URLs getting new bug fixes, right? Just trying to understand your reasons.

As I mentioned above, we should be able to do this just by aliasing the old URLs to the npm endpoint, provided all versions currently on jsDelivr are also on npm and there is no difference in file structure (which seems to be the case).

[jimaek]

If we do that we will need to test all URLs to make sure nothing breaks. Also need to check if it would impact the legacy /g/ function in any way.

@Jerskouille How urgent is this for you guys?

[MartinKolarik]

This wouldn't impact /g/ in any way, meaning it also wouldn't "fix" it. If we wanted to change this for /g/ too, it would need to be hardcoded deeply in our backend code, and that's a bit too much imho.

[Jerska]

so basically the only reason for this is users using old URLs getting new bug fixes, right? Just trying to understand your reasons.

This and the mismatch between our docs and the current library version they'll be served.

@Jerskouille How urgent is this for you guys?

Asking for any ETA would feel inappropriate here.
We don't have a bug that needs fixing right now, and our docs are obviously still in sync with the features available to our users.
It's just that the more time passes, the more risky it will get for us, in terms of either a bug happening that we can't fix for everybody or support requests because of the mismatch with our docs.

[MartinKolarik]

@Jerskouille can you give me a list of packages this applies to?

[Jerska]

With pleasure!

• algoliasearch@{2,3}
• algoliasearch.helper@2
• algoliasearch.zendesk-hc@{1,2}
• docsearch.js@{1,2}
• instantsearch.js@1
• places.js@1
• autocomplete.js@0

Other algolia projects where it's not useful:

• instantsearch-googlemaps: outdated

Thanks a lot for handling this, we really appreciate it!

[MartinKolarik]

Thanks. I checked the packages and we should be able to update these without problems:

• algoliasearch
• algoliasearch.helper
• algoliasearch.zendesk-hc
• docsearch.js
• instantsearch-googlemaps
• places.js

Older versions of instantsearch.js currently exceed our 50 MB limit but we might be able to update it too when we add support for increasing this limit for individual packages.

[jimaek]

Older versions of instantsearch.js currently exceed our 50 MB limit but we might be able to update it too when we add support for increasing this limit for individual packages.

How big are they?

[MartinKolarik]

About 80 MB.

[jimaek]

We could probably increase our global limit to 85 or something like that. There are lots of valid projects above the 50MB limit.

[Jerska]

This would be awesome! 💯

[vvo]

Hi @jimaek, we reduced the exported npm size of the instantsearch.js project, can you raise the limit a bit so that this project is still being updated while we do the necessary documentation updates everywhere?

[vvo]

Also wondering what decision you took as for continuing upgrading the projects @Jerskouille gave, that would be awesome for them to keep being upgraded. Seems algoliasearch is lagging behind for example.

[jimaek]

Can you give me an approximate size?

[vvo]

Do you mean the size of all the instantsearch.js files, all versions, under this current repo? And what should be a good maximum?

[jimaek]

The size of a single npm package unzipped. Martin said its about 80MB. With the new changes you made that size stayed the same?

We are starting to actively work on this. So we will update this issue with our progress within this or next week.

[vvo]

> npm install instantsearch.js
> du -hs node_modules
34M    node_modules

Thanks a lot for being so responsive.

[MartinKolarik]

@Jerskouille I found four files where our current CDN version differs from npm. The differences seem to be only in formatting and comments (e.g. trailing LF character). Handling those cases individually would be too complicated, and I believe unnecessary. However, if anyone uses SRI with those files, this change is going to break their website. Is that ok with you?

Affected files:
https://cdn.jsdelivr.net/algoliasearch/2.6.6/algoliasearch.min.js
https://cdn.jsdelivr.net/algoliasearch.helper/2.0.0-beta/algoliasearch.helper.min.js
https://cdn.jsdelivr.net/algoliasearch.helper/2.0.0-beta/algoliasearch.helper.js
https://cdn.jsdelivr.net/docsearch.js/2.3.3/docsearch.min.js

[Jerska]

@MartinKolarik It's definitely ok. I'm not sure we've ever advised anyone to use SRI, and I have no example coming to my mind.

• [email protected] should probably not be used
• docsearch, we provide a snippet that our users almost never modify
• On the amount of people using the pinpointed version of algoliasearch@2 which should be fairly small, I'm betting that no-one is using SRI, and if they are, I'll be ready to handle the support.

Feel free to override those files. :)

Thanks a lot again for taking care of this!

[jimaek]

We deployed this change to Heroku for testing.
https://jsdelivr-origin-dev.herokuapp.com/algoliasearch/2.6.6/algoliasearch.min.js

Can you use it to test all of your use-cases and make sure it works correctly?

[Jerska]

@jimaek This is working fine!
Thanks again for taking time on this and making sure nothing breaks!

[jimaek]

@vvo Can you confirm its good to go as well? Just to be sure :)

[vvo]

Nobody's using integrity check for now, this is also a good idea.

We are good, 🔥

[jimaek]

Ok, deploying to prod.

[jimaek]

Its live. Run your tests again and let me know if there are any problems.

[Jerska]

Thanks a lot to the two of you for making this happen!

[MartinKolarik]

Btw @Jerskouille, I know people mostly use the code snippets provided by you, but if you could set default files for your packages, it would be awesome 😄

[Jerska]

@MartinKolarik Wow, I hate to miss out on notifications like that! Sorry about that.

Actually we do use them! Do you remember which project you were talking about? :)

For the "shorter url", does it mean:

• package@version/ or
• package@version/filename_without_its_folders_hierarchy_above as jsdelivr used to provide before?

If it's the first option, since we're often offering both a CSS and JS file, then it would be a bit weird to have two really different looking urls.
If it was the second one, then I'd be happy to remove the dist from the URLs I provide, but it'd mean that this feature needs to support arrays (I know main doesn't, but you might have it on the jsdelivr field).

[MartinKolarik]

@Jerskouille I meant the first option. It would work well for algoliasearch, algoliasearch-helper, autocomplete.js, and places.js I think?

I understand that if you have both JS and CSS you'd like the URLs to look similar. In theory we could easily add another option for configuring a default css file (I wanted to do that a few years ago), but I'm not sure how many people would actually use it (since it needs to be configured by package author).

[Jerska]

If at some point the CSS default URL comes in your roadmap, I'm sure we'd be happy to use it, but it's definitely not a priority for us.

About the shortened URL, it seems like my colleagues are aware of it : https://community.algolia.com/places/documentation.html 👍

[MartinKolarik]

About the shortened URL, it seems like my colleagues are aware of it : https://community.algolia.com/places/documentation.html 👍

Ok, cool. It wasn't configured yet when I checked last month 👍

[Jerska]

I do have learnt something here, so this was definitely not a vain comment! Thanks again for all of that @MartinKolarik and @jimaek !

[Haroenv]

Hello, thanks again for all you're providing with JSDelivr! We really like it.

I'm noticing that the page for algoliasearch displays v3.22.1, but the current version is 3.24.1, is there a manual step in the deployment of new versions? We're planning to push a new major version in the coming months, thanks!

Maybe it's a caching issue since the link definitely exists!

[jimaek]

The website is outdated. We are working on the new version that will pull information directly from npm. Plus it will have download stats for all projects.

It should be live next month. Sorry about that.

[Haroenv]

No need to apologise, that means that when that will be updated, the link

https://cdn.jsdelivr.net/algoliasearch/3/algoliasearch.min.js

will point to

https://cdn.jsdelivr.net/algoliasearch/3.24.1/algoliasearch.min.js

instead of

https://cdn.jsdelivr.net/algoliasearch/3.22.1/algoliasearch.min.js

is that right?

[jimaek]

No, the website has nothing to do with the CDN.

/3/ aliasing is just cached. It would expire in a few days by itself but I purged it manually right now.
You should be getting 3.24.1 now.

A few months ago I sent instructions to @vvo on how algolia can purge our cache on-demand using our API. You can ask him or I can email you too.

[vvo]

Thanks a lot, we have that purge URL we just wanted to be sure it was all good now.