Server: Use npm & webpack to install & bundle server dependencies & scripts

[ChristophWurst]

Is your feature request related to a problem? Please describe.
The current situation is

• server vendor scripts are managed and installed via the long deprecated tool Bower
• vendor scripts are bundled with our own, fragile bundle script
• apps (both in this repo and outside) started proper bundling and ship their own dependencies, hence get more and more independent of server

The still leads to the following problems

• Since bower is deprecated we should get rid of it
• Our js dependencies are quite outdated, some have serious issues
• Our bundler does not spit out a source map, hence debugging is unnecessarily hard
• Our bundler is not as efficient as a tool like webpack that understands the AST and performs optimizations like tree shaking
• We still pollute the global window object with our dependencies
• Having dependencies globally makes it tempting for app devs to use them, preventing us from updating them without potential backwards compatibility breaks
• Many source files slow down page loads because we cannot rely on bundling for all scripts yet

Describe the solution you'd like
I would like to have

• npm for dependencies
• webpack for bundling
• babel for transpilation

This is also the stack we use for the Vue components and all newly rewritten apps. It works really nice and seems to be well understood.

For backwards compatibility and low-friction migration I would propose the following step-by-step plan

Step 1: replace bower with npm, add webpack

Dependencies should be moved to npm without changing the version number. The only exception right now would be jQuery because as far as I know we have custom patches applied. An alternative would be to temporarily release this custom jquery version as nextcloud-legacy-jquery on npm to complete the migration to npm and remove all Bower leftovers. This migration of our own, patched jQuery is a discussion of its own.

Webpack shall be used to produce a single bundle that just contains the vendor scripts to get started. For backwards compatibility there must be a tweak to expose them globally.

Step 2: add server scripts to webpack bundle

Now we can start adding our scripts to the webpack bundle. Again, for backwards compatibility we have to make sure everything that was global before has to be exposed globally again, like OC and OCA.

Internally, we should take this as a chance to organize all these crazy APIs that developed over time and start splitting them into smaller, manageable chunks (looking at you js/js.js).

At the same time we should mark legacy APIs as deprecate in order to build a real public API at some point. This is also a discussion of its own.

Step 3: update dependencies

This is the step with the highest likelihood to go :boom: but this is inevitable.

Describe alternatives you've considered
Ignore the fact that our current approach has reached a dead end.

Additional context
Possibly related tickets

• https://github.com/nextcloud/server/issues/8425

---

@nextcloud/javascript please let me know what you think. This is a very rough idea of how I could image this works. I actually have a local PoC branch where I tested step one. So far this seemed to work well but I wanted some input just to know that it's hopefully not a stupid idea before I dig any deeper.

[rullzer]

Yes please :)