Sapper: Snowpack (or ESM) for dev mode

[jthegedus]

Is your feature request related to a problem? Please describe.
Dev mode is slow to load changes. Snowpack has proven to be a great development mode alternative.

Describe the solution you'd like
It would be great to get ESM ~Snowpack v2~ in the dev mode for Sapper.

Describe alternatives you've considered
NA

How important is this feature to you?
Deciding between Sapper or rolling https://github.com/pikapkg/create-snowpack-app with the svelte plugin is hard when the developer experience is so much nicer.

OOTB features vs DX + hand-rolling is a hard call to make.

IE:

• Sapper vs
• Snowpack + Routify + other

Additional context
NA

[benmccann]

Snowpack is a bundler alternative to Rollup/Webpack?

@rixo was working on the hot reloading and might be interested in either trying this out or any feedback you have on the existing plugins

[jthegedus]

It serves ESM in dev mode and bundles with ~Rollup~ parcel-bundler for production mode. For context, Rich Harris talked about trying it out on Twitter recently https://twitter.com/Rich_Harris/status/1257778203290226689?s=20 and it's the same concept that is currently powering Vue's Vite project: https://github.com/vuejs/vite

I've no idea what would be entailed in adopting either Snowpack directly or moving to ESM for developer mode in Sapper, but the DX is night and day. I'm unsure if the Snowpack ~Rollup~ can be extended in the same way existing Sapper users would expect (but it is getting svelte-preprocess support)

[benmccann]

Oh, that's interesting (and a better explanation of what it is than their website provides!)

bundles with Rollup for production mode

It actually looks to me like it bundles with Parcel for production mode, but uses Rollup during development mode. Their explanation isn't quite clear to me though

I also see svelvet, which is built on snowpack. I'm not sure the difference between it and snowpack (https://github.com/jakedeichert/svelvet/issues/86). From, reading the readme for svelvet I wonder if it would support serving images over a CDN (https://github.com/jakedeichert/svelvet/issues/85). If svelvet doesn't then I also wonder if snowpack doesn't.

it is getting svelte-preprocess support

Where do you see that it's not supported? Is there somewhere we can track it? svelvet already supports it. The big issue svelvet has at the moment is that it doesn't support third-party components, but that looks like it should be fixed soon since there's a pending PR https://github.com/jakedeichert/svelvet/pull/68

Anyway, I'm not a sapper expert, but I don't see why you couldn't use snowpack with sapper today. It also sounds to me like you could use snowpack just in dev mode and then use the traditional rollup/webpack build tools for production: https://www.snowpack.dev/#leaving-snowpack https://github.com/jakedeichert/svelvet/issues/84. And in fact that probably would be a better idea: https://github.com/jakedeichert/svelvet/issues/83

[jthegedus]

Svelvet was built on Snowpack v1 and the Svelvet author has discussed not further developing the project (due to ESM not being as performant as a bundled build). Snowpack v2 is yet to come out of beta, and it's integration for Svelte file compilation is what I was referring to getting svelte-preprocess support (which should bring it up to par with Svelvet, but without the drawbacks as far as I can tell).

You are right, I misread Rich's Twitter thread correction. It uses Rollup on install of packages and then Parcel for production build.

I do not know how using Snowpack in dev would effect the SSR aspects of Sapper while developing.

The updated issue title of "(or ESM)" is probably a better target for Sapper than using Snowpack itself.

[antony]

I actually started a PR allowing Sapper to use Snowpack as a bundler, so if somebody wants access to the branch to continue, I'm more than happy to push it.

In discussions with @rixo it came to light that maybe nollup would be an easier, and equally fast option for building Sapper apps. I think this would be considerably easier to integrate.

[jthegedus]

My concerns for raising this issue are around DX speed and ease of use, if nollup gives us the same benefit with much less hassle then it sounds like a win.

While I suspect most Sapper users like it's use of Rollup for ease of extending it's configuration, would using nollup and mixing it with the existing Rollup config make this more difficult, or just as easy?

[pngwn]

This doesn't seem like a workable solution to me. Even if we could get things working nicely, we would need to maintain configuration for both snowpack and rollup/webpack (snowpack would only be used in dev mode, rollup/webpack for building) unless we rewrote Sapper to work with Parcel (which isn't going to happen). This dual config maintenance would also be pushed onto the user.

Rixo has spoken about getting many of the same benefits that snowpack brings via nollup and that probably seem like a better approach for now. Snowpack v2 hasn't been released yet, we can possibly revisit this when that changes.

[jthegedus]

Thanks, the nollup route makes sense. Is there an issue raised I can follow for the nollup progress or for others to :+1: their support ?

[antony]

I don't think one exists. Feel free to raise it, I'll push my branch in an
hour or two.

On Sun, 17 May 2020 at 11:01, James Hegedus [e-mail]
wrote:

Thanks, the nollup route makes sense. Is there an issue raised I can
follow for the nollup progress or for others to 👍 their support ?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/sveltejs/sapper/issues/1204#issuecomment-629772539,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABVORJ7GOAHFVAIIS776XLRR6YW7ANCNFSM4NDERPQQ
.

>

---

ꜽ . antony jones . http://www.enzy.org

[FredKSchott]

Hey all, lead maintainer of Snowpack here! Just found this thread, but wanted to ping to say I'm happy to help prioritize anything blocking a Sapper + Snowpack integration, or anything else Svelte + Snowpack related.

Edit: just realized a comment above is outdated. Snowpack supports plugging any bundler into your production build, with official plugins for both Webpack and Parcel. By the time the bundler sees your code though, it's already been compiled from .svelte -> .js, so hopefully you're not actually blocked by the need for a Rollup bundler (although an official Rollup bundler plugin should be added soon).

[dionysiusmarquis]

I fiddled around a bit to see If I can get Snowpack up and running for the client part of Sapper.
I used the sapper dev commad to create an sapper app under src/node_modules/@sapper/…. Then I used the snowpack dev command to serve the client part of the Sapper app. That actually worked after I changed all client related file extensions from .mjs to .js (is it possible to use .mjs in Snowpack context?).

As far as I can tell following points could make Snowpack usable as a dev server forSapper:

• The Snowpack dev server should be able to handle middlewares
• It should be possible to watch/build only the Sapper app part without triggering any bundler (and the app should only be updated if really neccesary)

The pipeline would be:
[Snowpack dev server request] -> [Sapper middleware SSR (app/server)] -> [Snowpack file serve (app/client)]

To enable Snowpack the ability to execute the Sapper server app middleware it might be necessary to build the server part via Rollup or Webpack first, because it also has to resolve Svelte files.

Here is a repo with the very first litte step:
https://github.com/dionysiusmarquis/sapper-template-rollup/tree/poc/snowpack

The already build Sapper client app lies under src/snowpack/@sapper/…. Just run yarn snowpack and the index page will be rendered with the files served via Snowpack incl. hmr. As the whole Sapper server part is skipped/missing it's not possible to navigate.

Maybe this POC helps a little.

[benmccann]

The .mjs thing is annoying for TypeScript too: https://github.com/sveltejs/sapper/issues/760#issue-458770734. I wonder if we might be able to switch it to generate .js and use "type": "module" in package.json instead

[Conduitry]

I don't remember the exact reasoning that went into this file naming scheme, but I seem to recall it being because webpack was doing something weird that they didn't consider it a bug. https://github.com/sveltejs/svelte/pull/2887 and https://github.com/sveltejs/svelte/pull/2992 were two PRs where some of this stuff changed, but I don't know how helpful those PRs or associated issues are for this discussion.

[rixo]

hopefully you're not actually blocked by the need for a Rollup bundler (although an official Rollup bundler plugin should be added soon)

@FredKSchott Really happy to hear about the any bundler support, and that Rollup plugin is planned. I think Snowpack will be a terrific solution once the perspectives it opens have been explored and exploited by its ecosystem.

@jthegedus @pngwn @antony Actually I think this issue should have remained open. The no-bundler / closer-to-the-source approach has a lot of potential, and will surely get a lot of interest. Even if we don't tackle it immediately, I think it makes sense to keep the discussion and the work around how it would fit with Sapper open.

[jthegedus]

@rixo @FredKSchott 's update around Rollup support (as a bundler) coming to Snowpack is what tips me over the edge with reopening this.

I'm not super familiar with Sapper's codebase (yet) and am unaware what difficulties using Snowpack's dev server & compilation model would have on the existing features like pages, server routes etc. But Rollup support seems to make any migration easier and can perhaps mitigate potential issues.

[dionysiusmarquis]

I also think it‘s a good idea to keep the issue open. I have a few other ideas to integrale Snowpack into the current Sapper compiler system. But one thing I was wondering while thinking about it: Is there any way to directly run the server app without bundling 🤔 ? We might run the client part via Snowpack Imports and web_modules. But if we need to bundle the SSR part anyways it feels a little bit “half way”.

[dionysiusmarquis]

I have a prototype running snowpack dev as the compiler. The compiler watch acts as Snowpack hmr client and listens to import.meta.hot.accept. As soon as any update message is received the sapper compile pipeline is triggered, just like the other bundler watch hooks. The client/*.js are directly proxied to snowpack dev. The server part is executed via node esm and pirates hooks for the .svelte files. The only two files that are written/bundled are build.json and service-worker.js. Everything else is served by Snowpack or running "natively" in node.
The Snowpack compiler compile hook for production builds will use a Rollup or Webpack compiler at the moment.

Currently one thing that would make the implementation more straight forward:

• esm-hmr Node client. Currently there is only a browser compatible version

Things that preventing the Snowpack compiler from acting excactly like Rollup and Webpack compiler

• Snowpack can not resolve relative node_modules folder (src/node_modules/@sapper/* imported via @sapper/*)
• Snowpack will install modules inside node_modules folder to web_modules (including src/node_modules/@sapper/* if they would be resolved). They will not be updated, that's expected behavior but also makes node_modules/@sapper folder structure not compadible with Snowpack
• .mjs will not be processed by Snowpack https://github.com/pikapkg/snowpack/issues/47
• service-worker.js has to be bundled because { type: "module" } is not jet implemented for Service Worker - only Web Worker https://bugs.chromium.org/p/chromium/issues/detail?id=824647

The "not bundling the Server" approach:
I can't say what such an approach will be to dev vs production. One thing that is significant different in terms of speed is that Node commonjs require is always sync and currently I compile the .svelte files parallel with @snowpack/plugin-svelte on demand. This makes the current "native" startup actually slower than running an already build script with cached chunks etc.

The current overall changes to get it running:
I used sapper dev --output src/@sapper and changed the imports from @sapper/* to ./@sapper/* inside the src/*.js files.
First I thought I could use the Snowpack hmr directly inside the browser and restart the server in the background (Just like the Webpack compiler, I think). The problem is that the service-worker.js might be out of sync because it can't be served by snowpack dev. So I used theSapper live feature just like the Rollup compiler. Then the page reload will be triggered after the service-worker.js is bundled and the server is restarted. This way everything is in sync but we will lose the benifit of not bundling, because we have to wait for the "slow" server bundling/"native" startup.

I want to prepare an example repo.
Which alternative is the most interessting for you as example repo?:

• Direct Snowpack hmr _(fastest but service-worker.js out of sync )_
• Waiting for service-worker.js bundling and "native" server restart _(slowest but everything in sync)_
• Only waiting forservice-worker.js bundling - no server restart _(fast but server part out of sync)_

[antony]

@dionysiusmarquis nice breakdown! If you are able to publish this anywhere I'd be happy to try it out.

[FredKSchott]

_Update (10/13/2020): This config format is outdated. See https://www.snowpack.dev/#api-reference for the latest._

Awesome breakdown. That all seems correct from my end. A couple extra notes:

• We don't support .mjs by default, but it should be possible via some custom config. I can add instructions if it would help.
• Not sure that I fully understand how Sapper does things, but It may be possible to mount the @sapper directory directly out of your node_modules, so that it's treated as source code instead of as a dependency. Could be worth exploring:

"scripts": {
  "mount:sapper": "mount node_modules/@sapper --to /@sapper"
}

// now, this should work:
import * as foo from '@sapper/foo';

[dionysiusmarquis]

@FredKSchott thanks for the input!
Currently I made Sapper to output the app files in .js Format if the bundler is Snowpack. Can't really say what the prevered/best way would be. Sapper or Snowpack side.

I tried your mount script.
I get:
[error]: scripts[mount:sapper]: "--to @sapper" must be a URL path, and start with a "/"
So far import * as foo from '@sapper/foo' will always resolve to /web_modules/@sapper for me.

@antony I'll do my best to provide an repo soon 😎

[FredKSchott]

_Update (10/13/2020): This config format is outdated. See https://www.snowpack.dev/#api-reference for the latest._

Ah, that was my bad, it should be:

"scripts": {
  "mount:sapper": "mount node_modules/@sapper --to /@sapper"
}

[dionysiusmarquis]

It is actually src/node_modules/@sapper. It contains the Sapper generated app specific files.
If I mount it to /@sapper I will still get:

⠼ snowpack installing... @sapper/app
✖ Package "@sapper/app" not found. Have you installed it?

[FredKSchott]

Bah, nevermind I confused myself. That shortcut for imports matches by the location on disk, and not the final URL. So instead that would be import 'src/node_modules/@sapper/app' which... isn't what you were asking for (but could be a workaround for now) :)

[benmccann]

@Conduitry thanks for the extra info in https://github.com/sveltejs/sapper/issues/1204#issuecomment-639101030. There's not much discussion there and it looks to me like .mjs was being used prior to those PRs. From what I can tell, Webpack doesn't have very good support for .mjs files either (I see @lukeed having to do some extra work even now to support it in https://github.com/sveltejs/sapper/pull/1262). It's really a Node.js thing. Node.js used to require it to use ES6, but they no longer do. However, to put ES6 in a .js file in Node, you have to add "type": "module" to your package.json. I suppose it'd be considered a breaking change since people would have to update their package.json files, but maybe it's something we can consider for 0.28 since it'd help support both Snowpack and TypeScript in addition to being the more modern way to do things?

[dionysiusmarquis]

Here is an update:

src/node_modules/@sapper is actually usable as needed with Snowpack. It's possible to mount @sapper to /_dist_/node_modules/@sapper and it will be resolved and treated like source code.

The browser part is running well so far. I still skip the current Sapper hmr handling. As war as I know it is also not working with Webpack so far.
The pipe is:

[ File change ] -> [ `snowpack dev` notifies over ws ] -> [ `Sapper` compiler watch hook triggered  ] -> [ `service-worker.js` build ]/[ server runtime provides fresh `Sapper` middleware ] -> [ Browser `Sapper` dev client receives update message ]

Unbundled server runtime:
It was quite hard to find a way to not bundle the server part to get an actual benefit using Snowpack. The template project needed about 500ms to rebuild the server with each litte change (using Rollup). Using esm made the server restart over a second slower just for esm initializing. I started writing a caching system to only recompile changed .svelte files. Unfortunate there is no way to use shared memory between processes in Node, as far as I can tell. Meaning I was shifting the cache as JSON strings at every server restart (child_process.send will always be stringified JSON as far as I know). This made the restart faster but still slower than rollup because of esm initialization. So I hijacked @sapper/server middleware() function to freshly require the server part if snowpack dev reported an update. This way it's possible to request the latest version of the SSR without restarting the server. Combined with the .svelte cache (but not shifted between processes, because we don't need to restart the process) I wrote earlier I came down to about 70-100ms vs 1-2s unbundled.

The current state of running unbundled code in Node (as far as I can tell):

• esm seems to be the simplest and maybe "fastest" way, if you want to run the code just like it would with an already bundled .js file. Unfortunate I was not able to freshly require the Sapper server part (https://github.com/standard-things/esm/issues/874). Meaning I had to restart the process, wich was way to slow.
• @babel/register It seems to handling everything quiet as esm but also seems to have more overhead because of its transpiling nature. How to get .svelte files compiled was alot harder to find out than with esm. But clearing require caches and getting the last server version was no problem.
• native Node runtime with "type": "module". This is something I couldn't get my head around completely. But If you use "type": "module" it is the default behavior (following the latest esm specs) to only import relative imports with file extension. Meaning every package out there already using "type": "module" and are running fine in (I guess every) bundler and helper like esm/babel have to use relative imports ending with .js e.g. In consequence It's not really possible to run "type": "module" packages natively in Node until all packages you need for you application added file extensions to their relative import. Maybe I'm missing something, but this was the biggest bummer for me so far. You can follow one discussion about it here: https://github.com/nodejs/modules/issues/444

I will make some last tests and code clean up and provide an repo after that

[FredKSchott]

Yup, I think the required .js file extension is catching a lot of people by surprise. That, and the fact that import 'components/button' no longer resolves to import 'components/button/index.js for directories.

[dionysiusmarquis]

Finally two repos for an first POC how an Snowpack integration could look like:
https://github.com/dionysiusmarquis/sapper-template-rollup/tree/template/snowpack

Hopefully you just need to run yarn snowpack after yarn dev.
(The repo will use the https://github.com/dionysiusmarquis/sapper/tree/feat/snowpack sapper fork)

What's still not perfect:

• Even though I got Snowpack to resolve @sapper to /web_modules/@sapper, dependencies under src/node_modules/ will not be installed. My current workaround: /src/snowpack-install.js (this will force the installation, but you have to add all additional dependencies manually)
• sapper /src/core/SnowpackHmr.ts is a copy and paste version of esm-hmr /src/client.ts (with added reconnect ability). It would be nicer to have a Node client provided by esm-hmr
• The unbundled server runtime is currently not part of sapper. I'm not sure how to handle this whole topic. Currently it's part of the template (/runtime.js) and will be run by sapper dev if the bundler is snowpack
• sapper doesn't support compiler without actual compiling. As soon as one watch hook is triggered, the server, service-worker and client compiler must execute the hook callback, otherwise the restart will never be triggered. As A workaround I created a EventBus, wich will emit an event to the other compilers to trigger their watch hook callback without actually compiling anything.

It's very WIP and I did only test the development environment. At the moment another rollup or webpack config is needed to build for production. I never testet webpack. Rollup could work as intended so far.

[websocket98765]

I like the idea of dropping Webpack, because of my dislike of its complexity.

• But, it's worth mentioning that when Snowpack was mentioned to the Next.JS team, their response was that webpack 5 is much faster than webpack 4 ("sub-second rebuilds") and consequently they didn't think Snowpack is needed. So it may be worth benchmarking Snowpack & Webpack 5 for both dev and build, to ensure build doesn't become slower with Snowpack since that uses Parcel for the build.
• The other consideration: which would more easily allow fast refresh if Svelte eventually adds a fast refresh, instead of hot reloading.

[antony]

Snowpack uses Rollup, not Parcel. Webpack 5 might be faster than Webpack 4, but it's still Webpack, which is a vastly complicated bundler in comparison to Rollup, which was un-coincidentally, written by Rich.

So I think Webpack would be out of the running for us because it's wildly unnecessary, when we have such strong support for Rollup.

As to whether Snowpack or Svite or Rollup HMR or Nollup will be the bundler of the future, well, it really depends heavily on where support finds itself. Right now they all do an excellent job, so it's really just watching where the community puts its efforts.

[pngwn]

‘Fast refresh’ as enabled in nextjs is not particularly great. Feedback above a few hundred ms feels noticeably slow and I am commonly waiting for it to reload after making changes. The nextjs projects I am currently working on aren't particularly large or complex so who knows what it would be like in that case.

It is worth mentioning that the webpack comparisons are in the context or a multi-minute reload problem. A fraction of this not a high bar.

[FredKSchott]

Just to clarify: Snowpack doesn't use any bundler internally during development! We do let you connect your favorite bundler for your production build, and for that we recommend Webpack over Parcel.

Also, just because it was mentioned above, Snowpack also supports Fast Refresh: https://github.com/pikapkg/create-snowpack-app/tree/master/packages/plugin-react-refresh

I'd love to see a Webpack 5 vs. Snowpack speed comparison, but I'm skeptical that they could get anywhere close to Snowpack performance wise. Mainly:

• Snowpack serves unbundled code during development, so by definition we should always be faster than a bundled dev environment.
• Snowpack builds and caches files individually. If a file never changes, we only ever build it once and can then cache it forever.
• Webpack builds and caches entire bundles, so they need rebuild larger chunks every time a single file within changes.

[antony]

After playing with Snowpack for a while on some Svelte projects, I'm really enjoying the experience. However, @FredKSchott internally, you do use Rollup, is what I mean. Not for bundling, but certainly it is used under the hood for something.

Now the issue with this is - unbundled dev and bundled production is pretty controversial, but I'm totally happy with how it works, but switching to Webpack for production when Rollup is being used for development is definitely a brige too far. So what I've done is written my own rollup bundler for prod.

I think I can see a path forward to making a decent snowpack integration for Sapper, and I actually look forward to it, but there would definitely need to be some way to use rollup for the production bundling (happy to share my (pretty customised)) rollup production bundler if required.

[FredKSchott]

@antony I'd love to see your Rollup bundler if you can share it. It's something that's been requested several times now so we'd love to support it with an official plugin.

Edit: oops, you already posted it here: https://www.pika.dev/npm/snowpack/discuss/494

Thanks!

[antony]

It's here: https://www.pika.dev/npm/snowpack/discuss/494

It would probably need to be more generic to fit all cases, but it works for my projects so it should make a good base.

[benmccann]

There is ongoing work to support Snowpack + Sapper in an experimental project currently called Svelte Kit

[JoshuaJarman]

Thanks, for the link, @benmccann !

I just saw Rich's video on @svelte/next and how that is using snowpack by default and replacing sapper, cool stuff. 👍