Esbuild: can we use Svelte in esbuild?

[hmmhmmhm]

Is there any way to use a front-end library like Svellte in this bundle?

[evanw]

Caveat: I'm not at all familiar with Svelte.

From an initial read of the project, it appears that it uses Rollup to do the bundling. The core of what Rollup does is indeed very similar to esbuild. However, it looks like Svelte integrates with Rollup through a plugin.

This won't work with esbuild because it's written in Go and isn't built to be extensible. I'm only intending for esbuild to target a certain sweet spot of use cases (bundling JavaScript, TypeScript, and maybe CSS). I don't think Svelte is mainstream enough to warrant building into the core of esbuild, and since esbuild doesn't have plugins it won't be possible to add Svelte support to esbuild.

[hmmhmmhm]

ooh...thanks for the answer..!

[sw-yx]

@evanw just wondering, is there some deeper philosophical reason why this project is not extensible (eg, unacceptable tradeoffs in perf) or is it just a matter of you didn't want to bother with it but yes it is theoretically possible? great proof of concept btw

[c2h5oh]

@sw-yx go is a compiled language, so adding plugins loaded at run time is not easy.

[evanw]

That's a good question. A few reasons:

• This project is already very ambitious and I want to limit the scope, since it's only a side project of mine. I also don't want to turn this into a big community project because I don't enjoy maintaining those.

• I want to use this project as an existence proof to the community of how fast JavaScript build tools can be. Plugins would complicate the architecture and distract me from building the MVP I'm trying to get to. Perhaps when it's done, another bundler that supports extensibility could be built using parts from this one.

• Plugins are harder in Go than they are in JavaScript. I thought they were impossible but apparently things like https://golang.org/pkg/plugin/ exist (although it doesn't work on Windows). I think if you did want to do "plugins" in Go, it might be better to organize plugins as a set of Go libraries that you can compose easily to compile your own build tool instead of trying to load libraries at run time.

[tooolbox]

I also don't want to turn this into a big community project

If you receive PRs, will you accept them?

apparently things like https://golang.org/pkg/plugin/ exist

From what I've read, go plugins are not really suited to the use case being discussed.

I think if you did want to do "plugins" in Go, it might be better to organize plugins as a set of Go libraries that you can compose easily to compile your own build tool instead of trying to load libraries at run time.

This is one strong candidate. If esbuild can be structured as a main() wrapping a set of libraries, (which it already seems mostly there) then anyone can come along and write a replacement main() with a superset of those libraries.

The other option would be using an embedded scripting language like https://github.com/d5/tengo or JS to do AST manipulation and so on. Haven't studied the esbuild source enough to know if that's a good idea but my gut says no.

[evanw]

If you receive PRs, will you accept them?

That depends on the PR. I'm guessing most likely no unless it's a small bug fix, at least not until this project is more mature. Right now esbuild hasn't even reached the minimum feature set that I want it to have. I want to keep esbuild as a personal project for now.

[akaibukai]

From an initial read of the project, it appears that it uses Rollup to do the bundling. The core of what Rollup does is indeed very similar to esbuild. However, it looks like Svelte integrates with Rollup through a plugin.

This won't work with esbuild because it's written in Go and isn't built to be extensible. I'm only intending for esbuild to target a certain sweet spot of use cases (bundling JavaScript, TypeScript, and maybe CSS). I don't think Svelte is mainstream enough to warrant building into the core of esbuild, and since esbuild doesn't have plugins it won't be possible to add Svelte support to esbuild.

What the rollup plugin will do at the end of the day is simply to call the Svelte compiler https://svelte.dev/docs#Compile_time (at least it's what I guess looking at the plugin source code) which is still JS code...

I don't know how you are handling the JS bundling but are you able to "call" JS functions (maybe using system call with node)?

I guess that in this case the heavy lifting will still happen in the JS Land instead of Go, so the benefits may not be as huge as what's in the benchmark of the readme..

But in any case I would love to see such improvement when working with Svelte...

[kazzkiq]

I guess that in this case the heavy lifting will still happen in the JS Land instead of Go, so the benefits may not be as huge as what's in the benchmark of the readme.

Then there is no benefit in adding Svelte to esbuild. If you're basically behaving the same as in Rollup (calling the JS compiler), I can't see much gains from it right now.

I guess it would be different story if we managed to write a Svelte compiler in Go? But then again, we still depend on plugin support in esbuild, and before that we still depends on it reaching maturity, and also project owner deciding to accept extensibility to the project (or anyone else using esbuild as core to build a more robust build tool in the future). Many "ifs" for the time being.

[akaibukai]

I can't see much gains from it right now.

Well, in a project one can have other bundling needs than Svelte, and even CSS stuff needs..

[weberc2]

I understand that the maintainer doesn't want to implement an extension architecture at this time, but I think it's worthwhile to discuss the possible implementations since the stated goal of this project is to inspire other projects (discussion about how to make it more extensible would presumably also serve similar projects). Some options that have been discussed include:

• Embedding a scripting language
• Making the project pluggable at compile time (modifying the entry point to support e.g. svelte and then recompiling esbuild on a per-project basis)
• Using the Go dynamic linking plugin package (note that I'll use 'plugin' to refer to this dynamic linking approach and 'extension' in the generic sense of something that extends esbuild).

The plugin/dymanic-linkage approach doesn't work on Windows IIRC and is generally more hassle than it's worth. I'm not familiar with any open source projects who employ the plugin package approach successfully.

The compile-time extension puts a big burden on users, especially new users who already have to learn the other aspects of esbuild. However, this is the approach that Caddy takes as far as I can tell. IIRC they even have/had a download page that would let you choose your combination of extensions and the server would compile them in on the fly such that you would be served a precompiled binary with exactly the right set of extensions--this is technically interesting but probably less ergonomic at least given the nature of a project's build tool (you probably want coworkers, new contributors, etc to be able to build your project without having to download a binary with just the right set of extensions compiled in).

Further, both of these approaches have the advantages and disadvantages of needing to be written in Go (on the plus side, the ecosystem will likely be much more performant because Go is performant and because all communication between core and extenisons can happen via shared memory instead of IPC, etc; on the downside, it would be an additional barrier of entry for potential extension developers).

The embedded scripting language approach is an interesting one, but usually this approach becomes too constrained and projects which start with an embedded scripting language extension interface eventually move to other models. One notable limitation is that the scripting languages tend to be slow and blocking, which means that other things can't advance. I believe this was a big motivator for Neovim (and maybe Vim now?) to support an async/IPC interface. This seems particularly relevant to a build tool on the assumption that these plugins are likely to be I/O bound (but maybe that's a bad assumption?). Another concern is that, like the first two options, the community must learn a language that they likely aren't familiar with (although conceivably you could embed JS/TS).

There's at least one other approach, which is to use the IPC interface (basically start up a server process for each plugin that communicates with the main esbuild process). As previously mentioned, that's what Neovim does as well as Terraform and probably several others. IPC is slower than shared memory, but there are serialization formats that make this cost negligible, and anyway I don't know how much serialization overhead would matter with respect to an application like esbuild. IPC interfaces are complex (mostly with respect to process management), but probably simpler on balance than the alternatives, and it has the distinct advantage of allowing plugins to be developed in whichever languages the community prefers.

[evanw]

There's already a work-in-progress plugin implementation on a branch. It's fairly complete and works end-to-end but I'm still iterating on the design a bit to make sure various use cases are satisfied. It uses the same IPC approach as the current createService API, which streams messages over stdin and stdout to a child process using a simple binary protocol. This means you can write plugins in JavaScript (there's also a Go plugin API if you'd like). You can follow along on issue #111 for updates.

I haven't used Svelte personally so I'm not sure how involved supporting it is, but I definitely have this use case in mind and I've made a simple Svelte plugin that seems to work. Here's what it looks like using the WIP plugin API:

let svelte = require('svelte/compiler')
let path = require('path')
let util = require('util')
let fs = require('fs')

// import value from './example.svelte'
let sveltePlugin = plugin => {
  plugin.setName('svelte');
  plugin.addLoader({ filter: /\.svelte$/ }, async (args) => {
    let convertMessage = ({ message, start, end }) => ({
      text: message,
      location: start && end && {
        file: filename,
        line: start.line,
        column: start.column,
        length: start.line === end.line ? end.column - start.column : 0,
        lineText: source.split(/\r\n|\r|\n/g)[start.line - 1],
      },
    })
    let source = await util.promisify(fs.readFile)(args.path, 'utf8')
    let filename = path.relative(process.cwd(), args.path)
    try {
      let { js, warnings } = svelte.compile(source, { filename })
      let contents = js.code + `//# sourceMappingURL=` + js.map.toUrl()
      return { contents, warnings: warnings.map(convertMessage) }
    } catch (e) {
      return { errors: [convertMessage(e)] }
    }
  })
}

This plugin can then be passed to the build API call like this:

```js
const { build } = require('esbuild')

build({
plugins: [sveltePlugin],
...
}).catch(() => process.exit(1))