Rules_nodejs: [feat] rollup dev server rule

[Toxicable]

🚀 feature request

Relevant Rules

Related to rollup_bundle and ts_devserver

Description

Following the discussion in #1188 it's clear that ts_devserver while being fast is not very compatiable with the wider npm ecosystem as it stands.
A way we can solve this compatiablity issue is by introducing a new rule which is more compatiable with the trade off beign performance.
We can then suggest that users start on this new rule, but possibly move to ts_devserver when they start scaling massively.

Describe the solution you'd like

A new rule: rollup_devserver
This should use the plugin: https://github.com/thgh/rollup-plugin-serve
Tracking this issue: https://github.com/rollup/rollup/issues/2385 for integration into rollup core.

The requirements of this rule are

• Consistancy with rollup_bundle
  
  
  
  • Use the same or derived config file
  
  
  • A build should never pass in one and fail in another (or vise versa)
  
  
• Accept all module formats that have been configured by plugins
• "live reload" the webpage when a new bundle has been produced

[jmaher409]

It seems like it would be possible to do something in between. Perhaps a new parameter could be added to ts_devserver that lists the npm packages that need to be rolled up into a single external dependencies bundle. Those dependencies could then be loaded from that bundle. It could even be taken a step further by writing a module resolver that knows about the external dependencies bundle and knows to load the shimmed dependencies from there. This way, we don’t lose the fast turnaround for code that changes often (source code) and only hit the bottleneck for a subset of external dependencies.

I’m stuck at this stage of integrating bazel as well and have been thinking about taking such an approach.

[Toxicable]

We're currently trying to remove our custom module resolver, I don't think it's a good idea to write another one.
While I like the idea of an in-between option, I don't think it'll work as well in practice as each stand alone solutions.
Ts_devserver is written in go and would now have to host two bundlers, also I believe it's source is in google3, but I don't think google3 has any reason to want this feature

[jmaher409]

Ahh ok that makes sense then. I’ll probably still keep trying to make
ts_devserver work for us as we are building a library so have few external
dependencies.

On Sat, Sep 28, 2019 at 9:37 AM Fabian Wiles [e-mail]
wrote:

We're currently trying to remove our custom module resolver, I don't think
it's a good idea to write another one.
While I like the idea of an in-between option, I don't think it'll work as
well in practice as each stand alone solutions.
Ts_devserver is written in go and would now have to host two bundlers,
also I believe it's source is in google3, but I don't think google3 has any
reason to want this feature

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_nodejs/issues/1209?email_source=notifications&email_token=ABYSF6CVZ625ST2ALGVBBGTQL6B23A5CNFSM4I3OR5NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725PFA#issuecomment-536205204,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABYSF6AOAEO6HMBO3RBGSXLQL6B23ANCNFSM4I3OR5NA
.

[alexeagle]

Have you tried just doing bazel run on the rollup binary with that server plugin? I know it won't speak the ibazel watch protocol but it would be good to understand if something blocks you using some arbitrary npm solution without needing custom Bazel rules

[Toxicable]

I'd assumem it should just work out of the box with npm_package_bin, but due to lack of ibazel, I assume it would be slow, ill give it some experiementing.
I do think this should be a first party rule, due to it's integration with ibazel

[alexeagle]

@lewish is working on making webpack-dev-server work - I would guess we should let that proceed and then we can model rollup on whatever we get working.

[Toxicable]

Aight, happy to help if any other input is needed. This is one of the bigger blockers for us due to npm 3rd party support

[yohcop]

I would be pretty interested in that. Maintaining 2 configs, rollup + webpack, would be annoying. Rollup seems to be producing smaller bundles, at least in my case, and the config is easier to deal with.

I tried yesterday to get it to work with bazel run as suggested, and the serve plugin (even without ibazel). AFAICT, I can't use npm_package_bin as no output is expected. From doc:

If you just want to run a program with bazel run, use the nodejs_binary rule.

Using a nodejs_bin, I had some difficulties (Basically, nothing was happening... the sh script was written but running it would just immediately exit.) It's possibly that I just haven't found the correct data and flags to pass just yet.

All that to say, I think it would be interesting to have this working, but I understand it may duplicate some webpack work. But if you have some wild guesses about what I could try to make it work, let me know. If i also manage to make it work, I'll share the results.

[yohcop]

I have something "working" ish. But I can't figure out the last piece, so maybe the answer is trivial, and somebody can tell me "oh yeah, just depend on :foo_es6" or something like that.

I have this rule:

nodejs_binary(
    name = "rollup-dev",
    data = [
        "index.html",
        "rollup.config.js",
        "//my/app",
        "@npm//rollup",
        "@npm//:node_modules",
    ],
    entry_point = "@npm//:node_modules/rollup/dist/bin/rollup",
    templated_args = ["--config"],
)

and bazel run :rollup-dev builds a bundle.js, and serves it.

I think the interesting part of rollup.config.js is:

export default {
  input: "my/app/index.js",
  output: {
    file: "bundle.js",
    format: "iife"
  },
  // ...snip ...
}

However, the //my/app:app target is a ts_library rule, and it produces an index.js file in es2015 format (forced by rules_nodejs I believe), and it starts with

(function (factory) {
    if (typeof module === "object" && typeof module.exports === "object") {
        var v = factory(require, exports);
        if (v !== undefined) module.exports = v;
    }
    else if (typeof define === "function" && define.amd) {
        define("my/app/index", ["require", "exports", /*...snip...*/ "my/app/root/index"], factory);
    }
})( /* ... snip ... */ });

That code is included as is in bundle.js produced by rollup, and neither if branches are true when executed in the browser (with a simple <script src="bundle.js"></script> at least). So... nothing happens.

Also of note, the served bundle.js doesn't include any dependencies. When using rollup_bundle, everything is there.

So I'm probably using this wrong. I tried for example input: "my/app/index.mjs" in rollup.config.js, but the file can't be found at runtime by rollup.

Any ideas, or questions? Thanks!

[alexeagle]

It would probably be useful to have your working hack to reference as we build the real thing. A draft PR or just a pointer to the branch.
@soldair had the idea that we call node with a --require script that shims the ibazel protocol by intercepting stdin and calling the fs watch API so it could work with any tools watch mode

[alexeagle]

Playing with this a bit:

• Yes, Bazel un-links things in the output tree, so a naive FS watcher will have trouble. Anyway you don't want to trigger until the build has finished, the output tree gets updated as each action completes.
• if you install the rollup-plugin-serve and call the serve() method in the rollup plugins, the node process will never exit. This means bazel build :bundle hangs forever if you have this in the same config you use to build your distribution package. Unfortunate.
• the rollup_bundle rule from @bazel/rollup has some nice affordances like knowing to ask rules upstream of it for their JSEcmaScriptModuleInfo provider so that Rollup gets the more optimizable JS from ts_library. So we'd like to keep using that rule for the devserver use-case. It doesn't return an executable so Bazel doesn't let us bazel run :bundle but it totally could.
• we could wrap the installation of the serve plugin with conditional logic that senses whether we're under bazel run (look for process.env['BUILD_WORKSPACE_DIRECTORY'] or similar)

[yohcop]

So an update on this. I never managed to get this working nicely with rollup dev server. However I found some sort of workaround.

I now build the rollup file as normal.

Then I use a genrule like this, to copy the files to some random directory at the end of the build, say /tmp/build

genrule(
    name = "pack_dev",
    srcs = [
        ":bundle",
        "index.html",
        "styles.css",
    ],
    cmd_bash = ("mkdir -p /tmp/build; " +
                "rm -rf /tmp/build/*; " +
                "for f in $(SRCS); do " +
                "  if [ -d $$f ]; then " +
                "    cp $$f/* /tmp/build; " +
                "  else "+
                "    cp $$f /tmp/build;"+
                "  fi; " +
                "done; " +
                "md5sum /tmp/build/* > $@; "),
    outs = ["list"],
)

(it flattens the directory created by the :bundle rule here)

And then I can use pretty much any of the js server with auto-reload. For example

bazel run @npm//light-server/bin:light-server -- -s /tmp/build -p 8080 -w "/tmp/build/*"

with

ibazel :pack_dev

Now this works: ibazel can rebuild the bundle on changes, and then this BUILD rule copies the output to /tmp/build (that's a side effect, so it also generates a file with the hashes of the input files in it....). The directoy is then picked up by light-server, and the page reloaded.

The only not-so-great experience is that actually bundling the code takes quite a bit of time for large projects. I wonder if there would be a way to re-use the ts_devserver logic that concatenates all the js files into one? Then I could write that concatenated file to /tmp/build ?

Thanks!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs in two weeks. Collaborators can add a "cleanup" or "need: discussion" label to keep it open indefinitely. Thanks for your contributions to rules_nodejs!

[github-actions[bot]]

This issue was automatically closed because it went two weeks without a reply since it was labeled "Can Close?"