Rules_nodejs: promote the js_library rule to the public API

[javatlacati]

I looked through the definitions file and couldn't find any nodejs_library definition, does that means that I can only have exactly one global BUILD and have a monolithic building process?

[alexeagle]

what would you expect such a nodejs_library rule to do?

If you want to compile libraries in isolation, use a compile-to-JS rule like ts_library
If you want to publish a package to npm, use npm_package

[an-kumar]

I think the point would be to have multiple nodejs packages that reference one another in the same repository. e.g:

/path/to/first/package
/path/to/second/package (dependent on first package)
/path/to/some/binary (dependent on second package; transitively dependent on first package)

[alexeagle]

@an-kumar yes, this is supported already

For TypeScript, you have

/path/to/first/package
  BUILD.bazel
    ts_library(name = "first", module_name = "first", srcs =["first.ts"])
/path/to/second/package
  BUILD.bazel
    ts_library(name = "second" module_name = "second", srcs = ["second.ts"], deps = ["//path/to/first/package:first"])
  second.ts
    import * as f from 'first';
/path/to/some/binary
  BUILD.bazel
    ts_library(name = "program", deps = ["//path/to/second/package:second"], srcs = ["program.ts"])
    nodejs_binary(name = "bin", deps = [":program"], ...)
  program.ts
    import * as s from 'second';

If you don't use TypeScript, this is still supported, but not yet documented because we aren't sure we understand the use case (why use Bazel if you're not running any compilers?)

https://github.com/bazelbuild/rules_nodejs/blob/befbd8961c18de98e254bea94db08ede78889f86/internal/js_library/js_library.bzl
lets you assign the module_name just like above TS example

[Globegitter]

@alexeagle Our use case would be to have shared code between different rollup bundles. Both would then just define that common js_library as a dep. Or would you there just directly depend on the files?

Edit: To be more precise, that particular use-case I am describing is in a monorepository structure, so it would be something like:

/WORKSPACE
/common # shared es6 modules
/app1 # rollup_bundle depending on some libraries in common
/app2 # rollup_bundle depending on some libraries in common
...

Edit 2:
Also something like a babel_library could be interesting to allow incremental babel transpilation.

[alexeagle]

you can just use a plain filegroup rule if you want to be able to reference the same group of .js inputs in several different bundling rules.

babel_library sounds like a great addition, want to try a PR for that?

[nimerritt]

@Globegitter we would also benefit from a babel rule as we are migrating a large service to TypeScript and it's nice to use Bazel to mix between the two as we move forward, especially given that the TypeScript rules don't take .js files.

[an-kumar]

@alexeagle well, similar to how in second.ts you can import from 'first', it would be nice if in a second.js you could require('first'). In this sense, the compilation is happening in the nodejs_binary rule, where the modules are being used -- creating a runnable nodejs binary with it's declared dependencies and transitive dependencies correctly in the node_modules path with the correct names.

this doesn't seem doable with plain filegroups.

[an-kumar]

@alexeagle it also doesnt seem like nodejs_binary has a deps attribute, am i missing something about your ts example?

[Globegitter]

Yeah also the one thing about no nodejs_library because it does not compile - isn't it the same as python_library? Not sure if that is doing much either. But yeah happy to try filegroup for now and can hopefully get around to PRing a babel_library.

[alexeagle]

@an-kumar nodejs_binary has no compile-time deps, only runtime deps. Bazel typically uses data[] for the latter, so that's where your .js files should end up to be require-able at runtime.

Is there any change needed for this issue, or can I close it?

[prestonvanloon]

You can't use a filegroup as a dep of ts_library because it doesn't have a "JS" attribute.

+1 to implement js_library.

Also see https://github.com/bazelbuild/rules_typescript/issues/201

[alexeagle]

There actually is js_library already. It's undocumented because I'm not sure about all the use cases.
https://github.com/bazelbuild/rules_typescript/blob/63bb1a33f5231c72d7d503454e986e10139ed2ff/BUILD.bazel#L64 is an example where we use it - not because it produces outputs that a filegroup wouldn't, but because we want a place to mark some extra info (in this case the AMD module name mapping, but it can also be useful for ESModule name mapping)

For this issue, I still suspect the .js files can be a filegroup in the data of a ts_library - you wouldn't type-check your usage of the JS code from TypeScript, just need it to be present to require it at runtime.

I'm still looking for a concrete use case that requires some change here...

[Globegitter]

Now that I have had some time to evaluate things more and after the insights gainedin #200, I think what I would expect from bazel for our use-case is to provide a js_library which can create the AMD output as expected by ts_devserver (better even js_devserver in the future) then other rules could be easily written on top of that, such as babel_library or json_to_js_library producing the same amd output for dev and es6 module output for prod/rollup. Does that make sense?

[alexeagle]

Yes, if you are writing JS with EcmaScript modules (import,export) then you could use some downleveler (TypeScript is one option) that knows how to produce named AMD output.

@mprobst is working on Bazel rules that do this for our Google-internal use case. Martin, do you have any plans for open-sourcing that work, or is it hopelessly coupled with internal-only stuff in the JS provider or something?

[excavador]

We need js_library for simple reason: internal code-generators/DSL

We write own code-generators, and we need some way to expose generated files to bazel.
For golang and python we use go_library and python_library.

Actual rules does not provide way to expose auto-generated files (only it to every nodejs_binary, bad way, to be honest)

As result we are implementing own version of rules_nodejs

[jwahlin]

I have a similar use case as @excavador. We generate .js files with improbable-eng/ts-protoc-gen using a custom rule. It's a pain to then import those files into the source js files. If I don't do any customization, I can only import them from bazel-out/darwin-fastbuild/bin... which is quite ugly and OS-specific. A js_library could potentially help resolve dependencies like this one so the imports in the source js files aren't ugly.

[mprobst]

@alexeagle we're using tsickle to drive this, as it is our general entry point to the compiler. The adjustments to tsickle to allow running on JS are open source, but that's the smallest bit of the work. The major effort was dealing with the structure of the internal js_library rules. I don't think that's worth open sourcing, it's mostly dealing with a ton of oldish cruft.

[DeveloperFromUkraine]

what would you expect such a nodejs_library rule to do?

If you want to compile libraries in isolation, use a compile-to-JS rule like ts_library
If you want to publish a package to npm, use npm_package

Sorry, after reading doc, I don't understand one point. If I got two projects one is playground and second is a library, what the best solutions for make all library component (modules) dependency for playground?

Thank you for your answer.

[Globegitter]

Just for reference I have written a simple js_library to get transitive dep loading with nodejs_binary as well as basic support for ts_devserver: https://github.com/ecosia/bazel_rules_nodejs_contrib#js_library

[pauldraper]

what would you expect such a nodejs_library rule to do?

@alexeagle, not very much.

Something similar to what I expect py_library or sh_library to do.

This is the familiar Bazel paradigm -- library (composable unit), binary (executable), test (executable test) -- applied to JS.

you can just use a plain filegroup rule if you want to be able to reference the same group of .js inputs in several different bundling rules.

Except file_group doesn't have deps or data.

---

Optionally, nodejs_library could do more, like enable static dependency checking.

[gregmagolan]

Ideally, we'd replace js_library with something that understands module_name, module_root & package.json

[gregmagolan]

note: amd_names attribute in by ts_proto_library labs rule: https://github.com/bazelbuild/rules_nodejs/pull/1159/files#r326720152

[alexeagle]

current proposal:

• js_library is just filegroup plus it outputs the JS*ModuleInfo providers
• npm_package with a package.json#name and package.json#main is the way to get a module_name/module_root

[pward123]

Does npm_package currently pull name and main for module_name/module_root or is that proposed? I have an issue reproduction where the lack of module_name seems to be the cause.

https://github.com/pward123/bazel-monorepo-test/tree/comparison

[alexeagle]

FYI seems likely this will slip for 1.0, needs a fair amount of design thinking

[gregmagolan]

For 1.0, we could add the legacy typescript provider and the DeclarationInfo provider to js_library so that users could use it in the deps of a ts_library?

Otherwise, ts_library will complain:

//foo:bar is neither a TypeScript nor a JS producing rule. Dependencies must be ts_library

The AmdNamesInfo provider in js_library can be dropped now that it is not used anywhere.

Ideally we could promote node_module_library to the public API instead and remove js_library all together (https://github.com/bazelbuild/rules_nodejs/blob/master/internal/npm_install/node_module_library.bzl) but node_module_library provides NpmPackageInfo so consuming rules will complain if not all NpmPackageInfo are from the same workspace. Seems that requirement could be dropped now with the advent of the linker but there would be some work involved there.

[alexeagle]

We now have a rough design to solve this by making pkg_npm work as a first-party dependency within a monorepo rather than just for publishing externally to npm. https://hackmd.io/@alexeagle/SkNE_w2QU

[pward123]

This looks great!

Would it be possible to support rollup_bundle so that @rollup/plugin-node-resolve automatically bundles monorepo dependencies?

[alexeagle]

we ran into the issue that pkg_npm produces a TreeArtifact (output directory) which means it can't also supply individual file outputs to satisfy providers like DeclarationInfo. So it needs to have a non-directory output mode.

This would be a non-breaking change so we'll put it off until 2.x after 2.0