Rules_nodejs: yarn_install & npm_install with managed directories can't handle deleted or manually regenerated node_modules folder

[gregmagolan]

In nodejs rules 0.30.0, the yarn_install & npm_install rules now use the new managed directories Bazel 0.26.0 feature so the bazel build will use the user's node_modules folder (eliminating the need to install node_modules twice: once locally and once for the bazel build).

Originally, when the managed director feature was in design phase, it handled the case where node_modules was deleted or recreated manually (via yarn or npm) and the repository rule would re-run and everything would work. However, some of this design and scope of the feature changed and the repository rule is no longer run if node_modules is deleted or manually recreated. It only runs after a full bazel clean --expunge or when one of its inputs such as package.json or yarn.lock or package-lock.json changes.

This means that if you manually rm -rf your node_modules folder the bazel build will no longer work. You'll see an error that looks like this:

greg@macbook bazel_with_users_node_modules (master) $ bazel test ...
...
INFO: Found 1 test target...
Target //:main up-to-date:
  bazel-bin/main_bin.sh
  bazel-bin/main
INFO: Elapsed time: 16.640s, Critical Path: 2.44s
INFO: 2 processes: 2 darwin-sandbox.
INFO: Build completed successfully, 7 total actions
//:main                                                                  PASSED in 0.4s

Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these arINFO: Build completed successfully, 7 total actions

greg@macbook bazel_with_users_node_modules (master) $ rm -rf node_modules

greg@macbook bazel_with_users_node_modules (master) $ bazel test ...
ERROR: /private/var/tmp/_bazel_greg/8cb9ee79286b251e74f070e4ed053074/external/npm/typescript/BUILD.bazel:123:1: no such package '@npm//node_modules/typescript': BUILD file not found on package path and referenced by '@npm//typescript:typescript'
ERROR: Analysis of target '//:main' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.224s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
greg@macbook bazel_with_users_node_modules (master) $ 

Running yarn or npm install at that point still will not fix the build as the repository rule needs to rerun to create the npm fine grained build targets.

greg@macbook bazel_with_users_node_modules (master) $ yarn
yarn install v1.13.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 🔨  Building fresh packages...
✨  Done in 0.34s.

greg@macbook bazel_with_users_node_modules (master) $ bazel test ...
ERROR: /private/var/tmp/_bazel_greg/8cb9ee79286b251e74f070e4ed053074/external/npm/typescript/BUILD.bazel:123:1: no such package '@npm//node_modules/typescript': BUILD file not found on package path and referenced by '@npm//typescript:typescript'
ERROR: Analysis of target '//:main' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.210s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

The fix for this will likely come in Bazel 0.26.1 so this is mainly a tracking issue here.

[gregmagolan]

/cc @alexeagle

[gregmagolan]

Note: if this issue is a blocker for you, you can easily opt out of managed directories: https://github.com/bazelbuild/rules_nodejs/wiki#opting-out and wait for the fix to come in Bazel 0.26.1

[gregmagolan]

Underlying Bazel issue here: https://github.com/bazelbuild/bazel/issues/8487

[gregmagolan]

Fix for this has landed in Bazel https://github.com/bazelbuild/bazel/commit/6efc5b787ad3164cc2fb779c73377695032b4524 but did not make it into the 0.26.1 release.

It may make it 0.27 which is currently in rc. If not, 0.28 is scheduled for July so should be 0.28.rc0.

[gregmagolan]

So some good news and bad news:

Bazel 0.27.0rc5, which is now mirrored in the npm package @bazel/bazel next, solves the issue when node_modules is deleted. Thanks @irengrig for getting that fix into Bazel. This solves the most common use case when you delete your node_modules folder and then re-run the bazel build.

The case where you rm -rf node_modules and then manually re-create it by running yarn install or npm install is not yet fixed in 0.27.0rc5. In that case, you'll still have to either

1) delete your node_modules folder and let Bazel create it on the next build via your yarn_install or npm_install rule

2) run bazel clean --expunge before running your build

The first of these two options is going to be faster so that your build remains incremental.

[gregmagolan]

https://github.com/bazelbuild/rules_nodejs/releases/tag/0.32.0 released which resolves this.

Please read the release notes as there are some breaking changes and you may need to rm -rf node_modules if you are upgrading to 0.32.0 from 0.31.x and were using symlinked node_modules with yarn_install/npm_install.

[gregmagolan]

PSA: https://github.com/angular/angular-bazel-example is finally updated to rules_nodejs 0.32.2. This shows working example of using symlinked node_modules with Angular & Bazel and the new managed directories features in Bazel 0.27.0.

[ashi009]

I'm still seeing this issue from time to time with rules_nodejs 0.32.2 and bazel 0.27.0:

ERROR: <redacted>/BUILD.bazel:21:1: every rule of type ts_devserver implicitly depends upon the target '@build_bazel_rules_typescript_devserver_deps//node_modules/requirejs:require.js', but this target could not be found because of: no such package '@build_bazel_rules_typescript_devserver_deps//node_modules/requirejs': BUILD file not found in directory 'node_modules/requirejs' of external repository @build_bazel_rules_typescript_devserver_deps
ERROR: Analysis of target '<redacted>' failed; build aborted: no such package '@build_bazel_rules_typescript_devserver_deps//node_modules/requirejs': BUILD file not found in directory 'node_modules/requirejs' of external repository @build_bazel_rules_typescript_devserver_deps

The host with this issue has a bad network connection. The host runes rm node_modules && bazel clean --expunge && bazel build //... on a daily cronjob. During the process, yarn may fail due to timeout, but it seems that yarn_install will continue to generate BUILD files for the incomplete node_modules.

[a-ignatov-parc]

@ashi009 we had the same problem with node_modules in an external repository. The only solution that helped us was to opt-out symlinked node_modules.

[gregmagolan]

@ashi009 we had the same problem with node_modules in an external repository. The only solution that helped us was to opt-out symlinked node_modules.

Yes. Thanks for pointing that out.

That's a known issue as there is no way to express to managed_directories in your WORKSPACE file that a symlinked node_modules folder in an external repository is managed. Also, if symlinked that node_modules folder ends up being in an external repository anyway so there is not much gained by symlinking in that case.

[ashi009]

Canceling a build at its loading phase will also trigger this.

The first build:

Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
# Received cancellation signal, interrupting

The second build:

Loading:
Loading: 0 packages loaded
ERROR: error loading package '': in /var/lib/buildkite-agent/builds/buildkite-agents-preemptible-nzjp-2/<redacted>/javascript/npm.bzl: Every .bzl file must have a corresponding package, but '@npm//:install_bazel_dependencies.bzl' does not have one. Please create a BUILD file in the same or any parent directory. Note that this BUILD file does not need to do anything except exist.
ERROR: error loading package '': in /var/lib/buildkite-agent/builds/buildkite-agents-preemptible-nzjp-2/<redacted>/javascript/npm.bzl: Every .bzl file must have a corresponding package, but '@npm//:install_bazel_dependencies.bzl' does not have one. Please create a BUILD file in the same or any parent directory. Note that this BUILD file does not need to do anything except exist.
INFO: Elapsed time: 0.200s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
INFO: Build Event Protocol files produced successfully.
FAILED: Build did NOT complete successfully (0 packages loaded)

My speculation is that repository_ctx doesn't have a way to tell if a repository has been fully loaded or not, so it will use a corrupted external repository without loading it again.

[irengrig]

Hi @ashi009 , thank you for reporting this.

My speculation is that repository_ctx doesn't have a way to tell if a repository has been fully loaded or not, so it will use a corrupted external repository without loading it again.
Actually, it has the logic to only record the repository value as complete after the corresponding Starlark code of repository rule runs.

I created an issue in Bazel project to investigate this behavior.

[uri-canva]

Can this issue be re-opened? Or should we create a new one?

[uri-canva]

New issue: https://github.com/bazelbuild/bazel/issues/8487