Rules_nodejs: Project files not found, but npm install seemed to have worked?

[hectim]

Attempting to build a React project with Bazel. I was able to get it working if I handle the node_modules myself, but when I try to have Bazel auto-manage them I run into errors. Was wondering if you could point me in the right direction.

WORKSPACE

git_repository(
  name = "build_bazel_rules_nodejs",
  remote = "https://github.com/bazelbuild/rules_nodejs.git",
  tag = "0.3.1",
)

load("@build_bazel_rules_nodejs//:defs.bzl", "node_repositories", "npm_install")

node_repositories(package_json = ["//:package.json"])

npm_install(
    name = "noodles",
    package_json = "//:package.json",
)

BUILD.bazel

package(default_visibility = ["//visibility:public"])

load("@build_bazel_rules_nodejs//:defs.bzl", "nodejs_binary")

nodejs_binary(
  name = "react",
  node_modules = "@noodles//:node_modules",
  entry_point = ".bin/react-scripts"
)

bazel run :react start throws an error

Error: Cannot find module '/private/var/tmp/_bazel_name/f3f45320aadac9e5bae9fe1a19fa2c7e/execroot/__main__/bazel-out/darwin-fastbuild/bin/react.runfiles/__main__/package.json'
    at Function.Module._resolveFilename (module.js:513:15)
    at Function.Module._load (module.js:463:25)
    at Module.require (module.js:556:17)
    at require (internal/module.js:11:18)
    at getPublicUrl (/private/var/tmp/_bazel_name/f3f45320aadac9e5bae9fe1a19fa2c7e/external/noodles/node_modules/react-scripts/config/paths.js:34:19)
    at Object.<anonymous> (/private/var/tmp/_bazel_name/f3f45320aadac9e5bae9fe1a19fa2c7e/external/noodles/node_modules/react-scripts/config/paths.js:61:14)
    at Module._compile (module.js:612:30)
    at Object.Module._extensions..js (module.js:623:10)
    at Module.load (module.js:531:32)
    at tryModuleLoad (module.js:494:12)
ERROR: Non-zero return code '1' from command: Process exited with status 1

whereas bazel run :react install gives a valid response

Unknown script "install".
Perhaps you need to update react-scripts?

So it seems that the node_modules were correctly installed but the script I'm referencing can't seem to find the project files where it usually looks. Is this simply a consequence of react-scripts not being familiar with the file locations or am I missing something necessary in the build file? Any help in the right direction would be appreciated.

edit: example repo here

[alexeagle]

Looks like this particular binary makes an assumption about the filesystem layout that doesn't hold under Bazel. getPublicURL at node_modules/react-scripts/config/paths.js:34 seems to assume that there is a package.json file in the working directory? Can you add some console.log debugging there?
If so, perhaps the answer is to run this binary with some special environment, eg. with a particular working directory.

[gunn4r]

I'm working on this with @hectim.

We've been able to get the binary to execute properly(?), but it's like react-scripts can't actually transpile the code.

The config that gets generated and used by webpack, within react-scripts looks like: (this was just commented out just before running the webpack compiler)

{ bail: true,
  devtool: 'source-map',
  entry:
   [ '/private/var/tmp/_bazel_gunnar/4406f20d87df48b5aa9395497a29dcd9/external/noodles/node_modules/react-scripts/config/polyfills.js',
     '/private/var/tmp/_bazel_gunnar/4406f20d87df48b5aa9395497a29dcd9/execroot/__main__/bazel-out/darwin-fastbuild/bin/reactscripts.runfiles/__main__/src/index.js' ],
  output:
   { path: '/private/var/tmp/_bazel_gunnar/4406f20d87df48b5aa9395497a29dcd9/execroot/__main__/bazel-out/darwin-fastbuild/bin/reactscripts.runfiles/__main__/build',
     filename: 'static/js/[name].[chunkhash:8].js',
     chunkFilename: 'static/js/[name].[chunkhash:8].chunk.js',
     publicPath: '/',
     devtoolModuleFilenameTemplate: [Function: devtoolModuleFilenameTemplate] },
  resolve:
   { modules:
      [ 'node_modules',
        '/private/var/tmp/_bazel_gunnar/4406f20d87df48b5aa9395497a29dcd9/execroot/__main__/bazel-out/darwin-fastbuild/bin/reactscripts.runfiles/__main__/node_modules' ],
     extensions: [ '.web.js', '.mjs', '.js', '.json', '.web.jsx', '.jsx' ],
     alias:
      { 'babel-runtime': '/private/var/tmp/_bazel_gunnar/4406f20d87df48b5aa9395497a29dcd9/external/noodles/node_modules/babel-runtime',
        'react-native': 'react-native-web' },
     plugins: [ [Object] ] },
  module: { strictExportPresence: true, rules: [ [Object], [Object] ] },
  plugins:
   [ InterpolateHtmlPlugin { replacements: [Object] },
     HtmlWebpackPlugin { options: [Object] },
     DefinePlugin { definitions: [Object] },
     UglifyJsPlugin { options: [Object] },
     ExtractTextPlugin {
       filename: 'static/css/[name].[contenthash:8].css',
       id: 1,
       options: {} },
     ManifestPlugin { opts: [Object] },
     SWPrecacheWebpackPlugin { config: {}, options: [Object], overrides: {}, warnings: [] },
     IgnorePlugin {
       resourceRegExp: /^\.\/locale$/,
       contextRegExp: /moment$/,
       checkIgnore: [Function: bound checkIgnore] } ],
  node:
   { dgram: 'empty',
     fs: 'empty',
     net: 'empty',
     tls: 'empty',
     child_process: 'empty' } }

The nodejs_binary:

nodejs_binary(
    name = "reactscripts",
    entry_point = "react-scripts/bin/react-scripts.js",
    node_modules = "@noodles//:node_modules",
    data = glob([
        'src/**',
        'public/**',
    ]) + ['//:package.json'],
    templated_args = ["--node_options=--preserve-symlinks"],
)

What happens when we run the generated binary (bazel run :reactscripts build) is a module parse error:

Module parse failed: Unexpected token (7:16)
You may need an appropriate loader to handle this file type.
| import registerServiceWorker from './registerServiceWorker';

[gunn4r]

I want to mention that I've tried Parcel bundler as well as rollup now and have similar issues. There is something fundamentally wrong with how we're doing this in regards to the node_modules / paths but we're pretty stuck as to where we're going wrong.

[alexeagle]

Don't have time to repro right now, you'll have to debug some more to find
1) the path that actually exists in the input directory where the tool reads from
2) the different path that the tool tried to read instead

then we'll figure out how to adjust one of those to make the resolution succeed.
Perhaps https://github.com/bazelbuild/rules_nodejs/issues/109#issuecomment-363800941 is also useful for you?

[alexeagle]

Sorry I wasn't able to help with this, I'm closing the issue as it seems stale and I don't know how to repro. Please comment or open a new issue if you have a repro.
Note that I started a webpack_bundle rule today, not sure if anyhting in there is helpful.

[MarkusTeufelberger]

Since I debugged something similar today with a colleague, here's what we found:

https://github.com/facebook/create-react-app/blob/d36603979591f37eed6f7a3efcba136e7547e75d/packages/react-scripts/bin/react-scripts.js#L32-L38 spawns the build.js script in a new node process. This is a problem, since in our case, one dev did NOT have node installed directly on the machine.

Line 33 above should have been process.argv[0] instead, since in our custom rule, we're already downloading and using the node binary (which is why the outer script worked out, but the inner one failed).

Also the react-scripts file doesn't do a great job erroring when the subprocess fails...

I don't know if that's your issue, but it might at least help in tracking down some things or being able to reproduce.

[gregmagolan]

@MarkusTeufelberger Which version of rules_nodejs are you using? The process.argv[0] shouldn't be necessary using the latest version 0.10.0 since nodejs_binary now puts the bazel managed node in the path so spawning a new process called node from your react-script.js should use the bazel managed node.

[MarkusTeufelberger]

We used a custom written rule, but got an error pattern that looked very similar to the one reported here. Maybe modifying the PATH already fixes the reported issue too (it definitely would have fixed ours). It was just something that caused a relatively large amount of head scratching (it works on one machine but not the other) and I thought it might be useful to contribute the cause here too, in case it was something you didn't yet consider or try when reproducing/fixing.

[gunn4r]

Just want to chime in that we abandoned this issue (and react-scripts) in favor of writing our own webpack and typescript build process. Took a lot of work but we've got Bazel working for us (with the exception of a decent local dev server as of yet) for our front end projects.

[kalbasit]

@gunn4r care to open source or blog about the build process you got setup?

[Jdban]

+1 to @kalbasit's question
@gunn4r Can we get some more info? :)

[pward123]

I've been trying to get a simple [email protected] running in bazel and have run across a couple things, but am still blocked.

First, the cra expects cwd to be the project root. I've ejected my scratch cra app and modified the config and scripts folders to use __dirname relative paths.

Second, bazel is monkey patching the require function, but babel is using the npm resolve module to load plugins. Since resolve doesn't have the patches, babel ignores the plugins and tosses errors because it doesn't transpile jsx.

See https://github.com/babel/babel/issues/7952 and https://github.com/bazelbuild/rules_nodejs/issues/211

I'm ok with the __dirname change, but this babel using resolve issue is still blocking me. I'm trying to get it working with a minimal change to the ejected webpack config, but its a bit slow going. I'll follow up if I figure out a solution.