Create-react-app: npm run build fails for EventEmitter

[godmar]

@gaearon asked that I file a separate issue.

How to reproduce: run create-react-app fail-event-emitter, add a line

import EventEmitter from 'events';

to src/index.js and then run npm run build, which fails with:

> [email protected] build /home/gback/fail-event-emitter
> react-scripts build

Creating an optimized production build...
Failed to compile.

static/js/main.068c41be.js from UglifyJs
Unexpected character '`' [/usr/lib/nodejs/events.js:260,0]

react-scripts 0.7.0 is used and node.js 6.9.1 (or 7.0)

The purported cause is that CRA does not transpile "dependencies" into ES5 and then the UglifyJS plugin fails when it encounters ES6 code.
The test case above was derived from a failure in react-bootstrap-tables, which includes a module that in turn references EventEmitter. Others have suggested to provide an alternate EventEmitter implementation in the application path, but it is not clear to me how react-bootstrap-table's dependencies can be made to use this alternate implementation.

[godmar]

Same issue when including "recharts": "^0.18.0". This package also includes EventEmitter.

It looks as though at this point, the chances of successfully building a 'production' build with create-react-app are rather low.

In related posts, it was mentioned as a work-around to include our own implementation of EventEmitter. Can someone describe how to do that? Specifically, I do not know how to ensure that dependencies such as recharts import the included implementation of EventEmitter rather than falling back onto node.js's. Is there a webpack setting? If so, react-scripts should probably include it.

Would be great if this were addressed. Currently the only work-around is to disable the 'UglifyJSPlugin', but this requires reaching inside CRA (or ejecting).

[henri-hulski]

We replaced the node EventEmitter with EventEmitter3 for Cerebral2.

But this was only possible because we are the authors of all dependencies which use the EventEmitter.

[gaearon]

Can you investigate where this problem is coming from? I'm sorry I don't have much time to investigate it right now but I'd be happy to accept a fix for it.

It looks as though at this point, the chances of successfully building a 'production' build with create-react-app are rather low.

Well this problem never occurred before, so it must be something new. Can you take time to figure out why it happens and propose a fix?

[gaearon]

In particular I would expect Webpack to take EventEmitter implementation from some collection of shims. The question is, then, why those shims aren't in ES5 anymore and where they are located. This is not something you would require CRA maintainers to investigate—you could try to dig into it by yourself and figure out which package is responsible for that code and when it changed.

[dvkndn]

Updated: comment deleted since I'm misunderstand the situation 😄

[jquense]

generally the webpack shims come from: https://github.com/webpack/node-libs-browser assuming something else isn't messing with the module resolution

[gaearon]

The question is, then, what changed there that the source of EventEmitter is no longer ES5.

[henri-hulski]

events.js uses template strings from node 6.0.0.
They were introduced in c6656db.

[henri-hulski]

So it comes from core node.
But not happens on all machines so not really sure why.

[dvkndn]

@henri-hulski it seems webpack is supposed to not use that native implementation from NodeJS, but instead use one from node-libs-browser.

[henri-hulski]

In my case it uses the native implementation.

[henri-hulski]

And the example in the first post by @godmar gets also exactly the same error message as I from events.js:

Unexpected character '' [/usr/lib/nodejs/events.js:260,0]`

[dvkndn]

right, but in my case it used the shim implementation. So there are some difference between our system that cause this issue, I guess?

I'm using Mac OS Sierra, and my Node is 6.3.0, could that be the difference?

[henri-hulski]

A detailed description of my case you find in cerebral/cerebral#389.

My system:
System: Debian GNU/Linux 8.6 (jessie).
Linux version: 4.7.0-0.bpo.1-amd64
Node version: 6.9.1
npm version: 3.10.8
react-scripts version: 0.7.0

In our team only 2 persons got this bug.
So yeah it depends on the configuration.

[henri-hulski]

The second configuration which got this error was:

Kernel : Linux 4.4.0-45-generic (x86_64)
Compiled : #66-Ubuntu SMP Wed Oct 19 14:12:37 UTC 2016
C Library : Unknown
Default C Compiler : GNU C Compiler version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.2)
Distribution : Linux Lite 3.0

npm: 3.10.8
node: 6.9.1
react-scripts: 0.7.0

[dvkndn]

Are all your team using the same version of distribution of Linux? There are people reported that they see similar issues in Ubuntu

[jquense]

have to check with the webpack team I think, but I believe depending on the target it will: include a shim, use the native code, or include nothing. https://webpack.github.io/docs/configuration.html# node the docs would suggest that not all node modules have this behavior but the docs may be incomplete.

newer node will have es6 code in the core implementations so maybe it's folks on newer node? the shims haven't been updated in 6 months so they probably aren't the problem.

[henri-hulski]

BTW
When fixing /usr/lib/nodejs/events.js, I get similar errors in /usr/lib/nodejs/internal/util.js:

Failed to compile.

static/js/main.c359f549.js from UglifyJs
Unexpected character '`' [/usr/lib/nodejs/internal/util.js:4,0]

Failed to compile.

static/js/main.3335427e.js from UglifyJs
Unexpected character '`' [/usr/lib/nodejs/internal/util.js:30,0]

Failed to compile.

static/js/main.cebb091d.js from UglifyJs
SyntaxError: Unexpected token: punc (.) [/usr/lib/nodejs/internal/util.js:64,0]

And I also got error when using node v5.12.0 before:

static/js/main.957b2071.js from UglifyJs
SyntaxError: Unexpected token: keyword (const) [/usr/lib/nodejs/util.js:564,0]

[henri-hulski]

@jquense So yeah I think you're right.

[godmar]

So is there a solution that does not require dropping/commenting out UglifyJS?

@dvkndn mentions shims for webpack - presumably, we need to tell it to transpile events.js into ES5 when loading it --- isn't this fundamentally at odds with @gaearon statement that 'dependencies' are not transpiled?

The offending commit (pointed out by @henri-hulski c6656db352973d6aea24cb1a3c76adf042b25446) is from Jan 20, 2016, and presumably made it into node releases shortly after? That's 10 months (aka an eternity in this line of work ;-))

[henri-hulski]

@godmar But as shown above I already had problems with node 5.12.0.
So the problem is the webpack configuration.
The target should be web.

[henri-hulski]

See https://github.com/webpack/webpack/blob/f18989c712894bba13ebfb6dfb3f50fc44b0d12a/lib/WebpackOptionsApply.js#L78

[godmar]

@henri-hulski Where do you add the target?
I tried adding it to node_modules/react-scripts/config/webpack.config.prod.js but no luck.
I tried both as a top-level option as well as inside the output { } section.

[godmar]

BTW, @henri-hulski node 5.12.0 was released 2016-06-23 so that's consistent with my statement since the commit is from Jan 2016.

Also, why would include NodeSourcePlugin fix the issue; the documentation says that it "adds polyfills for process, console, Buffer and global if used." - but hasn't EventEmitter be moved outside process. in node.js 6.*?

Should we file an issue with webpack and ask them what configuration is necessary? Presumably, it's not ok for webpack to assume that a build can use ES6, even if it went straight to the browser and didn't get caught up in UglifyJS, or is this ok to assume now?

[henri-hulski]

It should go on top level. But it's actually the default. So not sure what happens here.
See https://webpack.github.io/docs/configuration.html.

[godmar]

@henri-hulski I just noticed that too that target "web" is the default. I can change it to 'target: "node"' which then compiles, but obviously that's not a working build.

[henri-hulski]

Maybe look at the node option:
: true, "mock", "empty" or false

So something like "events" : true in the node section.

[godmar]

You mean based on this?

Tried

  node: {
    '<node buildin>': 'true',
    fs: 'empty',
    net: 'empty',
    tls: 'empty'
  }

with no effect.

Also, in the webpack 1.13.2 which react-script includes, there does not appear to be any special handling of events.js or EventEmitter (based on grepping the code)

But 'tls' shows up in node-libs-browser ... maybe we need to get webpack to somehow use this alternative events.js implementation? If so, how?

[godmar]

Ha!

I just did

npm install events --save

in my directory and that worked!
Am I dumb for not trying this right away?

And why does it work? I thought the way npm works is such that a package like recharts gets its own, or the default version of 'events' that the developer relied upon when building it... why can I override it in this way?

Very reminiscent of 2000's Java CLASSPATH hell.

[TheLarkInn]

Tagging my self to track this.

[henri-hulski]

@godmar Could you try

 node: {
    events: true
  }

It should work.
See https://github.com/webpack/webpack/blob/master/lib/node/NodeSourcePlugin.js#L83-L93.

node-libs-browser uses the same shrim as you:
https://github.com/webpack/node-libs-browser/blob/master/package.json#L18

[godmar]

I tried that, it didn't work.

[godmar]

Just reconfirmed (by removing node_modules/events, see above) - it doesn't work. I don't see how it would, either. There is no copy of 'events' anywhere in my node_modules:

$ find node_modules -type d -name events
node_modules/dom-helpers/events
node_modules/react-scripts/node_modules/jsdom/lib/jsdom/living/events
node_modules/react-scripts/node_modules/events
node_modules/redux-form/lib/events
node_modules/redux-form/es/events

[godmar]

(My larger question here is how webpack's resolver works. Why does it pick up ./node_modules/events over the built-in events when node_modules/recharts/ ... does not have a dependency on 'events'?)

[henri-hulski]

Webpack depends on node-libs-browser and node-libs-browser depends on events.
So it should be installed.

I think it happens actually in the code I'm linking above.

[henri-hulski]

    Object.keys(nodeLibsBrowser).forEach(function(lib) {
            if(options[lib] !== false) {
                compiler.resolvers.normal.apply(
                    new AliasPlugin("described-resolve", {
                        name: lib,
                        onlyModule: true,
                        alias: getPathToModule(lib, options[lib])
                    }, "resolve")
                );
            }
        });

Where options is the node section of the configuration.

[godmar]

Not for me:

$ ls node_modules/react-scripts/node_modules/node-libs-browser/node_modules/
isarray  readable-stream

But

$ npm ls events
── [email protected] 
└─┬ [email protected]
  └─┬ [email protected]
    └─┬ [email protected]
      └── [email protected] 

npm bug? I'm on 3.10.8

[gaearon]

So is there a solution that does not require dropping/commenting out UglifyJS?

To be clear the problem is not in UglifyJS. It's in Webpack including ES6 code from Node instead of shims in some cases for some reason.

Even if we didn't use UglifyJS it would still be very bad to ship ES6 code from Node to users who may not have recent browsers.

[godmar]

Here's more information. When I wipe my node_modules via /bin/rm -rf node_modules/* and do a npm cache clear, followed by npm install, I'm seeing that npm installs a copy of events@^1.0.0 in the directory node_modules/react-scripts/node_modules/events

The package.json file contains:

  "_requiredBy": [
    "/react-scripts/node-libs-browser"
  ],

node-libs-browser's package.json contains events as a dependency (then why is it installed at the sibling level at node_modules/react-scripts/node_modules/events rather than node_modules/react-scripts/node_modules/node-libs-browsers/node_modules/events

So somehow npm thinks that installing events there fulfills node-libs-browser's dependencies - but webpack doesn't look there.

So there's either a bug in webpack, or a configuration mistake - but it appears related to the fact that node-libs-browsers is included via [email protected]

BTW, were any of you able to reproduce the issue with just the barebone cra-created build?

[henri-hulski]

@godmar
Yes, I also cleared the cache and made a fresh install.
The newer npm optimizes the module tree, so it has not to be the same as the dependency tree.

[jquense]

an interim solution would be for CRA to include node-libs-browser itself and add alias's to its webpack config.

{
 resolve: {
   alias: {
     'events': path.resolve('./node_modules/events')
   }
 }
}

[godmar]

Fundamentally, shouldn't CRA instruct webpack to use node-libs-browser's version of all packages it provides? How is webpack supposed to know that in order to resolve rechart's dependency on 'events' it needs to look in node_module/react-scripts/node_modules/events rather than in /usr/lib/nodejs/event.js?

The documentation points at resolve.modulesDirectories - should this be set?

[gaearon]

@jquense I'm happy to take a PR for this.

[gaearon]

Tagging as 0.8.0 because it's super hi pri and we can't release without fixing this.

[fson]

The dependencies won't be bundled in 0.8.0 – will it fix this issue? It seems that npm should flatten the tree in that case and you'd end up with the events package in ./node_modules.

[gaearon]

Can/should we just explicitly depend on it?

[fson]

Sure we can, if we want to be certain. Possibly with @remove-on-eject because we haven't seen this to be a problem in apps using Webpack directly?

[fson]

I think it would be important to be able to reliably reproduce this issue in any case, before trying to "fix" it. Webpack itself should also alias these modules when resolving them, so it's possible that there's an underlying bug that also needs to be fixed.

[fson]

I published a canary build with the latest changes from master to make it easier to test. @godmar could you please try and see if you can still reproduce this when you create an app with this build?

create-react-app --scripts-version 0.7.0-alpha.9c45b252 fail-event-emitter

[godmar]

With the 0.7.0-alpha.9c45b252 build, I cannot reproduce the failure when I add import EventEmitter from 'events'; to src/index.js.

So I tried another of the modules listed on the node-libs-browser page:

import sys from 'sys';

which promptly fails with:

Creating an optimized production build...
Failed to compile.

static/js/main.b7f26f0d.js from UglifyJs
Unexpected character '`' [/usr/lib/nodejs/internal/util.js:4,0]

According to the node-libs-browser website, this is provided by defunctzombie's node-util package, which I believe is available under the name util. [email protected] is installed in node_modules at the top-level, its package.json says:

  "_requiredBy": [
    "/assert",
    "/node-libs-browser"
  ],

It appears that whatever 0.7.0-alpha.9c45b252 did was only a bandaid for events.js, but perhaps didn't address the underlying issue?

[fson]

That's really weird. 0.7.0-alpha.9c45b252 is merely built from the latest master branch revision, where we started no not bundle the dependencies of react-scripts package when publishing to npm. It doesn't include any specific fixes for events per se.

This is useful information anyway – and it shows that simply depending on events wouldn't fix the problem as this seems to affect other node libs too, even though in a seemingly random way.

[fson]

@godmar Can you run npm list in the project where importing sys fails and copy the whole output here?

[godmar]

Output of npm list: https://gist.github.com/godmar/b04b96e9648f8464c983c754482af8ab

Adding my versions for completeness:

$ create-react-app --version
create-react-app version: 0.6.0
$ node -v
v6.9.1
$ npm -v
4.0.3

[fson]

Okay, both events and util are listed under node-libs-browser. What about ls node_modules, what does it print?

[godmar]

Both installed: https://gist.github.com/godmar/921a7c3b2a59c5ac5e00a11281af92d5

[fson]

I'm out of ideas here, I still haven't been able to reproduce this with import EventEmitter from 'events' nor import sys from 'sys'. I'm installed the same versions (but I'm on macOS so maybe this could have something to do with it):

> node -v
v6.9.1
> npm -v
4.0.3

[godmar]

Was @henri-hulski able to reproduce the failure - can't quite parse his comment here ?

Do you have a webpack globally installed maybe? I do not.

[godmar]

I'm trying to track this down.

It appears that 'enhanced-resolver' is somehow involved as it is used by webpack.
I added the following statement to node_modules/enhanced-resolve/lib/Resolver.js about here

...
        return this.applyPluginsParallelBailResult(types[0], request, createInnerCallback(function innerCallback(err, result) {
if (result) console.log(`Response to ${currentRequestString} is ${result.path}`);
            if(callback.log) {
...

This shows:

Response to events in /home/gback/fail-event-emitter2/src is /home/gback/fail-event-emitter2/node_modules/events/events.js
Response to sys in /home/gback/fail-event-emitter2/src is /usr/lib/nodejs/sys.js

A difference between the two is that 'sys' is provided by 'util' whereas 'events' is provided by 'events'. So whatever aliasing/renaming node-libs-browser does is not applied.

BTW, the version of enhance-resolve that the webpack CRA includes uses is pretty old.

[gaearon]

BTW, the version of enhance-resolve that the webpack CRA includes uses is pretty old.

This is because webpack 1.x is pinned to enhanced-resolve@~0.9.0 so it seems expected. However if there were any bugfixes to [email protected] we should check if they got backported or not.

[godmar]

If I add an alias to webpack.config.prod.js like so:

    alias: {
      'sys': 'util',
      // Support React Native Web
      // https://www.smashingmagazine.com/2016/08/a-glimpse-into-the-future-with-react-native-for-web/
      'react-native': 'react-native-web'
    }

it builds:

Response to sys in /home/gback/fail-event-emitter2/src is /home/gback/fail-event-emitter2/node_modules/util/util.js

So maybe 'sys' is a bad example since it's been deprecated for a while?

Perhaps it would be helpful to have a unit test that imports everything in node-libs-browser?

[godmar]

PS: sure enough, adding

import timers from 'timers';

fails because:

Response to timers in /home/gback/fail-event-emitter2/src is /usr/lib/nodejs/timers.js

with ES6 syntax in buffers.js:

SyntaxError: Name expected [/usr/lib/nodejs/buffer.js:5,0]

[godmar]

BTW, if you want a snapshot of my working dir: https://godmar.cs.cloud.vt.edu/fee2.tgz

[henri-hulski]

@godmar
Yes I reproduced the failure with a fresh and clean install.

[terinjokes]

@godmar adding the alias to the config manually is definitely not something you should have to do. I'm taking a look to see if I can isolate what's causing it, thanks for the information on the versions you're using, as well as tarball of your CWD.

[godmar]

Based on the other thread about this issue, it seems that this occurs on Linux only but not on OSX with identical node/npm/react-scripts version. What could cause this? I'm on Ubuntu 16.0.4.1, for reference.

[terinjokes]

@godmar First thoughts: differences in case sensitivity, but I don't think that's in play here (but haven't entirely ruled it out either)

[terinjokes]

macOS, or more specifically, the default HFS+J derivative it uses, is case preserving (it remembers the case you created the file) but not case sensitive (typing LS works). I've seen this cause problems resolving dependencies in Node.js before where projects are developed on macOS but deployed to Linux.

But this is a little bit off topic, but I'm still digging into it (having taken a break for dinner).

[ghost]

I'm reposting from https://github.com/facebookincubator/create-react-app/issues/815.

I am experiencing this same error message after installing a different module, auth0-lock (https://github.com/auth0/lock). I have not ejected my project, but I found others were experiencing the same error message, such as @sirreal , and as mentioned at https://github.com/facebookincubator/create-react-app/issues/587. I saw that this issue is still open, while those other ones are closed, so I've decided to post about my experience here.

npm: 3.10.8
node: 6.9.1
os: Ubuntu 16.04 LTS

I ran npm cache clean and npm i -g npm. I also added url as a dependency, blew away my node_modules dir and re-ran npm install, but none of those fixed the problem, although my error stack trace is shorter:

Uncaught Error: process.binding is not supported
    at Object.process.binding (webpack:///./~/react-scripts/~/process/browser.js?:173:11)
    at Object.eval (webpack:////usr/lib/nodejs/util.js?:3:20)
    at eval (webpack:////usr/lib/nodejs/util.js?:942:30)
    at Object.<anonymous> (http://localhost:3000/static/js/bundle.js:3784:2)
    at __webpack_require__ (http://localhost:3000/static/js/bundle.js:556:30)
    at fn (http://localhost:3000/static/js/bundle.js:87:20)
    at eval (webpack:///./~/auth0-lock/lib/i18n.js?:20:13)
    at Object.<anonymous> (http://localhost:3000/static/js/bundle.js:4342:2)
    at __webpack_require__ (http://localhost:3000/static/js/bundle.js:556:30)
    at fn (http://localhost:3000/static/js/bundle.js:87:20)

It may be worth noting that just requiring auth0-lock module breaks it.

Finally, I'm in a room of 12 people and everyone else has a Mac and are not running into this issue.

Thought I'd post in case these are all related and it could help fix this issue! ¯_(ツ)_/¯

[ghost]

It looks like this definitely has something to do with package dependencies and webpack. It does not appear to be limited to any specific node module. After reading all the comments above and in the previous issues, I realized that certain users were able to get around the problem by installing a package closer to the root. For example, in issue https://github.com/facebookincubator/create-react-app/issues/815 both @brakowski and @NessDan were able to get it working by installing url as a direct dependency (Oct 12). Then, in the comments above, @godmar got it working by installing events. Well, lo and behold, I ran npm i util -S and the problem went away!

$ npm ls util
[email protected] /home/derek/Documents/garudacrafts/jrscode/code/lunchit/web
├─┬ [email protected]
│ └─┬ [email protected]
│   └─┬ [email protected]
│     └── [email protected]
└── [email protected]

The question remains: what is the root issue that is causing these problems?

[gaearon]

Something, somewhere (in Webpack's enhanced-resolve?) resolves to the native Node module instead of the one in node_modules. As for why, I don’t know.

[gaearon]

It would be helpful if somebody who's having this issue could:

• Copy project to a different folder (just so you're not afraid to break everything)
• Eject
• Check that the issue still exists
• Update dependencies and configs to Webpack 2 format
• Get it to compile
• Check that the issue still exists

If it's fixed in Webpack 2 then that would tell us something.

[ghost]

@gaearon I copied and ejected my project and I still had the same issue that I initially reported (i.e. including the auth0-lock module led to apparent native use of util package dependency). Note that I was not able to reproduce the issue as described in https://github.com/facebookincubator/create-react-app/issues/815 by following the steps in that initial post (i.e. ejecting a brand new react-create-pp test-app). I'm not sure how to update all my dependencies and configs to Webpack 2 - can you give a little guidance? This is all new territory for me, thanks. I'm heading afk shortly, but should have some more time to try it for y'all sometime tomorrow.

[godmar]

I copied and ejected, and the issue persists.

Which exact version of webpack do you want me to try and how do I do that? (BTW, is there a way to get npm to list unpublished/unreleased versions?)

[sokra]

I would be interested to see to output of node -e "console.log(require('node-libs-browser').events)". That's the module webpack uses for events.

[gaearon]

You can learn beta versions of webpack and webpack-dev-server by running npm dist-tag ls webpack and npm dist-tag ls webpack-dev-server. Copy the beta versions in your package.json and run npm install. Then follow the migration guide I linked to earlier to port your configs.

[godmar]

@sokra

in "react-scripts": "0.7.0-alpha.9c45b252", the output is $cwd/node_modules/events/events.js

in react-scripts 0.7.0, it's Cannot find module 'node-libs-browser'

[godmar]

Do you have NODE_PATH set on your Macintoshes?

On Ubuntu, I have installed package 6.9.1-1nodesource1~xe via the instructions here, which are repeated here, which sets NODE_PATH:

$ cat /etc/profile.d/nodejs.sh 
NODE_PATH=/usr/lib/nodejs:/usr/lib/node_modules:/usr/share/javascript
export NODE_PATH

Sure enough, the error disappears if I unset this variable. It seems that 'fallback', contrary to the comment here, is not considered after node_modules, but before.

I noticed this when I tried to port the 'fallback' line to webpack 2.0. The variable paths.nodePaths, used in webpack config's 'fallback' option, is set to the directories contained in NODE_PATH, as it seems was intended by issue #253

And you thought I was joking about CLASSPATH hell - that was exactly the waste of time 20 years ago when Java came out. Hours and hours wasted because some users had a CLASSPATH setting that pulled in classes from who knows where. The setting either came from bad advice in tutorials or a bad system setting.

[godmar]

OTOH, the webpack documentation isn't exactly clear on what the behavior is when root isn't set. Technically it's true that the 'fallback' directories kicked in because a package wasn't found in root since we don't specify any root.

[sokra]

NODE_PATH isn't used by webpack.

[henri-hulski]

@sokra This is not true for CRA.
See https://github.com/facebookincubator/create-react-app/pull/476.

And I can confirm that I have /etc/profile.d/nodejs.sh present with the same content as above:

$ cat /etc/profile.d/nodejs.sh 
NODE_PATH=/usr/lib/nodejs:/usr/lib/node_modules:/usr/share/javascript
export NODE_PATH

This really smells like this actually could be the issue.
Maybe someone with a Mac can set NODE_PATH as above and check if he can reproduce the issue.

[gaearon]

That's some solid detective work. I wanted to say I knew this would get us in trouble someday but then again, I merged that feature :P

[henri-hulski]

In our cases /usr/lib/nodejs/events.js was used instead the polyfill.
So clearly comes from NODE_PATH.

[godmar]

So what's the resolution? Given that the node.js Linux package maintainers put this there, despite NODE_PATH being officially discouraged seems to imply that a user-friendly work-around is needed. At least a warning may be appropriate.

Or undo #476 and replace it with an opt-in approach for those who want to shoot themselves in the foot with this.

You may also want to check that your interpretation of fallback in the absence of root and modulesDirectories is correct.

[gaearon]

We can remove NODE_PATH support if we fix https://github.com/facebookincubator/create-react-app/issues/607 instead. There is a PR (https://github.com/facebookincubator/create-react-app/pull/1081) but it looks incomplete. Feel free to look into what's missing there.

[ghost]

My /etc/profile.d/nodejs.sh also contains NODE_PATH=/usr/lib/nodejs:/usr/lib/node_modules:/usr/share/javascript export NODE_PATH and my error message referenced usr/lib/nodejs/util.js.

[sokra]

Ah I see. Seems like NODE_PATH is applied to resolve.root which has a higher priority than node_modules (resolve.modulesDirectories). It should be applied to resolve.fallback (which has a lower priority) to be equivalent to Node.js.

https://github.com/facebookincubator/create-react-app/pull/476/files#diff-96f3236c864bf3b9b45321218336dca0R70

[godmar]

@sokra Not so.

It's applied to fallback - please read my earlier comments where I discuss this.

(Also: remember that installing 'events' directly under node_modules fixed it, so "node_modules" did take precedence over the NODE_PATH paths in fallback.) It's just that the proper packages that node-libs-browser demanded weren't found if they weren't at the top-level.)

[gaearon]

Yea, I changed that later precisely for that reason. I’m not sure why those browser shims are treated differently..

[jimmyhmiller]

It looks like NODE_PATH is causing problems, but I would hate to see it removed without a real substitute. We are currently depending on that functionality. @gaearon are you opposed to some other environment variable that isn't already overloaded with meaning?

[gaearon]

@jimmyhmiller What are you using it for? If absolute paths, we do have another solution as I mentioned in https://github.com/facebookincubator/create-react-app/issues/1023#issuecomment-264178576.

[jimmyhmiller]

Yes we are using it for that. The problem is that the 'src/node_modules' requires us to do refactor on our structure where as the environment variable would allow us to make one change in one place. But also the src/node_modules solutions just doesn't seem to really address the issue.

The goal is to be able to not use any relative imports. Relative imports are incredibly fragile and annoying to work with. When we want to do refactoring of structure, relative imports are a real pain.

[gaearon]

The problem is that the 'src/node_modules' requires us to do refactor on our structure

Why? You can create symlinks from src/node_modules to maintain exactly the same structure as you have now. (Unless you're using Windows in which case symlinks can be sort of painful.)

Also, what do you call a refactor? Isn't it just a matter of moving directories to a different place without changing any code?

[jimmyhmiller]

We are not using windows ourselves, but work with clients who might be. So the symlink solution isn't ideal for us. You are right that it isn't a refactor in the sense of code, but it is a structure refactor that has no explanation in the problem domain, but only in the tooling.

I want to be able to say, X is in folder Y because it fits with this business domain or in order to have a good separation of concerns. Instead I have to say, "Oh yeah all our code is the node_modules folder, yeah I know it is a confusing name, but we have to have it in there in order to make our build tool recognize it if we want absolute paths". Working with clients, that's not something I want to have to explain. Not needing to explain these things is the exact reason I love create-react-app.

Your comment https://github.com/facebookincubator/create-react-app/issues/741#issuecomment-250265235 is exactly how I feel about that solution and working with clients who are totally new to javascript and react makes this all the more pertinent for us.

In the end, it certainly isn't the end of the world. It just seems that having a configuration that is optional and out of the way is better than a "hack". https://github.com/facebookincubator/create-react-app/issues/741#issuecomment-261792706

[gaearon]

We can keep support for NODE_PATH but then we need to figure out how to fix the issue here in a different way.

[jimmyhmiller]

So you wouldn't want to use a different environment variable? That would solve this issue and keep the functionality the NODE_PATH gives us.

[gaearon]

Because this defeats the purpose of not hardcoding support absolute paths inside webpack: tool compatibility. Right now things "just work" in IDEs/linters/Flow/etc that respect Node resolution mechanism.

[ghost]

While this discussion has moved to NODE_PATH, isn't the core issue here fact that the global copy of a module is being used in place of the copy installed as a project dependency? Why is it falling back (as @godmar confirmed) to the native version at all when it exists right there in node_modules as a dependency of a root module? Would testing with webpack 2 be the only way to help us determine if the root cause is indeed webpack?

[gaearon]

Would testing with webpack 2 be the only way to help us determine if the root cause is indeed webpack?

It would confirm whether a fix exists or is yet to be written.

[jimmyhmiller]

Trying to recreate this on mac to see if I can help with the fix. I don't have anything in any of these folders. Any ideas where the equivalent location would be on mac?

/usr/lib/nodejs:/usr/lib/node_modules:/usr/share/javascript

[godmar]

I started on this but stopped when I realized I didn't have enough knowledge of webpack to translate all the constructs. The secret gist "transition guide" does not answer all questions, or if it does it does so in a way that assumes internal knowledge of webpack. In fact, some questions (like how to handle !postcss are asked there, but not currently answered).

[gaearon]

You can remove anything non-essential as long as we can reproduce the issue.

[godmar]

I wasn't able to. Here's a link to how far I got. It stops with being unable to recognize JSX syntax presumably because babel isn't run or doesn't have the right settings.

Another conceptual issue is that I do not know the exact semantics of root/fallback because the webpack documentation does not explain how root and fallback interact with the "normal" search path.

Notably, is this replacement correct?

-    fallback: paths.nodePaths,
+    modules: [ 'node_modules', ...paths.nodePaths ],

I don't know without understanding the normative semantics. (Note that root/fallback/moduleDirectories was replaced with a single array in webpack 2.0) But it's critical to seeing whether there is a bug in webpack or whether you were misapplying its options.

[gaearon]

The new webpack (WIP) docs are at https://webpack.js.org/.

I think with the latest beta you need to add explicit postfixes to loaders. So it's not 'babel but 'babel-loader, etc.

[jimmyhmiller]

Adding an explicit resolve.root seems to fix this problem for me. I've made a branch with that change. (obvious still work before it would be mergeable)

@godmar Think you could test it out and see if it works for you?

[godmar]

Side note: all the code in enhanced-resolve that handles this seems to be written by @sokra - but you already knew this :-) Maybe he could explain the normative behavior?

My specific question. In this code we see how the options given to webpack's resolve control the resolution process.

It seems to me that fallback directories should be considered first. I think not specifying a root is user error?

@jimmyhmiller adding the ownNodeModules directory as root works for me too even when NODE_PATH/fallback is set.

[jimmyhmiller]

@gaearon How do you feel about that approach? If that's okay I will happily finish it up and make a PR.

[gaearon]

Per documentation root is found before node_modules are searched. So if you have redux in the src, Webpack would pick it up before node_modules/redux. This is the behavior I’m trying to avoid so I think that wouldn’t work.

[jimmyhmiller]

@gaearon Not sure what you mean. I set root to ownNodeModules not nodePaths. I just tried exactly what I think you are talking about. I npm installed redux and had a file called redux.js. When I import I get the actual redux not my file. (Even when setting NODE_PATH.)

All this does is makes sure webpack explicitly searches ownNodeModules before anything else.

[gaearon]

Oh this is interesting. Could you send a PR?

[godmar]

BTW, in the absence of adding in ownNodeModules, setting root: nodePaths and setting fallback: nodePaths is completely equivalent. If you look at the code I linked to

  makeRootPlugin("module", options.resolve.root),
  new ModulesInDirectoriesPlugin("module", options.resolve.modulesDirectories),
  makeRootPlugin("module", options.resolve.fallback),

and then enhance-resolve just invokes those plugins as a waterfall. Whether a directory is in 'root' or in 'fallback' doesn't matter if the respective other setting is empty.

[TheLarkInn]

We plan on documenting this on our new docs but I wanted to leave a diagram that may or may not assist in understanding the waterfall style of enhanced resolve:

[sokra]

@TheLarkInn This diagram is webpack 2, but CRA uses webpack 1 which uses a different resolver impl.

root is tried before node_modules, fallback is tried after node_modules. It's a bit confusing, that's why it was changed for webpack 2.

[TheLarkInn]

Ah sorry I totally forgot cra is webpack 1. Apologize for the confusion.

[godmar]

@sokra - this may sound blasphemous, but are you sure that webpack 1.13.2 works the way you describe? In particular, in the code, you do what I quoted above

The plugins for the directories in 'root' and the plugins for the directories in 'fallback' are simply concatenated (unless modulesDirectories is set, which in CRA it is not).

Once this reaches enhance-resolve, all you have is a list of plugins for each directory in root and each directory in fallback, concatenated - how can you look in 'node_modules' after root and before fallback?

---

(Correction to this entry: I had overlooked that by not setting modulesDirectories, CRA is relying on the default setting including 'node_modules', see below)

[sokra]

I tried it (1.13.3) and it does work. The order in which plugins are added is used. They are called in parallel but the results are used in order.

[godmar]

@sokra - I see. It's done however iff the default setting for moduleDirectories is not overridden. In other words, it relies on the default setting of modulesDirectories being ["web_modules", "node_modules"]. If you set modulesDirectories: [] it won't work.

So if you accept PR #1131 then the search order would be:

1. paths.ownNodeModules (which is node_modules/react-scripts/node_modules)
2. web_modules
3. node_modules
4. paths.nodePaths (this is where any NODE_PATH settings would come in)

is this correct?

[godmar]

I have a working theory. If webpack attempts to resolve a module name such as events or timers it will use the node-libs-browser shims if and only if those names are still unresolved after all of root, modulesDirectories, and fallback have been searched.

I'm basing this on these observations:

(1) NodeSourcePlugin (code) calls compiler.resolvers.normal.apply in "after-resolvers"

    compiler.plugin("after-resolvers", function(compiler) {
        var alias = {};
        Object.keys(nodeLibsBrowser).forEach(function(lib) {
            if(options[lib] !== false)
                alias[lib + "$"] = getPathToModule(lib, options[lib]);
        });
        if(Object.keys(alias).length > 0) {
            compiler.resolvers.normal.apply(
                new ModuleAliasPlugin(alias)
            );
        }
    });

(2) WebPackOptionsApply (code)

    compiler.resolvers.normal.apply(
...
        makeRootPlugin("module", options.resolve.root),
        new ModulesInDirectoriesPlugin("module", options.resolve.modulesDirectories),
        makeRootPlugin("module", options.resolve.fallback),
...
    );
...
    compiler.applyPlugins("after-resolvers", compiler);

This would explain why placing a shim manually in node_modules works even on Linux systems that have the built-in modules in NODE_PATH, and it would explain why on Macintoshes that don't set NODE_PATH the browser shim is found as well.

FWIW, the documentation of NodeSourcePlugin refers to it as providing polyfills - which generally means they are meant to be used only as a last resort.

If my theory is correct, then either the nodePaths patch should be reversed, or amended to include warnings if users' NODE_PATH contains the locations of node's built-in modules. On the other hand, we can't really tell if a user's NODE_PATH setting is intentional or not. It could be that a future version of node.js provides a perfectly usable module 'xyz' in /usr/lib/nodejs and the user wants to include it in their build.

Maybe have a separate variable "RESPECT_NODE_PATH" for the use cases where NODE_PATH should be respected, and don't respect it by default.

[gaearon]

How about we intentionally deviate and only respect relative NODE_PATH. That's what our users use it for anyway. And global is always dangerous.

[fson]

Or maybe check that the path is inside the project folder? I think it would be less surprising than not allowing absolute paths, what matters is where the files are located, not how that location is represented (absolute vs relative).

[gaearon]

I can imagine people using something like ../shared or equivalent. This is in any case a stopgap solution before we remove NODE_PATH support altogether.

[gaearon]

Apparently this problem only exists when you install Node from source.
I missed this detail.

[henri-hulski]

Nope. Not from source but from NodeSource Node.js Binary Distributions.
The recommended way to install recent Node.js versions on Linux is to use setup scripts (like this for Debian / Ubuntu) which set up their repo on the system so Node.js can be installed and updated through the system's package manager.

[gaearon]

Right. But those distributions do include source, don't they?
For comparison, I can't find source for events.js on my distribution and seems like I'm not alone.

[gaearon]

Fixed in #1194.

[henri-hulski]

@gaearon No it's not the source. In Linux libraries in general install themselves under /usr/lib, /usr/share/lib or similar folders.
Maybe in macOS they use one binary instead.

[gaearon]

Fixed in 0.8.2.
Please verify.

https://github.com/facebookincubator/create-react-app/releases/tag/v0.8.2

[niksudan]

My build now succeeds 👍

[godmar]

github is really cool. I filed an issue with nodesource/distributions, which now appears cross-linked here. Maybe we can find out why they have this/where it comes from.

Just wanted to add one thing: events.js is also baked into the node executable in Linux. (This is done via its bootstrapping process where they take a snapshot of the heap that subsequent runs load from.) In other words, doing a require('events') does not actually read /usr/lib/nodejs/events.js even on Linux. It's not clear why they include the source, especially since changing the source wouldn't affect executions of node.js.

[gaearon]

¯\_(ツ)_/¯

Thanks for all your help and for attempting to dig to the very root of the issue too.

[nikhil2882]

In my case, I was getting unable to minify ./node_modules/EventEmitter/src/index.js:16 and after a lot of research problem was still there. so at the end, I went to babel playground and converted the whole file to es5 and paste there and it worked, don't know why but it worked