Johnny-five: Browserify / Webpack support

Created on 13 Apr 2016  ·  16Comments  ·  Source: rwaldron/johnny-five

At a high level, the goal here would be to make requiring johnny-five in package.json (and perhaps firmata and a [chrome, browser, webusb]-serial library) sufficient for require('johnny-five') to work within browserify/webpack-ed code.

To start off, I created a couple of minimal repro repositories, steps below.

Browserify

Browserify seems to be _mostly_ working, just with the exception of missing mock-firmata—if users add that to devDependencies, it requires OK. Fix might just be to add mock-firmata to optionalDependencies, though I'm not sure what implications of that change might be.

Minimal repro repository: https://github.com/bcjordan/johnny-five-browserify

Repro steps:

git clone https://github.com/bcjordan/johnny-five-browserify.git
npm install
npm run build

Expected: index.html has a window.five object
Actual: error on build

Brians-MacBook-Pro:johnny-five-browserify brian$ npm install && npm run build && open index.html
npm WARN package.json [email protected] No description
npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] No README data

> [email protected] build /Users/brian/code/johnny-five-browserify
> browserify main.js -o bundle.js

Error: Cannot find module 'mock-firmata' from '/Users/brian/code/johnny-five-browserify/node_modules/johnny-five/lib'
    at /usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:46:17
    at process (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:173:43)
    at ondir (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:188:17)
    at load (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:69:43)
    at onex (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:92:31)
    at /usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:22:47
    at FSReqWrap.oncomplete (fs.js:95:15)

npm ERR! Darwin 14.5.0
npm ERR! argv "node" "/usr/local/bin/npm" "run" "build"
npm ERR! node v0.12.7
npm ERR! npm  v2.9.1
npm ERR! code ELIFECYCLE
npm ERR! [email protected] build: `browserify main.js -o bundle.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script 'browserify main.js -o bundle.js'.
npm ERR! This is most likely a problem with the firmata-browserify-test package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     browserify main.js -o bundle.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls firmata-browserify-test
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/brian/code/johnny-five-browserify/npm-debug.log

Webpack

Minimal repro repository: https://github.com/bcjordan/johnny-five-webpack

Repro steps:

git clone https://github.com/bcjordan/johnny-five-webpack.git
npm install
npm run build

Expected: index.html has a window.five object
Actual: error on build (below), error in console: bundle.js:9885 Uncaught Error: Cannot find module "repl"

Brians-MacBook-Pro:johnny-five-webpack brian$ ni && nrb && open index.html
npm WARN package.json [email protected] No description
npm WARN package.json [email protected] No README data
|
> [email protected] install /Users/brian/code/johnny-five-webpack/node_modules/webpack/node_modules/watchpack/node_modules/chokidar/node_modules/fsevents
> node-pre-gyp install --fallback-to-build

[fsevents] Success: "/Users/brian/code/johnny-five-webpack/node_modules/webpack/node_modules/watchpack/node_modules/chokidar/node_modules/fsevents/lib/binding/Release/node-v14-darwin-x64/fse.node" is installed via remote
\
> [email protected] install /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport
> node-pre-gyp install --fallback-to-build

[serialport] Success: "/Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport/build/Release/serialport.node" is installed via remote
[email protected] node_modules/webpack
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected]
├── [email protected]
├── [email protected] ([email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected])
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected])
├── [email protected] ([email protected], [email protected])
├── [email protected] ([email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected])
└── [email protected] ([email protected], [email protected], [email protected])

[email protected] node_modules/johnny-five
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected] ([email protected])
├── [email protected]
└── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected])

> [email protected] build /Users/brian/code/johnny-five-webpack
> webpack main.js bundle.js



Hash: a99ec70128630264c2ca
Version: webpack 1.12.15
Time: 1623ms
    Asset     Size  Chunks             Chunk Names
bundle.js  1.28 MB       0  [emitted]  main
   [0] ./main.js 38 bytes {0} [built]
    + 137 hidden modules

WARNING in ./~/johnny-five/~/serialport/~/bindings/bindings.js
Critical dependencies:
76:22-40 the request of a dependency is an expression
76:43-53 the request of a dependency is an expression
 @ ./~/johnny-five/~/serialport/~/bindings/bindings.js 76:22-40 76:43-53

WARNING in ./~/johnny-five/~/serialport/~/bindings/package.json
Module parse failed: /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport/node_modules/bindings/package.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
|   "name": "bindings",
|   "description": "Helper module for loading your native module's .node file",
|   "keywords": [
 @ ./~/johnny-five/~/serialport/~/bindings ^\.\/.*$

WARNING in ./~/johnny-five/~/serialport/~/bindings/README.md
Module parse failed: /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport/node_modules/bindings/README.md Line 2: Unexpected token ===
You may need an appropriate loader to handle this file type.
| node-bindings
| =============
| ### Helper module for loading your native module's .node file
|
 @ ./~/johnny-five/~/serialport/~/bindings ^\.\/.*$

ERROR in ./~/johnny-five/lib/board.js
Module not found: Error: Cannot resolve module 'mock-firmata' in /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/lib
 @ ./~/johnny-five/lib/board.js 75:19-42 145:6-29

ERROR in ./~/johnny-five/lib/repl.js
Module not found: Error: Cannot resolve module 'repl' in /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/lib
 @ ./~/johnny-five/lib/repl.js 1:11-26

ERROR in ./~/johnny-five/~/serialport/serialport.js
Module not found: Error: Cannot resolve module 'fs' in /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport
 @ ./~/johnny-five/~/serialport/serialport.js 22:9-22

ERROR in ./~/johnny-five/~/serialport/lib/list-unix.js
Module not found: Error: Cannot resolve module 'fs' in /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport/lib
 @ ./~/johnny-five/~/serialport/lib/list-unix.js 4:9-22

ERROR in ./~/johnny-five/~/serialport/lib/list-unix.js
Module not found: Error: Cannot resolve module 'child_process' in /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport/lib
 @ ./~/johnny-five/~/serialport/lib/list-unix.js 3:11-35

ERROR in ./~/johnny-five/~/serialport/~/bindings/bindings.js
Module not found: Error: Cannot resolve module 'fs' in /Users/brian/code/johnny-five-webpack/node_modules/johnny-five/node_modules/serialport/node_modules/bindings
 @ ./~/johnny-five/~/serialport/~/bindings/bindings.js 6:9-22
Brians-MacBook-Pro:johnny-five-webpack brian$
picture bcjordan

Most helpful comment

I just ran into this issue. I'm using the latest repository head:

[email protected] (git://github.com/rwaldron/johnny-five.git#dd8b1694817e9aa8818c6b2a97c2a5d8d622661f)

The errors I get are:

WARNING in ./~/bindings/bindings.js
Critical dependencies:
76:22-40 the request of a dependency is an expression
76:43-53 the request of a dependency is an expression
 @ ./~/bindings/bindings.js 76:22-40 76:43-53

WARNING in ./~/bindings/README.md
Module parse failed: /path/to/my/app/node_modules/bindings/README.md Unexpected token (2:3)
You may need an appropriate loader to handle this file type.
SyntaxError: Unexpected token (2:3)
    at Parser.pp.raise (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:920:13)
    at Parser.pp.unexpected (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:1483:8)
    at Parser.pp.parseExprAtom (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:330:12)
    at Parser.pp.parseExprSubscripts (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:225:19)
    at Parser.pp.parseMaybeUnary (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:204:17)
    at Parser.pp.parseExprOp (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:171:41)
    at Parser.pp.parseExprOp (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:173:19)
    at Parser.pp.parseExprOps (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:153:15)
    at Parser.pp.parseMaybeConditional (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:133:19)
    at Parser.pp.parseMaybeAssign (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:110:19)
 @ ./~/bindings ^\.\/.*$

ERROR in ./~/johnny-five/lib/board.js
Module not found: Error: Cannot resolve module 'mock-firmata' in /path/to/my/app/node_modules/johnny-five/lib
 @ ./~/johnny-five/lib/board.js 143:6-29

You can get rid of the error by adding new webpack.IgnorePlugin(/mock-firmata/) to webpack's plugins array. To stop webpack from trying to parse the readme, add new webpack.ContextReplacementPlugin(/bindings$/, /^$/) to that same array. 

plugins: [
    new webpack.IgnorePlugin(/^mock-firmata$/),
    new webpack.ContextReplacementPlugin(/bindings$/, /^$/)
]

The first warning complains about dynamic imports. The webpack guys think that they should not be used at all, but I don't think that is possible in this case. The best way to deal with this is to add bindings to the externals array:

So the webpack config looks like this:

module.exports = {
  plugins: [
    new webpack.IgnorePlugin(/^mock-firmata$/),
    new webpack.ContextReplacementPlugin(/bindings$/, /^$/)
  ],
  externals: ["bindings"]
}
picture brodo on 22 Apr 2016
👍10 ❤3 🎉3

All 16 comments

Thanks for the report.

Browserify seems to be mostly working, just with the exception of missing mock-serialport—if users add that to devDependencies, it requires OK. Fix might just be to add mock-serialport to optionalDependencies, though I'm not sure what implications of that change might be.

I think you meant mock-firmata—I just pushed a refactoring of the tests such that the source never needs to require mock-firmata directly.

rwaldron picture rwaldron on 13 Apr 2016

All of the webpack errors are related to webpack settings, aren't they? It looks like webpack is trying to include things that it either shouldn't or that need special instructions. Someone that has better understanding of webpack should advise

rwaldron picture rwaldron on 13 Apr 2016

@rwaldron thanks for taking this on! Updated test johnny-five-browserify repo to use v0.9.41, it looks like browserify might still be choking on this remaining require("mock-firmata") https://github.com/rwaldron/johnny-five/blob/master/lib/board.js#L143 — repro steps:

git clone https://github.com/bcjordan/johnny-five-browserify.git
npm install
npm run build

Error:

$ npm run build

> [email protected] build /Users/brian/code/johnny-five-browserify
> browserify main.js -o bundle.js

Error: Cannot find module 'mock-firmata' from '/Users/brian/code/johnny-five-browserify/node_modules/johnny-five/lib'
    at /usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:46:17
    at process (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:173:43)
    at ondir (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:188:17)
    at load (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:69:43)
    at onex (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:92:31)
    at /usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:22:47
    at FSReqWrap.oncomplete (fs.js:95:15)
bcjordan picture bcjordan on 18 Apr 2016

https://github.com/rwaldron/johnny-five/commit/624dc7af1d3f9fbbef95dfe76d8b1b8dd75fcb3d

are you using latest push of johnny-five?

henricavalcante picture henricavalcante on 22 Apr 2016

I just ran into this issue. I'm using the latest repository head:

[email protected] (git://github.com/rwaldron/johnny-five.git#dd8b1694817e9aa8818c6b2a97c2a5d8d622661f)

The errors I get are:

WARNING in ./~/bindings/bindings.js
Critical dependencies:
76:22-40 the request of a dependency is an expression
76:43-53 the request of a dependency is an expression
 @ ./~/bindings/bindings.js 76:22-40 76:43-53

WARNING in ./~/bindings/README.md
Module parse failed: /path/to/my/app/node_modules/bindings/README.md Unexpected token (2:3)
You may need an appropriate loader to handle this file type.
SyntaxError: Unexpected token (2:3)
    at Parser.pp.raise (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:920:13)
    at Parser.pp.unexpected (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:1483:8)
    at Parser.pp.parseExprAtom (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:330:12)
    at Parser.pp.parseExprSubscripts (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:225:19)
    at Parser.pp.parseMaybeUnary (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:204:17)
    at Parser.pp.parseExprOp (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:171:41)
    at Parser.pp.parseExprOp (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:173:19)
    at Parser.pp.parseExprOps (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:153:15)
    at Parser.pp.parseMaybeConditional (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:133:19)
    at Parser.pp.parseMaybeAssign (/path/to/my/app/node_modules/webpack/node_modules/acorn/dist/acorn.js:110:19)
 @ ./~/bindings ^\.\/.*$

ERROR in ./~/johnny-five/lib/board.js
Module not found: Error: Cannot resolve module 'mock-firmata' in /path/to/my/app/node_modules/johnny-five/lib
 @ ./~/johnny-five/lib/board.js 143:6-29

You can get rid of the error by adding new webpack.IgnorePlugin(/mock-firmata/) to webpack's plugins array. To stop webpack from trying to parse the readme, add new webpack.ContextReplacementPlugin(/bindings$/, /^$/) to that same array. 

plugins: [
    new webpack.IgnorePlugin(/^mock-firmata$/),
    new webpack.ContextReplacementPlugin(/bindings$/, /^$/)
]

The first warning complains about dynamic imports. The webpack guys think that they should not be used at all, but I don't think that is possible in this case. The best way to deal with this is to add bindings to the externals array:

So the webpack config looks like this:

module.exports = {
  plugins: [
    new webpack.IgnorePlugin(/^mock-firmata$/),
    new webpack.ContextReplacementPlugin(/bindings$/, /^$/)
  ],
  externals: ["bindings"]
}
brodo picture brodo on 22 Apr 2016
👍10 ❤3 🎉3

Sorry, I completely forgot to get rid of that from the source :\

Just waiting for a commit on master to go green then I'll do a release with that removed.

rwaldron picture rwaldron on 23 Apr 2016

Ok, please try [email protected]

rwaldron picture rwaldron on 23 Apr 2016
❤2

Thanks a lot, that fixes the error for me!

brodo picture brodo on 23 Apr 2016

@bcjordan have you tried this yet?

rwaldron picture rwaldron on 4 May 2016

Excellent, Browserify works too! Closing out. Thanks @rwaldron!

bcjordan picture bcjordan on 4 May 2016

Awesome! Thanks for the patience and great reports everyone!!

rwaldron picture rwaldron on 5 May 2016

It wasn't quite clear to me what the resolution was on this. Did you end up externally loading bindings.js outside of the bundle? I have managed to calm webpack down, but not to successfully load bindings.js as a dependency for other modules inside the bundle.

akrueger picture akrueger on 31 Aug 2016
👍4

Add NormalModuleReplacementPlugin to webpack.config.js:

plugins: [
    new webpack.NormalModuleReplacementPlugin(
        /^bindings$/,
        require.resolve("./bindings")
    )
]

Create binding.js:

module.exports = exports = function (str) {
    if(str === "binding"){
        const result = require("../node_modules/ref/build/Release/binding.node");
        result.path = "../node_modules/ref/build/Release/binding.node";
        return result;
    }
    else if(str === "ffi_bindings.node"){
        const result = require("../node_modules/ffi/build/Release/ffi_bindings.node");
        result.path = "../node_modules/ffi/build/Release/ffi_bindings.node";
        return result;

    }
};

Install node-loader:

npm install node-loader

Configure rules -> node-loader on webpack.config.js:

rules: [
    {
        test: /\.node$/,
        use: 'node-loader'
    }
]

Configure extensions -> .node on webpack.config.js:

extensions: ['.ts', '.js', ..., '.node']
kslhunter picture kslhunter on 17 Apr 2017
👍7 🎉4 ❤1

@kslhunter do u still need
module.exports = { plugins: [ new webpack.IgnorePlugin(/^mock-firmata$/), new webpack.ContextReplacementPlugin(/bindings$/, /^$/) ], externals: ["bindings"] }
I did what you posted above but I got another error:
Uncaught TypeError: __webpack_require__(...) is not a function at eval (ref.js:5)
do u have an idea?

sylhero picture sylhero on 15 Aug 2017

@sylhero I do not need it. I solved using my solution only.

kslhunter picture kslhunter on 2 Jul 2018

If anyone's interested, I've got a very early version of a bindings-loader I'm using: https://github.com/simonbuchan/bindings-loader

Let me know if it helps or there are any changes that would help!

simonbuchan picture simonbuchan on 4 Oct 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

cannehag picture cannehag  ·  12Comments
Spy474 picture Spy474  ·  10Comments
lezabour picture lezabour  ·  12Comments
TheBekker picture TheBekker  ·  6Comments
IainIsCreative picture IainIsCreative  ·  10Comments