Vue-cli-plugin-electron-builder: Errors retrieving paths, node-module externalization?

[lspohn]

Describe the bug

Hello! Thanks a lot to your for providing this plugin.

I tried to migrate my project from electron-webpack to vue-cli-plugin-electron-builder cause HMR is broken in electron-webpack atm.

I created a new project with vue cli and then merged the automatically created files with my project and deleted the unneccesary ones.

When I run serve:electron, I get the error shown in the screenshot. Note that leveldown is a native dependency in my project and shouldnt be included in Webpack.

With electron-webpack, the "renderer.js" webpack file has 4,000 lines.
With vue-cli-plugin-electron-builder, I get to 19,000 lines in "app.js"!

I suspected that webpack didn't externalize the node-modules.

So I then changed the vue.js.config to externalize all packages in my package.json by webpack (see below). Then, I get a strange path error. It seems to try to fetch modules from:

"C:[appfolder]node_moduleselectrondistresourcesdefault_app.asarnode_modules"
"C:[appfolder]node_moduleselectrondistresourcesnode_modules"
"C:[appfolder]node_moduleselectrondistresourceselectron.asarrendererapiexports"
and throws an error (cause package is located in "C:[appfolder]node_modules")

To summarize:

• Something's wrong with the module externalization
• Something's wrong with the path.

Thanks a lot for your help.

To Reproduce
Steps to reproduce the behavior:

• Add node-modules as webpack.externals
• Check projects with native libraries

Environment (please complete the following information):

• OS and version: Win 10 x64 Prof
• Node version: 8.11.4
• npm version: recent
• yarn version (if used): recent
• other vue plugins used:
  "vue": "^2.5.17",
  "vue-async-computed": "^3.4.0",
  "vue-electron": "^1.0.6",
  "vue-router": "^3.0.1",
  "vue-shortkey": "^3.0.2",
  "vuedraggable": "^2.16.0",
  "vuetify": "^1.1.11"
  "@vue/cli-plugin-babel": "^3.0.0",
  "@vue/cli-plugin-eslint": "^3.0.0",
  "@vue/cli-plugin-unit-jest": "^3.0.0",
  "@vue/cli-service": "^3.0.0",
  "@vue/eslint-config-standard": "^3.0.0",
  "@vue/test-utils": "^1.0.0-beta.20",
  "vue-cli-plugin-electron-builder": "^1.0.0-beta.6",
  "vue-template-compiler": "^2.5.17",
• custom config for vcp-electron-builder:

var nodeExternals = require('webpack-node-externals');
module.exports = {
  pluginOptions: {
    electronBuilder: {
      chainWebpackRendererProcess: config => {
        config.externals(nodeExternals(
          {
            modulesFromFile: true,
            whitelist: ['vue']
          }
        ))
        return config
      },
  }
}
}

[nklayman]

For me, everything worked fine for build. However, it does fail for serve (will continue to look into). As for the larger app.js, this is caused by vue-cli's build config. All of the renderer process build (except for electron-specific additions) is the same as an SPA (regular build command) build. If you think externalization should be enabled by default for electron builds only, I will look into that.

[lspohn]

Hey, thanks for your fast answer. It seems that other boilerplates externalize dependencies listed in package.json automatically. But I don't know if that is of any advantage - so maybe I'd just add the option to do so in a future release.

Coming back to my problem - WebPack with your plugin also includes libraries that are built as native library and therefore should definitely not become included as .js.

Do you know how to extend the options in your script so that doesn't happen & so that the native libraries (I think they are compiled as .node-Module) are loaded?

When I used electron-webpack, no further config was neccessary, it did work out of the box. Maybe the config from that can be adapted?

[nklayman]

Alright, I have a solution! Set vue.config.js to:

module.exports = {
    pluginOptions: {
        electronBuilder: {
            chainWebpackRendererProcess: config => {
                config.module
                    .rule('node-loader')
                    .test(/\.node$/)
                    .use('node')
                    .loader('node-loader')
                config.resolve.alias.set(
                    'leveldown',
                    'leveldown/build/Release/leveldown.node'
                )
                return config
            }
        }
    }
}

This redirects leveldown to the actual leveldown.node file and configures node-loader (which you will need to install with yard add -D node-loader). This might be a poor way to do this (I know very little about native modules/externals). If you think so, I will look to find a better way.

EDIT: make sure to run yarn electron-builder install-app-deps after installing/upgrading node_modules and before running serve:electron. build:electron does this automatically.

[nklayman]

Electron-Webpack uses node loader as well. I think that they don't need the alias mapping becuase they use the recommended electron structure, while vcp-electron-builder has to use that of vue-cli. They might have some other setting that makes it work that I would love to implement, so let me know if you find anything.

[lspohn]

Hi! That solved the issue with Leveldown, great. Unfortunately, I get other bugs which are related to native dependencies.

As the bugs don't happen with electron-webpack, I really don't know what's going on. I'll further investigate this.

Thanks a lot for your help so far!

[nklayman]

Here is the config for electron-vue. It seems to be the same as what I have, although __node and __dirname are set to true, that could be the issue.
https://github.com/SimulatedGREG/electron-vue/blob/master/template/.electron-vue/webpack.renderer.config.js

[nklayman]

Alright, so the issue is that module.paths doesn't include [path to app]/node_modules. As a temporary fix you can add it to module.paths with push() and externalize leveldown. Then import leveldown from 'leveldown' will work (make sure to remove alias config). Now, I just need to find a way to set it where it doesn't interfere with SPA build/serve.

[nklayman]

What do you think about injecting it into the index.html? This is what vue-electron and electron-webpack do.

[lspohn]

Hi!

I added the absolute path to node_modules via: config.resolve.modules.prepend(), and then tried to externalize all dependencies. Curiously, it still can't find the modules and throws an error.

This all is quite strange. I could get the app running with the aliasses, but had the drawback that I had to specify exact paths to the native builds - but they have to be changed when building for other platforms. Also, I still get strange errors within my app, while with electron-webpack all worked fine without any further config.

Maybe, adding the node-loader by default and provide a flag to externalize all dependencies listed in package.json (+ whitelists, siminar to electron-webpack), into you project would be great. Then users like me, which deal with various native depenencies, don't get the performance benefit of bundling everything but it should then work at least.

I didn't understand what you mean with injecting it into "index.html" - but if the others do it, I'd just copy it. Both other projects seem to be less good maintained. So I really appreciate the work you put into this!

[nklayman]

Sorry, not config.modules. Instead put require('module').globalPaths.push(path to node_modules) inside the head of you index.html file. I think the path has to be absolute. By injecting into the index.html, I mean adding that automatically.

[lspohn]

Ah, I tried - getting the error message:

external "leveldown"?07eb:1 Uncaught ReferenceError: leveldown is not defined
at eval (external "leveldown"?07eb:1)
at Object.leveldown (app.js:20021)
at __webpack_require__ (app.js:725)

(my IDE also said, "Unresolved variable globalPaths")

This is my vue.config.js now.

chainWebpackRendererProcess: config => { config.module .rule('node-loader') .test(/\.node$/) .use('node-loader') .loader('node-loader') config.externals({leveldown: 'leveldown', sodium_native: 'sodium-native', grpc: 'grpc', node_sass: 'node-sass'}) return config }

Ideas? Thank you!

[nklayman]

Hmm, I'm not sure what's wrong. I should have the auto-externalization feature ready later today, so you can just wait for that.

[nklayman]

Use the node externals plugin as well for a temporary fix

[nklayman]

I'd rather not use that as it is faster to bundle of requiring

[nklayman]

I got it to work without the node-externals plugin. Just need to write tests and we're good.

[lspohn]

Very excited, I'll try it out quickly then!

[nklayman]

It's here! Give it a shot! While it won't be included till 1.0.0-rc.1, you can use it now by setting vue-cli-plugin-electron-builder's version to nklayman/vue-cli-plugin-electron-builder (pulls from github repo). After doing that (and re-installing deps), add this code:

To <head> of index.html:
<% if (VUE_APP_NODE_MODULES_PATH !== "false") { %><script>require('module').globalPaths.push('<%= VUE_APP_NODE_MODULES_PATH %>')</script><% } %>

To beginning of background.js (just before any native dep imports):

const isDevelopment = process.env.NODE_ENV !== 'production'
if (isDevelopment) {
  // Don't load any native (external) modules until the following line is run:
  require('module').globalPaths.push(process.env.NODE_MODULES_PATH)
}

and remove the second const isDevelopment = process.env.NODE_ENV !== 'production'

Remove all the vue.config.js stuff and you can uninstall webpack-node-externals and node-loader.

Then, you can use native modules! The will be externalized, while nothing else will (improves performance).

Let me know if it works! Sorry it took so long. Writing the tests took way longer than expected and school got in the way :).

[lspohn]

Hey!

Just tried it out!

I get the same error like in post 1. It seems to try to load the JS version of leveldown? When I add the "alias", it throws an error because node-loader is missing.

Did you include some script for loading the .node files which include the native binaries? (Or maybe: all the node files in node_externals so that I don't have to specify alias for each package)

Greetings and tanks! 👍

[nklayman]

Try creating a blank project with leveldown and the latest vcp-electron-builder. A blank project works fine for me. I am on Linux though, but will test with windows soon.

[lspohn]

Doesnt work either!

"Cannot find module 'leveldown'"

=> Yarn.lock doesn't include a loader for .node files. Is this missing?

[nklayman]

It shouldn't have a .node loader as it requires the package at runtime, not through webpack. I think it is a windows-related issue

[nklayman]

Also, does it say "Installing native deps with electron-builder" when you run the serve command? Does it work in builds?

[nklayman]

Make sure you invoke the generator of the github version. Create a fresh project and run yarn add --dev nklayman/vue-cli-plugin-electron-builder and then vue invoke electron-builder to make sure it invokes the latest version.

[lspohn]

Ok, I did so.

added leveldown to dependencies.

Then in main.js "import leveldown from 'leveldown'" to test it.

Still same error. Could be related to windows.

Installing native deps works and shows.

No errors with build!

There is no dependency to your package with looks like a node-loader. Maybe something is missing?

Btw: I would add the rebuild process to "postinstall". Rebuilding on every serve takes too much time.

we added in our package.json: "postinstall": "npm rebuild node-sass && yarn rebuild-deps"

[nklayman]

Can you send me the index.html after it gets compiled (use the sources tab in electron's devtools)?

[nklayman]

Windows was having issues. I just released a fix. You will not have to re-invoke the generator again.

[lspohn]

On a fresh project, it works now!

But still not in my main project, where I merged all files generated in the fresh one. However, leveldown is there required in a library by a module itself. Still the error from above.

[nklayman]

Send me the compiled index.html of the existing project and run require('module').globalPaths in the console and send me the output.

[lspohn]

Here is the Error!

globalPaths = C:UserslcsspIdeaProjectsxxx-xxx-xxx↵ode_module

Seems a problem of String escaping.

I html:

[nklayman]

Make sure your index.html has <script>require('module').globalPaths.push('<%= VUE_APP_NODE_MODULES_PATH %>')</script><% } %> in it. As of the latest commit (you may need to update) it replaces all \s with /s, fixing the character escaping problem.

[nklayman]

I also set electron-builder install-app-deps to be part of postinstall. If you have an existing postinstall script, it will be added on.

[lspohn]

Still not!

globalPaths is now:

C:/Users/lcssp/IdeaProjects/xxx-xxx-xxx/node_modules

But windows requires slashes!

[nklayman]

Forward slashes worked fine for me, try setting it manually with backslashes.

[nklayman]

There are some weird caching issues with github repos as nom modules. Make sure to clear your cache and try again.

[nklayman]

Or pull the github repo and link it with yarn link.

[nklayman]

1.0.0-rc.1 has landed with support for native modules. Remove any changes you made after adding leveldown, upgrade to the latest version of vcp-electron-builder, and re-invoke the generator. Then, leveldown should work fine. The generator will not effect any code other than adding the missing requirements for native modules.