electron-forge (5.x & 6.x)+Sharp crashing w/o much debug hints

[andrewrt]

• [X ] I have read the contribution documentation for this project.
• [ X] I agree to follow the code of conduct that this project follows, as appropriate.
• [ X] I have searched the issue tracker for an issue that matches the one I want to file, without success.

TLDR:

[email protected] + sharp image module work well in OSX.
sharp image module stand-alone works in Windows 10 (git-bash), but
const sharp = require('sharp') causes the app (renderer) to not load / just show white).

For [email protected]:
In electron-forge 6.x when running, I get an error now - but it doesn't give enough info for me to be able to debug further (see below) the error doesn't tell me what is doing the "syscall" / "read" so I'm hoping someone might have some further ideas on how to get more info on the failed import/require.

Native Module Rebuild Attempts:

I have tried using electron-rebuild and node-gyp to force rebuild sharp for electron, but that changed nothing. Both show that sharp was successfully added / rebuilt.
Using node-gyp to install:
https://gist.github.com/andrewrt/fa3a08f7346ef2c023565ed07c1fecd7

Here's a simple repo to reproduce the problem:
https://github.com/andrewrt/ef6-sharp

Console output when you run electron-forge with the environment variable DEBUG=electron-forge:*. (Instructions on how to do so here). Please include the stack trace if one exists.

andrew@ANDREWT0C72 MINGW64 ~/code/js/electron/test-ef6 (master)
$ electron-forge --verbose start
WARNING: DEBUG environment variable detected.  Progress indicators will be sent over electron-forge:lifecycle
2019-11-07T23:38:37.387Z electron-forge:async-ora Process Started: Checking your system
2019-11-07T23:38:37.391Z electron-forge:check-system checking system, create ~/.skip-forge-system-check to stop doing this
2019-11-07T23:38:38.507Z electron-forge:async-ora Process Succeeded: Checking your system -- after 1.1s
WARNING: DEBUG environment variable detected.  Progress indicators will be sent over electron-forge:lifecycle
2019-11-07T23:38:40.100Z electron-forge:async-ora Process Started: Locating Application
2019-11-07T23:38:40.100Z electron-forge:project-resolver searching for project in: C:\Users\andrew\code\js\electron\test-ef6
2019-11-07T23:38:40.103Z electron-forge:project-resolver electron-forge compatible package.json found in C:\Users\andrew\code\js\electron\test-ef6\package.json
2019-11-07T23:38:40.107Z electron-forge:async-ora Process Succeeded: Locating Application -- after 3ms
2019-11-07T23:38:40.111Z electron-forge:require-search searching [ '@electron-forge/plugin-webpack',
  'C:\\Users\\andrew\\code\\js\\electron\\test-ef6\\@electron-forge\\plugin-webpack',
  'C:\\Users\\andrew\\code\\js\\electron\\test-ef6\\node_modules\\@electron-forge\\plugin-webpack' ] relative to C:\Users\andrew\code\js\electron\test-ef6
2019-11-07T23:38:40.115Z electron-forge:require-search testing @electron-forge/plugin-webpack
2019-11-07T23:38:40.115Z electron-forge:require-search testing C:\Users\andrew\code\js\electron\test-ef6\@electron-forge\plugin-webpack
2019-11-07T23:38:40.119Z electron-forge:require-search testing C:\Users\andrew\code\js\electron\test-ef6\node_modules\@electron-forge\plugin-webpack
WARNING: DEBUG environment variable detected.  Progress indicators will be sent over electron-forge:lifecycle
2019-11-07T23:38:41.568Z electron-forge:plugin:webpack hooking process events
2019-11-07T23:38:41.568Z electron-forge:async-ora Process Started: Preparing native dependencies
2019-11-07T23:38:43.795Z electron-forge:async-ora Process Renamed: Preparing native dependencies  -->  Preparing native dependencies: 0 / 1
2019-11-07T23:38:43.795Z electron-forge:async-ora Process Renamed: Preparing native dependencies: 0 / 1  -->  Preparing native dependencies: 1 / 1
2019-11-07T23:38:43.799Z electron-forge:async-ora Process Succeeded: Preparing native dependencies: 1 / 1 -- after 2.2s
2019-11-07T23:38:43.799Z electron-forge:plugins plugin: "webpack" has taken control of the start command
2019-11-07T23:38:43.807Z electron-forge:async-ora Process Started: Compiling Main Process Code
2019-11-07T23:38:45.539Z electron-forge:async-ora Process Succeeded: Compiling Main Process Code -- after 1.7s
2019-11-07T23:38:45.539Z electron-forge:async-ora Process Started: Launch Dev Servers
2019-11-07T23:38:45.615Z electron-forge:async-ora Process Succeeded: Launch Dev Servers -- after 76ms
2019-11-07T23:38:45.615Z electron-forge:async-ora Process Started: Compiling Preload Scripts
2019-11-07T23:38:45.615Z electron-forge:async-ora Process Succeeded: Compiling Preload Scripts -- after 0ms
2019-11-07T23:38:45.627Z electron-forge:async-ora Process Started: Launching Application
2019-11-07T23:38:45.723Z electron-forge:async-ora Process Succeeded: Launching Application -- after 96ms


Webpack Output Available: http://localhost:9000

2019-11-07T23:38:45.723Z electron-forge:plugin:webpack hooking electron process exit


An unhandled rejection has occurred inside Forge:
{ [Error: EISDIR: illegal operation on a directory, read] errno: -4068, code: 'EISDIR', syscall: 'read' }

Electron Forge was terminated. Location:
{}
2019-11-07T23:38:48.855Z electron-forge:plugin:webpack handling process exit with: { cleanup: true }
2019-11-07T23:38:48.855Z electron-forge:plugin:webpack cleaning webpack watcher
2019-11-07T23:38:48.855Z electron-forge:plugin:webpack cleaning http server
2019-11-07T23:38:48.859Z electron-forge:plugin:webpack stopping logger

What command line arguments are you passing?

`DEBUG` and `--verbose`

What does your config.forge data in package.json look like?

see repo for package/config and reproducing the issue - https://github.com/andrewrt/ef6-sharp/blob/master/src/app.js

Please provide either a failing minimal testcase (with a link to the code) or detailed steps to
reproduce your problem. Using electron-forge init is a good starting point, if that is not the
source of your problem.

Simple project to reproduce:
https://github.com/andrewrt/ef6-sharp/blob/master/src/app.js

[andrewrt]

Issue logged w/ the sharp author - assumed electron-forge v6 might fix, to no avail (https://github.com/lovell/sharp/issues/1951)

[andrewrt]

as noted in https://github.com/lovell/sharp/issues/1951 - got it working.
It was an issue with webpack config - so adding "externals" fixed

[andrewrt]

A quick heads up - the above gets it working in dev mode, but when packaging - the sharp module gets left out in the packaged app.

Trying to figure out how @shadeofgod 's CopyPlugin example would work in the electron-forge flow.

if I add the copy plugin to my webpack.renderer.config.js as so - I still get sharp not found:

const rules = require('./webpack.rules');
const CopyPlugin = require('copy-webpack-plugin');

rules.push({
  test: /\.css$/,
  use: [{ loader: 'style-loader' }, { loader: 'css-loader' }],
});

module.exports = {
  // Put your normal webpack config below here
  module: {
    rules ,
  },
  externals: {
    "sharp": 'commonjs sharp',
  },
  plugins: [
    new CopyPlugin([
      {
        from: './node_modules/sharp/',
        to: './output/node_modules/sharp/', // still under node_modules directory so it could find this module
        // ignore: [...]
      },
    ]),
  ]
};

output from the console:

Uncaught Error: Cannot find module 'sharp'
Require stack:
- C:\Users\andrew\code\js\electron\sharp-ef6\out\test-ef6-win32-x64\resources\app\.webpack\renderer\main_window\index.html
    at Module._resolveFilename (internal/modules/cjs/loader.js:717)
    at Function../lib/common/reset-search-paths.ts.Module._resolveFilename (reset-search-paths.ts:40)
    at Module._load (internal/modules/cjs/loader.js:622)
    at Module._load (electron/js2c/asar.js:717)
    at Function.Module._load (electron/js2c/asar.js:717)
    at Module.require (internal/modules/cjs/loader.js:775)
    at require (internal/modules/cjs/helpers.js:68)
    at Object.<anonymous> (external "sharp":1)
    at n (bootstrap:19)
    at Module.<anonymous> (app.js:33)

[mauricedoepke]

I am facing the same problem.

Woudl be great if anyone had an idea on how to fix this.

[aperkaz]

@mauricedoepke any updates on this? I am facing the same issue...

[mauricedoepke]

@aperkaz
Even after lots of research and fiddling with the webpack config I was not able to make sharp work with electron-forge.

Some alternative solutions are:

• using jimp, apure js image manipulations lib which runs everywhere, but is painufully slow and runs out of memory for larger images.
• using https://github.com/kleisauke/wasm-vips which currently works only in the browser context (uses experimental wasm features that are not yet available in the electron node context)
• using https://github.com/silvia-odwyer/photon which is also wasm based and should run in browser and node (I did not try it with electron forge)

But as even the wasm solutions are much slower then sharp (https://libvips.github.io/libvips/2020/09/01/libvips-for-webassembly.html) I switched to https://www.electron.build/ instead
With electron build I was able to get sharp working.
If you use other native modules then sharp, you need to make sure though, to exclude sharp js from being rebuild by electron-rebuild. This will mess up sharp and you have to reinstall it.

For this reason I have this script in my package.json:

"rebuild": "yarn add sharp && copyfiles sqlite3.tar.gz ./node_modules/better-sqlite3/deps && electron-rebuild -o better-sqlite3",

Here I install a fresh version of sharp to fix possible corruptions of the installation and then only rebuild sqlite with electron rebuild to not corrupt sharp again.

This is a bit hacky, but works.

If you have detail questions, just write me via my email in my profile.

[aperkaz]

@mauricedoepke I managed to get it to work (on the main process), by using babel and modifying the webpack configs.

Repo with working example: https://github.com/aperkaz/electron-forge-sharp

I could not make it work from a render process, so any PR is welcome 🙂

[timfish]

@vercel/webpack-asset-relocator-loader will not support every native node module and the majority of maintainers are not going to want to or be able to support the way it works.

For this reason I've created this plugin that helps include native modules during packaging.

You pass it a list of modules and it searches for all dependencies and includes them in node_modules in the packaged app.