Electron-builder: Bundling Electron During Build Process | Support NightmareJS

[kamescg]

• Version: "electron-builder": "^18.0.1",

electron-builder is amazing and thank you so much for the all the hard work in maintaining the package.

I've been building a NightmareJS Dashboard, which requires Electron as a dependency, so I have to run a post-build script to workaround for bundling an "electron" or "electron-prebuilt" dependency back into the bundled application.

From what I can gather about electron-builder it

1. Performs a string comparison for electron and electron-prebuilt in the dependency chain and quits the packaging process if found.
2. Therefore forking electron module and naming electron-nightmare, it's possible to bypass the warning and include Electron as a dependency.
3. However, during the packaging, electron-builder will look for an Electron binary signature(?) and rebuild any dependency matching that signature during the packing process.

After peeking into the package application, it seems my new file "electron-nightmare" is transformed in the same manner Electron is, even adding it to the app.asar.unpacked folder. When this happens NightmareJS is unable to spawn a Dashboard interface, because it's expecting the original Electron binary, but can only find the new transformed Electron dependency.

Based off the current setup of electron-builder, how feasible is it to add the ability to inject a clean Electron binary, back into the dependency chain, during the build process, without interfering with electron-builder build process?

[develar]

1. option to allow. Not by default because often it is an error.
2. Option to ignore on sign.

If you can provide sample app — it will help. Or send PR.

[kamescg]

@develar Thanks for the quick response. That's really cool. Thank you.

I'm happy to submit a pull request.

Was hoping maybe someone familiar with the code could point me in the right direction? I'm totally unfamiliar with the electron-builder repo, besides of course using it to bundle my applications.

I looked at the debug information and associated code it wasn't initially apparent to me where electron-builder was finding my "electron-nightmare" dependency, transforming/bundling the forked Electron binary into the signed application.

Any insights?

[develar]

You need to modify ignore function — https://github.com/electron-userland/electron-builder/blob/master/packages/electron-builder/src/macPackager.ts#L193 to exclude nightmare from code sign.

[develar]

If you want to just pack your electron binary into resources dir — please use extraResources option. To be sure that it will be not packed into asar and will be located in convenient place (hint — use from/to form of extraResource file pattern).

[kamescg]

I'll take a look at the extraResources option first, because that sounds like the best short/long term solution for this particular setup I'm using.

[develar]

Code sign will be broken since we sign extraResources also. But... who knows:)

[kamescg]

Packaging for OSX right now - simple enough settings it seems?

"extraResources": [
    {
      "from": "node_modules/electron-nightmare",
      "to": "electronNightmare"
    }
  ],

[develar]

Yes. It will work even if nightmare in the dev deps. But! Please consider exclude nightmare package.json because all magic that electron builder applies for deps, is not applied for extra resources by design.

[kamescg]

Thank You

[develar]

Extra files and extra resources are copied after asar packaging.

Magic — dep package file preprocessed to remove npm/yarn/not required fields to ensure security/delta update/consistent signature.

[develar]

For what do you want to pack it into asar file?

[kamescg]

The reason it would be beneficial to include the pure Electron binary back into the app.asar file is because the current Webpack import configuration does not support stepping up the path system into the Resources files, therefore I can not require Electron as a dependency into my bundled application.

To put it simply, the Application for all intents and purposes can not require modules that aren't included in the app.asar file.

Solution
Unless, we switch Webpack to have a more "global" view of the parent Folder and it's siblings.

node: {
    __dirname: false
  },

I enabled this in my webpack.config.base.js file, which allows me to step into the Resources folder, and access the sibling appShadow folder, which contains the Electron binary required for NightmareJS to instantiate.

Of course this causes issues for Electron entry files, because it's starts looking for resources in the wrong places.

When I run my application and attempt to run NightmareJs the following error is given in the console. Essentially, it's looking in the root directory, instead of the module folder, because of switching the "__dirname" setting to false.

I haven't looked any further into issue, and it might be simple fix.

ENOENT: no such file or directory, open '/Users/kamescg/Documents/Projects/SynapseHub/app/path.txt'   

So it seems we might have 2 solutions moving forward.

1. Correct the forked Electron package, fixing the path/fs errors, and continue to use the extraResources option as the go-to solution

However, I'm not convinced yet, because it's most likely going to introduce bugs into the path system for everything else... So we fix one thing and potentially make it unstable for everything else.

That being said, I would totally love the idea of multiple Electron applications on my computer sharing dependencies/resources. Sharing core dependencies, like my library of React components, Firebase wrappers, and other packages amongst the multiple Electron apps I'm working on sounds intriguing.

1. Create a PR to add a CLI flag or package.json setting, so we can ignore the forked Electron binary, including the "pure" Electron binary in the app.asar file.

[kamescg]

I'm going for Option #2

From what I can gather, I need to find the code in the platformPackager.ts?

I can see all the modules being rebuilt in the yarn.ts, but I can't find where the forked Electron binary, electron-nightmare, is being transformed?

Looking in the doPack method, but I can't find explicitly where the forked Electron binary is being found and transformed?

https://github.com/electron-userland/electron-builder/blob/master/packages/electron-builder/src/platformPackager.ts#L123

[develar]

1. Have you tried require('module').globalPaths.push(your path)?

2. To execute native binary it must be unpacked. Electron can do it for you — on each run unpack 50 mb exe .... so, I don't recommend to pack it into asar.

[kamescg]

1. I can't find documentation for globalPaths and it's unavailable on require as a method?

2. That makes sense... So Electron can't be in the app.asar binary anways...

The issue remains how to get Webpack to include a module from the Resources folder?

[develar]

1. See example https://github.com/electron-userland/electron-builder/wiki/Loading-App-Dependencies-Manually

2. Not clear for me ;)

[kamescg]

@42cph4nzt48 The current version of Electron in my package.json is 1.6.10

"electron": "^1.6.10",

What I've been doing to publish the Electron/NightmareJS application is to run a post-build script, which unzips the app.asar folder, located in the OSX Application folder. However, I need to compile for Windows, which is why I need to solve this issue, so it runs during packaging.

You need the asar CLI to extract the app.asar using asar extract app.asar app

Then I replace my compiled electron-nightmare Electron binary, which solves the issue running in production..

[develar]

If you are not able to solve issue, you can provide link to project and I will try to fix it when i will have time.

[kamescg]

@develar Thanks for the assistance - I think I have a post-build script for Windows done. In the meantime, I'll probably just use this, so I can ship the projects to clients today.

I'll get a repo ready this week to review and figure out the best steps forward :)

[mariodavid]

@KamesCG would you share your solution? I had the exact same issue (coming from here)
I want to package my electron app using electron-builder with a nightmare.js script.

That would be great! Thx

[kamescg]

Essentially I made placeholder for Electron by requiring a forked electron-prebuilt package and naming it electron-nightmare.

I want to stress...my STRATEGY is NOT the CORRECT one.

I just needed to get this shipped for a small group of people and this jankyness was fine. @mariodavid There seems to be enough people facing this problem we really need to create a better best practice for handling Electron/NightmareJS applications, bundled with ElectronBuilder/Webpack.

1. Fork the electron-prebuilt package, so we can circumvent the electron-builder string comparison and satisfy Webpack requirements with our placeholder. Electron Builder transforms the original Electron binary, and would transform the forked Electron binary as well, if we don't add the package as an extraResource.

"electron-nightmare": "https://github.com/KamesCG/electron-nightmare.git",

1. Add the electron-nightmare package in the extraResources setting within the top level package.json file.

electron-builder will NOT bundle electron-nightmare into the app.asar package, so we have to extract the app.asar folder into a folder named app.

"build": {
    "extraResources": [
      {
        "from": "node_modules/electron-nightmare",
        "to": "appShadow/node_modules/electron-nightmare",
        "filter": [
          "**/*"
        ]
      }
    ],
...

1. Extract the app.asar package in the Application Resources(osx)/resources(win/linux) folder by running asar extract app.asar app in the Command Line.

2. Move the untransformed, electron-nightmare module package from the resources/appShadow/node_modules folder into the resources/app/node_modules folder.

As mentioned by @develar Binaries can't be run from an app.asar file, so no running asar pack app app.asar for compression of the package again.

[Raidus]

@KamesCG great solution!

It works out in my case.

Except for the last step: compression of the package again.

I'm not sure if it is related to asar (realted asar issue) or something what I've did wrong along your instruction.

The full example is on https://github.com/Raidus/test-electron-nightmare

Do you have an idea?

My configuration
MacOS Sierra (10.12.5)
Node 6.11.0
npm 3.10.7

[kamescg]

@Raidus Glad it's working for other people :)

Sorry... I should have made it more clear (I reread what I wrote). We CAN NOT pack the asar back together and MUST leave it as a folder named app uncompressed.

The effect of this is we can NEVER pack Electron pack into the app.asar folder, but we can recompress everything else again and just leave Electron uncompressed, which is really the next route we route we need to take for this to be viable for people to build applications with and what develar was getting at (I think) in his responses.

This has to do more with the application bundle process and Webpack. Electron must be passed back into Nightmare as a requirement, using a relative path system, instead of a localized "app" path system using the current setup.

My first thought is Electron needs to be dynamically required in the main render process and passed to render process (BrowserWindow) when Nightmare is getting initialized.

The main issue is relying to heavily on Webpack as the "final" application bundle process. I think ultimately another abstraction layer has to built on top of our 'interface' and 'logic' bundle processes, where Desktop related requirements can be met, instead of mostly Browser'ish type requirements being met.

More distinguishing between browser/desktop requirements might help the community address issues arising from requiring more fine-tuned control over the final packaged product.

[Raidus]

@KamesCG Thanks again for the great explanation! I really appreciate your detailed answer 👍

[toffebjorkskog]

Did you @Raidus get it ti work as well?

[kamescg]

@toffebjorkskog Need any help?

[ssimo3lsuhsc]

I actually want to use two Electron processes in my app, which has both automated and non-automated components. I have tried Nightmare, and it will work for the automated components. However, to just show a page and ask for input, it really doesn't work. I don't see why I should have it time out. In a real life world, apps do wait infinitely for input.

[kamescg]

@ssimo3lsuhsc Javascript is asynchronous - you have to deal with execution using callbacks, promises or generator functions.

[WebPro9369]

@KamesCG Thanks for this very nice solution!
Can you please let me know how to use such a solution for Windows package(installer, not unpacked version)?
Thanks

[JonathanKolnik]

Hey @KamesCG I followed your instructions and that didn't seem to solve my issue which I put in more detail here https://github.com/electron-userland/electron-builder/issues/2747. Not sure if it is related, but any insight you have would be greatly appreciated. Note: this happens in my dev env not only when packaging the app.

Thanks!

[dzt]

Has anybody ever figured out how to repackage back into an asar archive?