Electron-packager: Packager runs slower due to moving the app between drives, lots of dependencies, and using NPM

[fzyzcjy]

• [x] I have read the contribution documentation for this project.
• [x] I agree to follow the code of conduct that this project follows, as appropriate.
• [x] I have searched the issue tracker for an issue that matches the one I want to file, without success.

Please describe your issue:
I cloned the electron quick start project, and then run this command: set DEBUG=electron-packager && electron-packager ./ --arch=x64 --platform=win32 (in windows 10) to package it. However, it takes me 40 seconds to finish!!! And when I used it in my real electron app, it takes FOREVER to finish...

From the output below, I am very confused about two things:

• The time it claims to cost is not equal to what it actually costs. For example, the last 2 line says +235ms but in fact several seconds.
• Why does it take such a long time? Why the whole task take ~1min? Why the rename task takes +14s? Shouldn't rename as fast as a flash?

Additional information: it seems that these two things takes really long time:

• prune.
• copy the folder from temporary location to the dist location.

PS. About half a month before, I ran this command and the speed was good. The electron-quick-start can be packaged very fast, and my project can be packaged in less than 1min.

It really drives mad, so if anyone can help, I would really appreciate it!!! Thank you guys!!!

Console output when you run electron-packager with the environment variable DEBUG=electron-packager. Please include the stack trace if one exists.

The output is like this (I have marked where it stucks) :


    D:\Swap\electron-quick-start>set DEBUG=electron-packager && electron-packager ./ electron --arch=x64 --platform=win32 --overwrite --no-prune
      electron-packager Electron Packager 8.7.2 +0ms
      electron-packager Node v7.4.0 +0ms
      electron-packager Host Operating system: win32 (x64) +0ms
      electron-packager Packager Options: {"_":["./","electron"],"all":false,"deref-symlinks":true,"download":{"strictSSL":true},"overwrite":true,"prune":false,"quiet":false,"arch":"x64","platform":"win32","dir":"./","name":"electron","protocols":[]} +0ms
      electron-packager Target Platforms: win32 +16ms
      electron-packager Target Architectures: x64 +0ms
      electron-packager Inferring appVersion from version in D:\Swap\electron-quick-start\package.json +10ms
      electron-packager Inferring target Electron version from electron in D:\Swap\electron-quick-start\package.json +5ms
      electron-packager Application name: electron +1ms
      electron-packager Target Electron version: 1.6.12 +0ms
      electron-packager Ignored path regular expressions: [ '/node_modules/electron($|/)',
      '/node_modules/electron-prebuilt(-compile)?($|/)',
      '/node_modules/electron-packager($|/)',
      '/\\.git($|/)',
      '/node_modules/\\.bin($|/)',
      '\\.o(bj)?$' ] +1ms
      electron-packager Downloading Electron with options {"strictSSL":true,"platform":"win32","arch":"x64","version":"1.6.12"} +3ms
    Packaging app for platform win32 x64 using electron v1.6.12
      electron-packager Creating C:\Users\fzyzcjy\AppData\Local\Temp\electron-packager\win32-x64-template +7s
      electron-packager Extracting C:\Users\fzyzcjy\.electron\electron-v1.6.12-win32-x64.zip to C:\Users\fzyzcjy\AppData\Local\Temp\electron-packager\win32-x64-template +0ms
      electron-packager Initializing app in C:\Users\fzyzcjy\AppData\Local\Temp\electron-packager\win32-x64\electron-win32-x64 from C:\Users\fzyzcjy\AppData\Local\Temp\electron-packager\win32-x64-template template +1s

      ** STUCK HERE **

      electron-packager Ignored paths based on the out param: [ 'D:\\Swap\\electron-quick-start\\electron-darwin-ia32',
      'D:\\Swap\\electron-quick-start\\electron-darwin-x64',
      'D:\\Swap\\electron-quick-start\\electron-darwin-armv7l',
      'D:\\Swap\\electron-quick-start\\electron-linux-ia32',
      'D:\\Swap\\electron-quick-start\\electron-linux-x64',
      'D:\\Swap\\electron-quick-start\\electron-linux-armv7l',
      'D:\\Swap\\electron-quick-start\\electron-mas-ia32',
      'D:\\Swap\\electron-quick-start\\electron-mas-x64',
      'D:\\Swap\\electron-quick-start\\electron-mas-armv7l',
      'D:\\Swap\\electron-quick-start\\electron-win32-ia32',
      'D:\\Swap\\electron-quick-start\\electron-win32-x64',
      'D:\\Swap\\electron-quick-start\\electron-win32-armv7l' ] +0ms

      ** STUCK HERE **

      electron-packager Pruning modules via: npm prune --production +15s

      ** STUCK HERE TOO **

      electron-packager Renaming electron.exe to electron.exe in C:\Users\fzyzcjy\AppData\Local\Temp\electron-packager\win32-x64\electron-win32-x64 +14s
      electron-packager Running rcedit with the options {"version-string":{},"file-version":"1.0.0","product-version":"1.0.0"} +0ms
      electron-packager Moving C:\Users\fzyzcjy\AppData\Local\Temp\electron-packager\win32-x64\electron-win32-x64 to D:\Swap\electron-quick-start\electron-win32-x64 +235ms

      ** STUCK HERE TOO **

    Wrote new app to D:\Swap\electron-quick-start\electron-win32-x64

What command line arguments are you passing? Alternatively, if you are using the API, what
parameters are you passing to the packager() function?

electron-packager ./ --arch=x64 --platform=win32

Please provide either a failing minimal testcase (with a link to the code) or detailed steps to
reproduce your problem. Using electron-quick-start
is a good starting point.

Just run the command above.

[malept]

I would first delete your outputted package directories and determine how many files/folders are in your app directory. Copying time is proportional to the number of files/folders. Pruning time is proportional to the total number of modules in devDependencies+dependencies installed, including sub-dependencies.

[fzyzcjy]

Thank you for your reply!

The dir tree:

• node_modules: ~150MB
• others: ~100KB
  (PS. the outputted dir(we-screen-win32-x64) is ~130MB)

The package.json:

{
  "name": "we-screen",
  "version": "1.0.0",
  "description": "",
  "main": "main.js",
  "scripts": {
    "start": "electron ."
  },
  "devDependencies": {
    "electron": "~1.6.2",
    "gulp": "^3.9.1",
    "rimraf": "^2.6.1"
  },
  "dependencies": {
    "electron-packager": "^8.7.2",
    "hammerjs": "^2.0.8",
    "jquery": "^3.2.1",
    "node-notifier": "^5.1.2"
  }
}

This is a small project and has less than 1K lines of code. I have a quick SSD. I don't think the number of my dependencies is quite big...

I am hoping to hearing from you and much appreciation to your kind help!

Regards
Turtle

[malept]

One quick note while I'm still investigating: electron-packager should not be in dependencies unless you're using it inside your app (as opposed to just packaging your app, like 98% of users).

[malept]

copy the folder from temporary location to the dist location.

That's slow because you're copying between drives. You might want to set your tmpdir to somewhere on your D: drive.

I tested moving electron-packager from dependencies to devDependencies and pruning went from 6s to 2s.

Those two things should change the execution time of Packager noticably.

[fzyzcjy]

@malept

Hi,

That's slow because you're copying between drives.

Thank you! I have copied the working dir to C:\ and it is a bit faster now.

moving electron-packager from dependencies to devDependencies

Sorry, but that doesn't work for me. After removal, it still takes ~1.5min (with 100%CPU, 1~2G RAM)... This is almost the same as before.

due to moving between drives and lots of dependencies

Er, I wonder whether it has really a lot of dependencies? Most of them are just pure js packages like jquery and hammerjs.

malept closed this an hour ago

Sorry, but this issue has not been solved yet... It is still very slow... :(

PS. My computer: CPU i5-7200U, RAM 8G DDR4 2133MHz, with 256G SSD. I think that is not a extremely slow computer? And your compile speed (2s) is just really fast. Is it because you have a super powerful computer? :D

So I wonder how can I make it faster...

Thank you for your quick reply!
Regards,
Turtle

[malept]

Another thing you can try is installing yarn and then using --package-manager=yarn.

[fzyzcjy]

It works like a charm!!! Thanks!!! You are so great!!! Now it only takes 7 seconds!!!

[fzyzcjy]

Here is my code for changing TEMP dir automatically:

var currentDriver = process.cwd().substr(0,1);
var cmd = 'set TEMP='+currentDriver+':\\Temp && set DEBUG=electron-packager && '+
        'electron-packager ./ MiaoZhi --arch=x64 --platform=win32 --package-manager=yarn --overwrite';
exec(cmd);

[malept]

There is an flag in Packager called --tmpdir.

[malept]

Also, if you end up running Electron Packager via JS, you probably want to use the JS API instead.

[fzyzcjy]

Thank you!