Pkg: Possibility to reduce packages size by 4x

[pi0]

Hi! Just done a quick test on windows builds. Bundle size can be reduced from 24,964 KB to 6,646 KB (3.75x smaller) with doing simple xz compression. This may even help more when having more npm dependencies. Adding this option may be super useful for small and embedded CLI/Webservers :) I've just opened this discussion before going further to know others idea on this.

UPDATE: To be clear, this topic and what i am talking about is SFX packages not js minification & optimizing.

[pascalandy]

@pi0 Do you see any potential impact your NodeApp's performance?

I'm considering this for my Ghost installation. Cheers!

[roccomuso]

@pi0 How did you do that compression? could you provide an example?

[jgoux]

Hi,

as we're talking about bundle size, does pkg apply some kind of optimization / tree shaking / minification internally?

If I apply something like prepack + uglify before packaging, will my final packaged app be lighter?

[anmonteiro]

running strip on the generated binary should also help on UNIX platforms

[igorklopov]

The idea behind pkg is to introduce minimal changes over original nodejs. Btw, if you run strip you can lose the ability to use native addons (.node files).

[pi0]

Hi. Sorry answering late :)

@pascalandy About performance impact: I think if we do extraction on startup, it may only affect startup time and not run-time itself.

@roccomuso My first test was just using tar! But it would be easy adding self-extract bits on binary's head.(exe inside exe)

@anmonteiro About js minification, pkg is absolutely not webpack! And i don't think this would help a lot, main size suffer is from Node.js binary itself. And we could just make a blind compression on the whole things. (Also @igorklopov is correct about .node files)

[pi0]

Some interesting projects we could get idea/ use:

• https://github.com/megastep/makeself
• http://www.peazip.org/self-extracting-archives.html

[lenovouser]

@anmonteiro strip doesn't really work. It reduces the binary size but the app doesn't run anymore. You get the interactive node console 😆

That is very weird. Why is this happening?

[igorklopov]

strip strips payload that contains your project packed. Without that payload the binary behaves like original node.js binary.

[anmonteiro]

Oh hrm, sorry for the bad suggestion. I use strip with Nexe and it works, apparently it doesn't with pkg's approach. Sorry for the noise

[astefanutti]

Referencing #121 for UPX issue (https://github.com/upx/upx).

[roccomuso]

Btw I wrote a node.js wrapper for UPX, https://github.com/roccomuso/upx

[beeing]

What about the idea of using libsquash similar to https://github.com/pmq20/node-packer ?

upx may cause false positives on some antivirus detection

[Clement-TS]

It seems that the nexe owner gets a clue on this: https://github.com/nexe/nexe/issues/366#issuecomment-333534629

He uses UPX over node.js itself (~/.nexe/linux-x64-8.9.4), and I can say it works.
However this do not work on the pkg equivalent: ~/.pkg-cache/v2.5/fetched-v8.9.0-linux-x64:

$ pkg -t node8-x64 app.js -o dist/app
> [email protected]
assert.js:42
  throw new errors.AssertionError({
  ^

AssertionError [ERR_ASSERTION]: Placeholder for not found
    at injectPlaceholder (~/project/node_modules/pkg/lib-es5/producer.js:217:38)
    at injectPlaceholders (~/project/node_modules/pkg/lib-es5/producer.js:236:3)
    at ~/project/node_modules/pkg/lib-es5/producer.js:158:9
    at FSReqWrap.oncomplete (fs.js:135:15)

Any idea?

[roccomuso]

@Clement-TS I got the same error trying to compress the node binary before using pkg. Let's wait @igorklopov

[marcotuna]

Tried today with upx 3.95 on Linux and the problem still persists.
upx -oNodeJSProjectOutput --overlay=copy --strip-relocs=0 --compress-exports=0 --compress-icons=0 --compress-resources=0 --no-align NodeJSProject

[ofarukcaki]

My windows .exe build size reduced 7.xx MB from 21.xx MB with upx, thanks for mentioning it 🌟

[pascalandy]

EDIT: Here is an example of how I use upx to compress Nodejs in my Dockerfile for Ghost: https://github.com/firepress-org/ghostfire/blob/3.0.3-alpine/Dockerfile#L49

[Zireael]

My windows .exe build size reduced 7.xx MB from 21.xx MB with upx, thanks for mentioning it 🌟

Can you say what parameters you used to pack Windows .exe? I tried (UPX 3.95 Win10x64)

upx -o"V:\packed.exe" --overlay=copy --strip-relocs=0 --compress-exports=0 --compress-icons=0 --compress-resources=0 --no-align "V:\app.exe"

but the executable is failing to start.
Thanks in advance!

[ofarukcaki]

My windows .exe build size reduced 7.xx MB from 21.xx MB with upx, thanks for mentioning it star2

Can you say what parameters you used to pack Windows .exe? I tried (UPX 3.95 Win10x64)

upx -o"V:\packed.exe" --overlay=copy --strip-relocs=0 --compress-exports=0 --compress-icons=0 --compress-resources=0 --no-align "V:\app.exe"

but the executable is failing to start.
Thanks in advance!

I didn't use any special flags. Just dragged my .exe and dropped over upx. Thats what I did

[Zireael]

I didn't use any special flags. Just dragged my .exe and dropped over upx. Thats what I did

Thanks, but just Drag&Drop didn't work for me. I'll try packaging with Nexe later, I've read somewhere that they were natively optimised for upx (and support customising .exe file icon out of the box)

[MarcMagnin]

Hey, any update on the matter? UPX + pkg doesn't seems to work so far and that would be really neat if it was.

[Wallacy]

FWIW:

Just tested here using gzexe (Linux) and works good. From 134Mb to 43MB, and i good only 1s performance penalty on load (did not test other runtime operations).