Node: Pointer compression and Node modules

[hashseed]

V8 is fairly close to reaching a state where we can enable pointer compression. It will be a build time flag for the embedder to enable. Implications are:

• Significant memory savings. Some experiments show 30% smaller V8 heap size, in Chrome.
• Slightly worse performance, at least for now. We are confident that it could actually perform better.
• Upper limit of 4GB memory, at least for now.
• Different ABI, so Node modules for non-pointer-compressed builds are incompatible with pointer-compressed builds.

Discuss!

[addaleax]

Upper limit of 4GB memory, at least for now.

So, if we enable this at build time, there’s no way for people to exceed that limit? I would guess that some currently set a larger --max-old-space-size than 4GB, and some people actually make use of that.

Different ABI, so Node modules for non-pointer-compressed builds are incompatible with pointer-compressed builds.

Are both modes going to stay around for a while, or is it more of a thing where V8 would say e.g. if pointer compression works out well then the non-pointer-compressed mode is going to go away?

[mmarchini]

Are there any docs explaining what pointer compression is/how it works?

[addaleax]

Upper limit of 4GB memory, at least for now.

Also: Per Isolate or per process?

[hashseed]

I'm not sure whether there is a public design doc. The principle is that as long as you limit your address space to 2^32 = 4GB, 32 bits are enough to address that whole address space, if you have a fixed base address to compute the full 64 bits.

That address space would be per isolate.

There are some tricks we could employ, to increase the address space, e.g. by aligning to 4 bytes (which we already do) and shifting by 2 bits, effectively addressing 16GB, or aligning to 8 bytes
(at the cost of wasting some bytes for padding) and shifting by 3 bits, to address 32GB. None of that's implemented though.

I would expect for the foreseeable future to have both build modes.

[jasnell]

I would expect for the foreseeable future to have both build modes.

What would be the deciding factor for moving completely to the new mode?Would there be any risk that V8 would switch before we have validation that the new mode works for all necessary Node.js use cases?

[hashseed]

I'd veto a V8 change that would break any reasonable Node.js use case.

[refack]

Does this significantly benefit Node.js environments, i.e. should we consider turning it on when ready?

Ref: 736GB RAM server with 3D-XPoint

[gireeshpunathil]

IMO, pros:

• memory saving for apps (e.g: micro-services) that desire for small footprint
  cons:
• incompatibility with native addons?
• reduced use cases (due to build time limitation) - no apps can deterministic-ally predict memory requirements upfront?
• all the native diagnostic tools need redesign?
• cost of pointer compression and decompression is accounted to the app?

[hashseed]

All very valid points. Although regarding performance: our hope is that pointer compression will pay for itself due to fewer cache misses.

[gsbelarus]

We use --max-old-space-size with higher values than 16 Gb and are planning to use more than 32Gb in the near future. So, any plans to limit memory for nodejs server are kind of alarming and disturbing for us.

[jasnell]

I can definitely see definite benefit to having this option but not on by default, or even on in our distribution builds.

[sam-github]

Agree with @jasnell and others, turning on by default on major platforms sounds like a regression, at least until issues with performance and supporting increasing the heap size are dealt with.

It might make sense for some new platforms, like iOS, where memory is not expected to be plentiful.

[jorangreef]

We use --max-old-space-size for a 32GB heap.

[mhdawson]

I also agree that the limit of 4G means it is probably not viable for Node.js in the short term. Being able to build wi the option to experiment and compare does seem useful though. We might even want a CI job to regularly build/test with it.

[mmarchini]

I can see some embedded devices leveraging this feature, as well as servers if the cache hit ratio prediction holds up.

@hashseed is there any chance this could become a runtime flag in the future (like --jitless)?

[hashseed]

@hashseed is there any chance this could become a runtime flag in the future (like --jitless)?

Not sure whether that's technically viable. Certainly being considered though.

[yorkie]

incompatibility with native addons?

Compressed pointers should have no effect between those add-ons in N-API, the N-API runtime should take the difference itself.

[codebytere]

Note as i don't see it in this issue yet; the design doc can be found here

[jasnell]

@mcollina @nodejs/v8 ... where are we at on this?

[gengjiawen]

It's not enabled by default. The unofficial build can be found here: https://github.com/nodejs/unofficial-builds/tree/master/recipes/x64-pointer-compression.

[mmarchini]

There's an issue on nodejs/tsc with more comprehensive information and discussion on pointer compression, I believe this issue can be closed.