Node: APIs removed in V8 7.0 and native addons

[addaleax]

Currently, CITGM failures are dominated by addons failing because of APIs that were removed in V8 7.0 (Example run).

Most of the removed functions are simple wrappers that take now-required arguments from other sources, for example, the now-removed value->ToString() was essentially just a shorthand for value->ToString(Isolate::GetCurrent()->GetCurrentContext()).FromMaybe(Local<String>()).

Most of these APIs (all widely used ones, at least) do currently not print deprecation warnings when building with Node 10.

So, the question is: What, if anything, do we do about this?

We have the option of maintaining the deprecated APIs ourselves for a while, but I’m not sure how people feel about that.

We can also do ecosystem outreach on our own, but I highly doubt that that is going to restore CITGM results in time for Node 11, and given the large number of addons that are affected by this, we definitely can’t address all cases where this is an issue by ourselves.

[devsnek]

wasn't our big plan always to shim 68 and 69 for 10.x and them push 70 to 11.x?

[targos]

@devsnek this issue is about 11.x

[targos]

/cc @nodejs/v8-update

[bnoordhuis]

Most of these APIs (all widely used ones, at least) do currently not print deprecation warnings when building with Node 10.

What if we float a patch that changes that? Doesn't break ABI but should result in a flurry of bug reports against add-ons that need upgrading.

I highly doubt that that is going to restore CITGM results in time for Node 11

If 2018-10-23 is still the target date then I agree. Floating shims for a release cycle doesn't trouble me.

[addaleax]

The V8 commit in question is v8/v8@5acf20551217ddce89c20c7c8848ae971eb192c2, btw. I don’t know if there are others that are relevant.

Most of these APIs (all widely used ones, at least) do currently not print deprecation warnings when building with Node 10.

What if we float a patch that changes that? Doesn't break ABI but should result in a flurry of bug reports against add-ons that need upgrading.

That seems like a good idea. Looking into it, it’s doable but it’s not trivial because some of the replacement APIs don’t even exist in Node 10.

[refack]

BTW: 7.0 is not stable yet, we could petition for reverting of https://github.com/v8/v8/commit/5acf20551217ddce89c20c7c8848ae971eb192c2.

[addaleax]

@nodejs/v8 ^^^

[hashseed]

I just took a look. It seems the change in question is a mix of removing APIs that are marked V8_DEPRECATED and V8_DEPRECATE_SOON. We are conforming to the policy of marking a deprecated API for at least one version before removing. For the latter, we unfortunately are not.

@danelphick has offered to partially revert this change to address the latter. I.e. instead of removing, we would move them to V8_DEPRECATED. We will still remove APIs that were already marked V8_DEPRECATED though.

However, the underlying issue here is that V8 version bumps happen a lot more often than Node.js. If V8 was to accommodate to Node.js' deprecation policy, we would need a whole year to remove an API. We did that previously with the legacy debugging protocol, but I don't think this is something we can do in general.

I think we need an actual discussion on this underlying issue, going forward.

The motivation behind this particular change is to share objects between isolates in order to save memory. That means that where we derived the isolate from the object via GetIsolate(), we no longer can do that and require the embedder to pass the isolate explicitly.

[addaleax]

@hashseed Some of the APIs just used Isolate::GetCurrent() under the hood … is that a valid option?

The motivation behind this particular change is to share objects between isolates in order to save memory.

That sounds exciting :)

[hashseed]

I think in this particular change it's possible to shim the removed API with a floating patch.

However, this may not solve similar issues in the future.

[hashseed]

Would it be possible to restate Node.js' deprecation policy towards native modules to exclude V8's API? Rationale:

• It's easier for us :)
• Node.js doesn't own V8's API.
• NAPI is a viable alternative.
• As module author, you could track node's master branch to detect API issues early.

[targos]

We have some text in the collaborator guide: https://github.com/nodejs/node/blob/master/COLLABORATOR_GUIDE.md#breaking-changes-and-deprecations

Note that errors thrown, along with behaviors and APIs implemented by dependencies of Node.js (e.g. those originating from V8) are generally not under the control of Node.js and therefore are not directly subject to this policy. However, care should still be taken when landing updates to dependencies when it is known or expected that breaking changes to error handling may have been made. Additional CI testing may be required.

[addaleax]

Would it be possible to restate Node.js' deprecation policy towards native modules to exclude V8's API?

I don’t think that’s realistic for frequently-used APIs like some of the ones being removed here… a lot of legacy native addons do not receive much maintenance, and so having a deprecation warning for at least a full Node.js cycle would be ideal…

[devsnek]

what if we just float a patch for keeping 11.x unbroken and change our policy and break in 12.x

[targos]

This is the diff of deps/v8/include between v10.x-staging and master:

https://gist.github.com/targos/efd41838f402e78c4caa96d4d18168de

[danelphick]

As Yang stated above, we're going to put back all the V8_DEPRECATE_SOON methods that were deleted in that change (but instead mark them V8_DEPRECATED). As such I've uploaded https://chromium-review.googlesource.com/c/v8/v8/+/1251422.

[refack]

However, the underlying issue here is that V8 version bumps happen a lot more often than Node.js. If V8 was to accommodate to Node.js' deprecation policy, we would need a whole year to remove an API. We did that previously with the legacy debugging protocol, but I don't think this is something we can do in general.

I brought up the request to revert, just as a way to buy a little time (that will allow for a smoother release of node11 with V8 7.0).
IMHO all the points that were raised WRT to V8's deprecation policy are understood, and should be taken into consideration by Node.js.

I'd would like to compliment @targos for the initiative to land 7.0 in master sooner rather than when it went stable (and the TSC for approving).
I believe that should be the policy going forward, so we can face such issue sooner.

[joaocgreis]

Most (if not all) of the failures I see in CitGM include NAN. Shouldn't https://github.com/nodejs/nan/ be updated instead of changing anything here?

[refack]

This is the list of failed compilation I got from the above CITGM run on ubuntu16

• [ ] level-v4.0.0
• [ ] serialport-v7.0.2
• [ ] bcrypt-v3.0.1
• [ ] ref-v1.3.5
• [ ] zeromq-v4.6.0
• [ ] time-v0.12.0
• [ ] node-sass-v4.9.3
• [ ] iconv-v2.3.0
• [ ] microtime-v2.1.8
• [ ] leveldown-v4.0.1
• [ ] sqlite3-v4.0.2
• [ ] node-report-v2.2.1
• [ ] node-gyp-v3.8.0

It also seems to me that most (if not all) fail via nodejs/nan

[digitalinfinity]

With respect to N-API being an alternative for the modules, we're tracking evaluating the impacted modules in https://github.com/nodejs/abi-stable-node/issues/346 - the intention is to evaluate if these are good candidates to move to N-API, and if so, engage with module authors on starting a port. However, N-API adoption is slow so we'll probably have to deal with issues like this for the near term.

[refack]

First step https://github.com/nodejs/nan/pull/808
Now we need review, land, publish, then push the above list to update and publish.

[addaleax]

I’ve opened PRs at #23158 and #23159 to bring the V8 API for Node 10 and master closer together, would be cool if people could take a look at that.

[refack]

I probably should have posted this here:

Sample output from Windows (screenshots for color effect ;) [email protected]
nan test with #23159:

nan test with just 'V8_IMMINENT_DEPRECATION_WARNINGS=1'

Should we turn it on in common.gypi? For node10? or just for `master?

[hashseed]

Enabling warnings for V8_DEPRECATE_SOON would make sense imo.

[hashseed]

@danelphick landed the previously discussed partial revert to V8 7.0 here.

[targos]

@addaleax I have a commit that updates V8 here if you want to include it in your PR against master: https://github.com/targos/node/commit/f68cf0b216b65b12fa8702ec9a99f41a053a71a2

[addaleax]

@targos @hashseed Sure, I can do that, but … is it the better choice to add these methods back using internals that V8 obviously wants to get rid of? With the current state of my PR, these methods could theoretically stay as long as Isolate::GetCurrent() exists, without V8 having to worry about supporting them

[hashseed]

I would expect that calling Isolate::GetCurrent could be more expensive. Otherwise sgtm.

[addaleax]

@hashseed I’m okay with that, esp. if the methods are marked deprecated to begin with.

[targos]

How do we proceed with the V8 update then? I suppose it will conflict with https://github.com/nodejs/node/pull/23158?

[addaleax]

@targos I guess I could include your patch + a revert that we can then continue to cherry-pick (and/or merge it into my commit)?

[addaleax]

Should we consider this resolved, or do people feel that we need to have a larger discussion around our management of V8 API deprecations/changes somewhere?

[refack]

IHMO if we turn on V8_IMMINENT_DEPRECATION_WARNINGS for addons (https://github.com/nodejs/node/pull/23414#issuecomment-428763624, https://github.com/nodejs/node/issues/23167), the bigger issue will will be mitigated.

[addaleax]

@refack I guess you can feel free to open a PR with that?

[jasnell]

@addaleax ... what is the status on this?

[addaleax]

@jasnell I think we’re good as far as the situation around v10.x/v11.x is concerned… the rest here is more about future-proofing

[jasnell]

Ok, taking it off the 11.0.0 milestone then

[Trott]

Should this remain open? If so, is there anything specific that is actionable at this time?

[refack]

We added https://github.com/nodejs/node/pull/23426 which adds warnings for APIs that are V8_IMMINENT_DEPRECATION. This should give addon writers earlier notice about depredations.
I think that was the low hanging fruit.

[BridgeAR]

There was no activity for over a year and we are now at V8 8.0. I guess this is resolved and I am closing this. Please reopen in case there is more left to do.