node crashes when v8.getHeapSnapshot is called in succession.

Created on 8 Oct 2020  Â·  8Comments  Â·  Source: nodejs/node

  • Version: v14.13.1
  • Platform: linux
  • Subsystem: v8

What steps will reproduce the bug?

// given code
const v8 = require('v8');
v8.getHeapSnapshot()
v8.getHeapSnapshot()

What is the expected behavior?


no crash, the API returns streams

Additional information


I can easily recreate it, so any more info can be collected if needed.

memory v8 module

Most helpful comment

@RaisinTen Not quite… the process for creating a snapshot is:

  1. V8 starts creating the snapshot, adding some strings to the heap profiler’s internal string table
  2. V8 performs a GC, so that the heap snapshot does not contain objects that are not actually relevant to the snapshot
  3. V8 continues creating the snapshot, adding more strings to the internal string table
  4. Node.js wraps the snapshot (C++ object) in a JS object, using a weak reference – i.e. once the JS object is GC’ed, we delete the C++ snapshot object

Now… that last bit is a bit tricky: V8 implements “Delete the snapshot object” in 2 different ways:

  1. If there is only one complete snapshot object, perform “delete all snapshot objects + reset metadata”, where “metadata” includes the internal string table
  2. If there are more complete snapshot objects, delete only the particular snapshot object in question.

Part of the problem is the word “complete” here.

Now, when running the script from the issue here …

  • During creation of the second snapshot, in step 2, V8 performs a GC.
  • Since the first snapshot is weakly held, this deletes it.
  • The delete operation also deletes metadata, because at that point, the first snapshot is the only complete one
  • In step 3, the second snapshot tries to access the internal string table – the one that was deleted just before this
  • The process crashes or hangs :boom:

All 8 comments

Can you post the stack trace here?

@joyeecheung - Thanks for the reply and here is the stack trace of the code.
``` #0 0x0000000000fd8522 in v8::internal::wasm::WasmFullDecoder<(v8::internal::wasm::Decoder::ValidateFlag)1, v8::internal::wasm::EmptyInterface>::DecodeFunctionBody() ()

1 0x0000000000fdb7b5 in v8::internal::wasm::WasmDecoder<(v8::internal::wasm::Decoder::ValidateFlag)1>::OpcodeLength(v8::internal::wasm::Decoder, unsigned char const) [clone .constprop.0] ()

2 0x000000000102d046 in v8::internal::wasm::DecodeLocalNames(v8::internal::Vector) ()

3 0x0000000000fdd1d9 in v8::internal::wasm::WasmCompilationUnit::CompileWasmFunction(v8::internal::Isolate, v8::internal::wasm::NativeModule, v8::internal::wasm::WasmFeatures, v8::internal::wasm::WasmFunction const, v8::internal::wasm::ExecutionTier) ()

4 0x0000000000fde268 in v8::internal::wasm::Decoder::errorf(unsigned int, char const*, ...) [clone .constprop.0] ()

5 0x0000000000fce530 in v8::internal::wasm::WasmFullDecoder<(v8::internal::wasm::Decoder::ValidateFlag)1, v8::internal::wasm::EmptyInterface>::DecodeNumericOpcode(v8::internal::wasm::WasmOpcode) ()

6 0x00000000009b4ea6 in v8::internal::(anonymous namespace)::SetLengthProperty(v8::internal::Isolate*, v8::internal::Handle, double) ()

7 0x0000000000be2f2b in v8::internal::Runtime_HasElementWithInterceptor(int, unsigned long, v8::internal::Isolate) ()

8 0x0000000000be44d6 in v8::internal::IC::UpdatePolymorphicIC(v8::internal::Handle, v8::internal::MaybeObjectHandle const&) ()

9 0x0000000000be4b56 in v8::internal::LoadIC::UpdateCaches(v8::internal::LookupIterator*) ()

10 0x00000000013feab9 in BLAKE2b_Final ()

11 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

12 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

13 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

14 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

15 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

16 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

17 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

18 0x0000000001397d62 in SSL_CTX_use_serverinfo_ex ()

19 0x0000000001395a7a in ssl_set_cert ()

20 0x0000000001395858 in ssl_do_config ()

21 0x0000000000cc0501 in v8::internal::(anonymous namespace)::ElementsAccessorBase, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)20> >::CopyTypedArrayElementsSlice(v8::internal::JSTypedArray, v8::internal::JSTypedArray, unsigned long, unsigned long) ()

22 0x0000000000cc136f in v8::internal::(anonymous namespace)::ElementsAccessorBase, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)19> >::CopyTypedArrayElementsSlice(v8::internal::JSTypedArray, v8::internal::JSTypedArray, unsigned long, unsigned long) ()

23 0x0000000000b89254 in v8::internal::YoungGenerationMarkingTask::MarkObject(v8::internal::Object) ()

24 0x00000000009cd906 in v8::internal::Builtin_Impl_Stats_ConsoleContext(int, unsigned long, v8::internal::Isolate) ()

25 0x00000000009cdb93 in v8::internal::Builtin_Impl_Stats_ConsoleTime(int, unsigned long, v8::internal::Isolate) ()

26 0x00000000009cf1f0 in v8::internal::Builtin_ConsoleTimeStamp(int, unsigned long, v8::internal::Isolate) ()

27 0x000000000096db6a in v8::debug::WasmScript::GetFunctionHash(int) ()

28 0x0000000000000000 in ?? ()

```

Just noticed that I ran into this independently of this issue – https://chromium-review.googlesource.com/c/v8/v8/+/2464823 fixes this.

@addaleax so is this a problem with the v8 GC or node?

@RaisinTen It’s a V8 bug, yes, not caused by the GC implementation but related to GC timing. We could work around it if necessary, but usually the V8 team responds very quickly during working hours (… faster than Node.js for sure :slightly_smiling_face: )

@addaleax :slightly_smiling_face:
I read it and this is what I understood:

  1. When we query the first snapshot object, it gets created and returned
  2. Then the second object is queried and the object creation is not complete yet
  3. Due to the GC timing, the second object gets deleted along with the first one before we actually get to handle the second object
  4. Now we are returned an address which we are not allowed to dereference, so we get a segfault

Did I get it right? I don't think I understand how we can work around this though.

@RaisinTen Not quite… the process for creating a snapshot is:

  1. V8 starts creating the snapshot, adding some strings to the heap profiler’s internal string table
  2. V8 performs a GC, so that the heap snapshot does not contain objects that are not actually relevant to the snapshot
  3. V8 continues creating the snapshot, adding more strings to the internal string table
  4. Node.js wraps the snapshot (C++ object) in a JS object, using a weak reference – i.e. once the JS object is GC’ed, we delete the C++ snapshot object

Now… that last bit is a bit tricky: V8 implements “Delete the snapshot object” in 2 different ways:

  1. If there is only one complete snapshot object, perform “delete all snapshot objects + reset metadata”, where “metadata” includes the internal string table
  2. If there are more complete snapshot objects, delete only the particular snapshot object in question.

Part of the problem is the word “complete” here.

Now, when running the script from the issue here …

  • During creation of the second snapshot, in step 2, V8 performs a GC.
  • Since the first snapshot is weakly held, this deletes it.
  • The delete operation also deletes metadata, because at that point, the first snapshot is the only complete one
  • In step 3, the second snapshot tries to access the internal string table – the one that was deleted just before this
  • The process crashes or hangs :boom:

@addaleax thank you for the awesome explanation! :slightly_smiling_face:

Was this page helpful?
0 / 5 - 0 ratings

Related issues

cong88 picture cong88  Â·  3Comments

jmichae3 picture jmichae3  Â·  3Comments

willnwhite picture willnwhite  Â·  3Comments

fanjunzhi picture fanjunzhi  Â·  3Comments

Brekmister picture Brekmister  Â·  3Comments