Node: Current issue with asan

[gengjiawen]

Current asan failed using the following config

./configure --enable-asan --debug
make -j8
make test-only

due to https://github.com/nodejs/node/blob/31217a8e88d7414579284267f8715112bf8a0fc6/src/node.cc#L950-L958

My question is:

• Is there a way to fix this ?
• If not, can I make the test go on with those failures ?

cc @addaleax @bnoordhuis @devnexen

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f6f24a48f1e in __interceptor_realloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10df1e)
    #1 0x55be1f9ceed2 in char const** node::UncheckedRealloc<char const*>(char const**, unsigned long) ../src/util-inl.h:346
    #2 0x55be1f9cdb32 in char const** node::UncheckedMalloc<char const*>(unsigned long) ../src/util-inl.h:361
    #3 0x55be1f9cb8c3 in char const** node::Malloc<char const*>(unsigned long) ../src/util-inl.h:380
    #4 0x55be1f9c77c9 in node::Init(int*, char const**, int*, char const***) ../src/node.cc:953
    #5 0x55be1f81ff9f in NodeTestFixture::SetUpTestCase() ../test/cctest/node_test_fixture.h:79
    #6 0x55be1f7d0d9d in testing::TestSuite::RunSetUpTestSuite() (/root/node/out/Debug/cctest+0x1f87d9d)
    #7 0x55be1f7f27d5 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::TestSuite, void>(testing::TestSuite*, void (testing::TestSuite::*)(), char const*) (/root/node/out/Debug/cctest+0x1fa97d5)
    #8 0x55be1f7e327f in void testing::internal::HandleExceptionsInMethodIfSupported<testing::TestSuite, void>(testing::TestSuite*, void (testing::TestSuite::*)(), char const*) (/root/node/out/Debug/cctest+0x1f9a27f)
    #9 0x55be1f7973d2 in testing::TestSuite::Run() ../test/cctest/gtest/gtest-all.cc:4289
    #10 0x55be1f7aee33 in testing::internal::UnitTestImpl::RunAllTests() ../test/cctest/gtest/gtest-all.cc:6752
    #11 0x55be1f7f44e1 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/root/node/out/Debug/cctest+0x1fab4e1)
    #12 0x55be1f7e4524 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/root/node/out/Debug/cctest+0x1f9b524)
    #13 0x55be1f7abea9 in testing::UnitTest::Run() ../test/cctest/gtest/gtest-all.cc:6340
    #14 0x55be1f81ac55 in RUN_ALL_TESTS() ../test/cctest/gtest/gtest.h:14896
    #15 0x55be1f81ab3b in main ../test/cctest/gtest/gtest_main.cc:45
    #16 0x7f6f243ef1e2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x271e2)

Direct leak of 7 byte(s) in 1 object(s) allocated from:
    #0 0x7f6f249d13dd in strdup (/lib/x86_64-linux-gnu/libasan.so.5+0x963dd)
    #1 0x55be1f9c798e in node::Init(int*, char const**, int*, char const***) ../src/node.cc:957
    #2 0x55be1f81ff9f in NodeTestFixture::SetUpTestCase() ../test/cctest/node_test_fixture.h:79
    #3 0x55be1f7d0d9d in testing::TestSuite::RunSetUpTestSuite() (/root/node/out/Debug/cctest+0x1f87d9d)
    #4 0x55be1f7f27d5 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::TestSuite, void>(testing::TestSuite*, void (testing::TestSuite::*)(), char const*) (/root/node/out/Debug/cctest+0x1fa97d5)
    #5 0x55be1f7e327f in void testing::internal::HandleExceptionsInMethodIfSupported<testing::TestSuite, void>(testing::TestSuite*, void (testing::TestSuite::*)(), char const*) (/root/node/out/Debug/cctest+0x1f9a27f)
    #6 0x55be1f7973d2 in testing::TestSuite::Run() ../test/cctest/gtest/gtest-all.cc:4289
    #7 0x55be1f7aee33 in testing::internal::UnitTestImpl::RunAllTests() ../test/cctest/gtest/gtest-all.cc:6752
    #8 0x55be1f7f44e1 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/root/node/out/Debug/cctest+0x1fab4e1)
    #9 0x55be1f7e4524 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/root/node/out/Debug/cctest+0x1f9b524)
    #10 0x55be1f7abea9 in testing::UnitTest::Run() ../test/cctest/gtest/gtest-all.cc:6340
    #11 0x55be1f81ac55 in RUN_ALL_TESTS() ../test/cctest/gtest/gtest.h:14896
    #12 0x55be1f81ab3b in main ../test/cctest/gtest/gtest_main.cc:45
    #13 0x7f6f243ef1e2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x271e2)

[devsnek]

That entire function is due to be deprecated/removed at some point, maybe v14? Also in the short term, you can mark the function with __attribute__((no_sanitize_address)).

[gengjiawen]

Is it possible we remove this in master branch ? @addaleax

[addaleax]

I’m assuming this is happening because of the cctest and not the main binary? I’ll think about a good way around this…

[gengjiawen]

I got some progress using ninja build. Looks like there are some leaks on master.

I am using docker image gengjiawen/node-build to test this.

./configure --debug --enable-asan --ninja
ninja -C out/Debug
python3 tools/test.py -J --mode=debug

Full log:
https://gist.github.com/gengjiawen/734b674e277b61c1a8998c6b57bffc75

[gengjiawen]

Also there is a heap-use-after-free when testing parallel/test-fs-opendir.
cc @Fishrock123

[gengjiawen]

Looks like there is some issue with wasi too. cc @nodejs/wasi

Full log:
https://app.circleci.com/jobs/github/gengjiawen/node-play/15/parallel-runs/0/steps/0-102

Another thing is asan on macOS not showing stacktrace, I am not sure why
https://github.com/gengjiawen/node/blob/ci/asan/.github/workflows/CI.yml

./configure --debug --enable-asan && make -j4 && python3 tools/test.py -J --mode=debug

[cjihrig]

@gengjiawen the CircleCI link is a 404 for me.

[gengjiawen]

I can use that link, maybe CircleCI has bug in it's new design website.

Can you visit this: https://circleci.com/gh/gengjiawen/node-play/15?pipelines-ui-opt-out ?

[cjihrig]

Thanks, I can see it now. I'll be sending a fix.

[gengjiawen]

Also there is some issues related to worker cc @nodejs/workers

https://circleci.com/gh/gengjiawen/node-play/15?pipelines-ui-opt-out

[gengjiawen]

I am thinking make this an github action, for our tracing memory issue.

Hopefully this will bring more people involved on this and make Node.js stablility to next level.

[addaleax]

https://github.com/nodejs/node/pull/32406 should fix this (finally :slightly_smiling_face:)