Node: Building nightly node 12 on alpine is broken since 2018-12-06 (alpine linux)

Created on 17 Dec 2018 · 12Comments · Source: nodejs/node

I've been building node12 nightly to create alpine linux docker images since December 1st using the source packages available at the nodejs website.

Ever since release nightly20181207cbf0e5a1f4, the build fails with the same error:

  g++ -o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/v8_libbase/deps/v8/src/base/platform/platform-posix-time.o ../deps/v8/src/base/platform/platform-posix-time.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=0' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_EMBEDDER_STRING="-node.5"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_INTL_SUPPORT' '-DV8_CONCURRENT_MARKING' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_EMBEDDED_BUILTINS' '-DV8_EMBEDDED_BYTECODE_HANDLERS' -I../deps/v8  -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-strict-aliasing -m64 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/.deps//build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/v8_libbase/deps/v8/src/base/platform/platform-posix-time.o.d.raw   -c
  LD_LIBRARY_PATH=/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/lib.host:/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps/v8/gypfiles; mkdir -p /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen; python ../tools/gen-postmortem-metadata.py "/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/debug-support.cc" ../src/objects.h ../src/objects-inl.h ../src/objects/allocation-site-inl.h ../src/objects/allocation-site.h ../src/objects/code-inl.h ../src/objects/code.h ../src/objects/data-handler.h ../src/objects/data-handler-inl.h ../src/objects/fixed-array-inl.h ../src/objects/fixed-array.h ../src/objects/js-array-inl.h ../src/objects/js-array.h ../src/objects/js-array-buffer-inl.h ../src/objects/js-array-buffer.h ../src/objects/js-objects-inl.h ../src/objects/js-objects.h ../src/objects/js-regexp-inl.h ../src/objects/js-regexp.h ../src/objects/js-regexp-string-iterator-inl.h ../src/objects/js-regexp-string-iterator.h ../src/objects/map.h ../src/objects/map-inl.h ../src/objects/name.h ../src/objects/name-inl.h ../src/objects/scope-info.h ../src/objects/script.h ../src/objects/script-inl.h ../src/objects/shared-function-info.h ../src/objects/shared-function-info-inl.h ../src/objects/string.h ../src/objects/string-inl.h
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_implementation.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_uconfig.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_uconfig_target.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/tools/icu/icu_implementation.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/tools/icu/icu_uconfig.stamp
make[1]: *** No rule to make target '../deps/v8/test/torque/test-torque.tq', needed by '2f2d73a3ab02a00709f3dbbaf3638568d55301e2.intermediate'.  Stop.

I'm building the code using the base alpine linux docker image for version 3.8.1, with no custom packages (only official ones required to build the image).

Here's how I'm building from the source:

    ./configure --prefix=/usr --fully-static --without-npm
    make -j$(getconf _NPROCESSORS_ONLN)

Every nightly on december before that is build properly.

You can check lsmoura/node:12nightly if you want to see other nightly builds working.

Thanks for all the work from the nodejs developers!

V8 Engine build

Source

lsmoura

Most helpful comment

I'm preparing a PR to fix this

targos on 17 Dec 2018

❤2

All 12 comments

I think deps/v8/test/torque/test-torque.tq came with the V8 7.1 update.

deps/v8/test is excluded from the source tarball:
https://github.com/nodejs/node/blob/a6f69ebc05f4033b012b523661f6c3f62f3469b1/Makefile#L984

cc @nodejs/build-files @nodejs/v8-update

richardlau on 17 Dec 2018

Should I be taking extra steps to prepare the source to build, then? maybe removing the requirement to build deps/v8/test/torque/test-torque.tq from the makefile?

Or maybe the nightly source file is not being built properly?

lsmoura on 17 Dec 2018

Since it's in test, does the gyp file need to reference it? Maybe try removing
https://github.com/nodejs/node/blob/a6f69ebc05f4033b012b523661f6c3f62f3469b1/deps/v8/gypfiles/v8.gyp#L27

richardlau on 17 Dec 2018

@lsmoura It's a gap in our process -- The nightly builds are built from a git checkout, not the source tarball. I'm not aware that we have any builds that build from the source tarball -- cc @nodejs/build ?

richardlau on 17 Dec 2018

well, 50% of my motivation to make these builds are to help the nodejs team with something. Even if it is "I've found these errors and I don't know if you're aware of them". Mission accomplished, somehow (=

I'll automate this process soon enough and make logfiles available...

Thanks for the tips. I'll give it a try.

lsmoura on 17 Dec 2018

👍1

I'm preparing a PR to fix this

targos on 17 Dec 2018

❤2

This fixed temporarily on my build script, after unpacking the file:

  cp "deps/v8/gypfiles/v8.gyp" "deps/v8/gypfiles/v8.gyp.tmp"
  grep -v test-torque deps/v8/gypfiles/v8.gyp.tmp > deps/v8/gypfiles/v8.gyp

lsmoura on 17 Dec 2018

By the end of the build process, I got this one other missing piece:

  g++ -o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/torque-generated/builtins-test-from-dsl-gen.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=0' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_EMBEDDER_STRING="-node.5"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_INTL_SUPPORT' '-DV8_CONCURRENT_MARKING' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_EMBEDDED_BUILTINS' '-DV8_EMBEDDED_BYTECODE_HANDLERS' '-DUCONFIG_NO_SERVICE=1' '-DU_ENABLE_DYLOAD=0' '-DU_STATIC_IMPLEMENTATION=1' '-DU_HAVE_STD_STRING=1' '-DUCONFIG_NO_BREAK_ITERATION=0' -I../deps/v8 -I../deps/v8/include -I/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen -I../deps/icu-small/source/i18n -I../deps/icu-small/source/common  -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-strict-aliasing -m64 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/.deps//build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o.d.raw -Os -fomit-frame-pointer -Os -fomit-frame-pointer -c
g++: error: /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/torque-generated/builtins-test-from-dsl-gen.cc: No such file or directory
g++: fatal error: no input files
compilation terminated.
make[1]: *** [deps/v8/gypfiles/v8_initializers.target.mk:196: /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o] Error 1

lsmoura on 17 Dec 2018

@lsmoura see https://github.com/nodejs/node/pull/25097/files#diff-011f2f230002e291fd17e99c2012fcab

targos on 17 Dec 2018

👍1

So, what might a test worker for src tarballs look like? We can probably only afford to do it on a single platform, and that'll likely be one of the Ubuntu 16.04 or 18.04 Docker containers we have in the mix for these kinds of miscellaneous things.

make tar
tar zxvf <tarball> -C ${NODE_TEST_DIR}
cd ${NODE_TEST_DIR}/<expected path>
./configure <with normal test options>
make <with normal test options>
.... ?

Would we copy in the tests and run a full test suite on the compiled build? Is that necessary or just confirming that it compiles is enough?

rvagg on 18 Dec 2018

So, what might a test worker for src tarballs look like? We can probably only afford to do it on a single platform, and that'll likely be one of the Ubuntu 16.04 or 18.04 Docker containers we have in the mix for these kinds of miscellaneous things.
make tar
tar zxvf <tarball> -C ${NODE_TEST_DIR}
cd ${NODE_TEST_DIR}/<expected path>
./configure <with normal test options>
make <with normal test options>
.... ?
Would we copy in the tests and run a full test suite on the compiled build? Is that necessary or just confirming that it compiles is enough?

We shouldn't need to copy in any tests (Node.js tests are included in the source tarball). The things that don't get into the source tarball are tests for deps (which we don't run on the regular CI anyway) and things from tools which I think prevents linting.

richardlau on 18 Dec 2018

I've just encountered this with the v12.11.0 tarball. Almost identical error:

INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
INFO: configure completed successfully
make -C out BUILDTYPE=Release V=0
  touch /node-v12.11.0/out/Release/obj.target/tools/v8_gypfiles/v8_version.stamp
make[1]: *** No rule to make target '../deps/v8/test/torque/test-torque.tq', needed by 'df39af2ee1844610cfa693dbfb109555906de7ec.intermediate'.  Stop.
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:101: node] Error 2

Regression maybe?