Deno: Abnormal termination with ./tools/build.py

[channg]

[39/898] ACTION //:bundle(//build/toolchain/linux:clang_x64)
FAILED: gen/bundle/main.js gen/bundle/main.js.map
python ../../deno/tools/run_node.py ./node_modules/rollup/bin/rollup -c ../../deno/rollup.config.js -i ../../deno/js/main.ts -o gen/bundle/main.js --silent --environment BASEPATH:../../deno
[40/898] CXX obj/third_party/v8/torque/TorqueParser.o
ninja: build stopped: subcommand failed.

I hope I can get a little help.

[NewLunarFire]

Please make sure that
a) You have insalled Rust and Ccache and have rustc and ccache on your path
b) You are up to date with this repo and the third_party submodules

And please provide the full stack trace. There seems to be some errors earlier in the build process. Also, what OS are you on?

Most of the errors I have encoutered so far with deno come from an improperly configured environment. Once everything is configured properly you should no longer run into build issues.

[channg]

I make sure that rustc and ccache are in my path

*CentOS Linux release 7.5.1804 (Core) *

The stack information is below

Thank you very much

[138/977] CXX obj/third_party/v8/v8_libplatform/default-platform.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[141/977] CXX obj/third_party/v8/v8_libplatform/default-foreground-task-runner.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
../../third_party/v8/src/libplatform/default-foreground-task-runner.cc:53:465: warning: code will never be executed [-Wunreachable-code]
  do { if (std::string* _msg = ::v8::base::CheckEQImpl< typename ::v8::base::pass_value_or_ref<decltype(IdleTaskSupport::kEnabled)>::type, typename ::v8::base::pass_value_or_ref<decltype(idle_task_support_)>::type>( (IdleTaskSupport::kEnabled), (idle_task_support_), "IdleTaskSupport::kEnabled" " " "==" " " "idle_task_support_")) { V8_Fatal("../../third_party/v8/src/libplatform/default-foreground-task-runner.cc", 53, "Check failed: %s.", _msg->c_str()); delete _msg; } } while (0);
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                ^~~~
1 warning generated.
[142/977] CXX obj/third_party/v8/v8_libplatform/task-queue.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[143/977] CXX obj/third_party/v8/v8_libplatform/default-worker-threads-task-runner.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[144/977] CXX obj/third_party/v8/v8_libplatform/trace-config.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[145/977] CXX obj/third_party/v8/v8_libplatform/trace-object.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[146/977] CXX obj/third_party/v8/v8_libplatform/trace-writer.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[147/977] CXX obj/third_party/v8/v8_libplatform/worker-thread.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[148/977] CXX obj/third_party/v8/v8_libplatform/tracing-controller.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[152/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/LexerAction.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[153/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/LexerATNSimulator.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[168/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/OrderedATNConfigSet.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[169/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/PredictionMode.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[170/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/ParserATNSimulator.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[171/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/PlusBlockStartState.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[172/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/PlusLoopbackState.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[173/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/PrecedencePredicateTransition.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[174/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/PredicateTransition.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[175/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/PredicateEvalInfo.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
[176/977] ACTION //:bundle(//build/toolchain/linux:clang_x64)
FAILED: gen/bundle/main.js gen/bundle/main.js.map 
python ../../tools/run_node.py ./node_modules/rollup/bin/rollup -c ../../rollup.config.js -i ../../js/main.ts -o gen/bundle/main.js --silent --environment BASEPATH:../..
[177/977] CXX obj/third_party/v8/third_party/antlr4/antlr4/PredictionContext.o
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++/trunk/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-isystem ../../buildtools/third_party/libc++abi/trunk/include' [-Wunused-command-line-argument]
ninja: build stopped: subcommand failed.

[ry]

@channg It seems to be failing during rollup. Can you try executing this command manually

cd out/debug
./node_modules/rollup/bin/rollup -c ../../rollup.config.js -i ../../js/main.ts -o gen/bundle/main.js --silent --environment BASEPATH:../..

And also try removing the "--silent"

[channg]

@ry Thank you very much, but it didn't make my build successful. The stack information is the same as before, and maybe I need to provide more information.

[ry]

@channg Please paste the output of the commands above (and remove the --silent option)

[channg]

[root@localhost debug]# ./node_modules/rollup/bin/rollup -c ../../rollup.config.js -i ../../js/main.ts -o gen/bundle/main.js --silent --environment BASEPATH:../..
Killed
[root@localhost debug]# ./node_modules/rollup/bin/rollup -c ../../rollup.config.js -i ../../js/main.ts -o gen/bundle/main.js --environment BASEPATH:../.. 

../../js/main.ts → gen/bundle/main.js...
Killed

[ry]

Is it segfaulting? You'll have to debug it more... I can't say from this.

[kt3k]

@channg I think it's killed by OOM killer. When I build deno in Linux VM, I needed to allocate 4GB memory for it. (It failed with 2GB memory.)

[ry]

@channg Try building with -j1, like this ./tools/build.py -j1 ... it will take longer but use less memory.

[channg]

It should be that memory is too small.But will Travis ci allocate 4GB of memory?

[kt3k]

travis seems allocating at least 4GB memory (4 GB for container build, 7.5 GB for VM build) https://docs.travis-ci.com/user/reference/overview/

[kitsonk]

I took a look at the memory usage of rollup, and there isn't anything obvious. The biggest factor is that the whole application is built around Promises and those seem to, as you might expect, GC difficult, because the whole rollup is one big long chain.

I modified the rollup.config.js to output a bit more info, this is the current master:

$ node ./node_modules/rollup/bin/rollup -c ../../rollup.config.js -i ../../js/main.ts -o gen/bundle/main.js --environment BASEPATH:../..

../../js/main.ts → gen/bundle/main.js...
Bundle information:
  Output size: 7.41Mb
  Input size: 7.42Mb
  Reduction: 0.21%
  Module count: 86

Memory usage:
  rss: 1071.32 MB
  heapTotal: 1030.23 MB
  heapUsed: 979.82 MB
  external: 0.03 MB

I also had it output a heapdump during the analysis. Here is the interesting bits: