Node: Failed to compile from source for armv7l (busybox) on

[mrvini]

• Version: 6.2.1
• Platform: busybox (Linux dm3730_zoom_torpedo 3.0.0-ts-armv7l #23 Thu Feb 18 13:29:10 PST 2016 armv7l GNU/Linux)
• Subsystem:

I am trying to compile node version 6.2.1 for armv7l (tried using out of the box one before, it would not run for me at all). While I was successfully able to compile all versions prior 6.2.0 (i.e. from 4.x -6.1), 6.2.0 and 6.2.1 start giving me a problem with icu and throws the following error

Options: {'verbose': 1, 'filterfile': 'icu_small.json', 'toolpath': '/var/jenkins_home/jobs/Node/workspace/out/Release', 'deltmpdir': 1, 'outfile': 'icudt57l.dat', 'datfile': '../../deps/icu-small/source/data/in/icudt57l.dat', 'locales': 'en,root', 'endian': 'little', 'tmpdir': '/var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icutmp'}
icu_small.json: icutrim.py config: Trim down ICU to just a certain locale set, needed for node.js use.

FAILED: /var/jenkins_home/jobs/Node/workspace/out/Release/icupkg -tl ../../deps/icu-small/source/data/in/icudt57l.dat 

/var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icutmp/icudt57l.dat
tools/icu/icudata.target.mk:13: recipe for target '/var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icutmp/icudt57l.dat' failed
make[1]: *** [/var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icutmp/icudt57l.dat] Error 1

The script that i am using to build it is the following:

#!/bin/sh
export AR=~/timesys/bin/armv7l-timesys-linux-gnueabi-ar
export CC=~/timesys/bin/armv7l-timesys-linux-gnueabi-gcc
export CXX=~/timesys/toolchain/bin/armv7l-timesys-linux-gnueabi-g++
export LINK=~/timesys/bin/armv7l-timesys-linux-gnueabi-g++

export TARGET_ARCH="-march=armv7l"
export TARGET_TUNE="-mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp -mthumb-interwork -mno-thumb"

#DEFINE FLAGS
export CXX_TARGET_ARCH="-march-armv7l"
export V8_TARGET_ARCH="-march-armv7l"

make clean
./configure with --without-snapshot;
export DESTDIR="~/dist/node-6.2.1"
make -j4 install

Am I missing some kind of dependencies? Please advise if

[bnoordhuis]

I'm not saying it's the cause, but if you want to build with -mfloat-abi=softfp, you should at least pass --with-arm-float-abi=softfp to configure. If you can't get icu to build, see if --without-intl works.

Just to be clear, you're compiling on the target machine itself, right? You're not cross-compiling?

/cc @srl295

[mscdex]

I would assume they are cross-compiling if they're setting those environment variables.

[mrvini]

We are cross-compiling, @mscdex is correct. I will try to set the flag variable and will report on the results shortly. Thanks

[srl295]

So the /var/jenkins_home/jobs/Node/workspace/out/Release/icupkg file should be run on the _host_ architecture. Is it possible to verify the architecture of that file?
_edit_ From icu-generic.gyp:

      'target_name': 'icupkg',
      'toolsets': [ 'host' ],

so, icupkg ought to be built for the host side.

Actually, I can skip the icutrim step for the small ICU builds and save us some build time, but I am not sure if it will necessarily fix the issue here.

If someone can get instructions for setting up the cross-compilation environment I could try to debug this some also.

[mrvini]

@bnoordhuis, I've tried to compile with the correct flags as you've specified and results were successful.

Host System:Linux 38f5fc85140e 3.13.0-61-generic #100-Ubuntu SMP

configure line

./configure --without-snapshot --dest-cpu=arm --dest-os=linux --with-arm-float-abi=softfp --without-intl  

To set it for cross compiling, I believe all you need to do is to point env. variables like in the issue that I've posted above

thank you

[srl295]

@mrvini ok - what packages do i need?
edit and can you also try compiling with --with-intl=full-icu --download=all ? This will download the full ICU and build it in a slightly different code path. while i'm debugging this it could add a data point.

[mrvini]

@srl295, we are using timesys distribution, and the entire distribution is 530MB, I can attached what's in bin folder if that's sufficient (its only 37MB). We are using timesys embedded linux flavor on the top of busybox. Please let me know

[srl295]

@mrvini I mean - what do I need to compile on the host?

[mrvini]

@srl295 , I've added flags that you've recommended

./configure --without-snapshot --dest-cpu=arm --dest-os=linux --with-arm-float-abi=softfp --with-intl=full-icu --download=all;

now I see slightly different error

../../deps/icu/source/data/in/icudt57l.dat "/var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icudt57l.dat"
/var/jenkins_home/jobs/Node/workspace/out/Release/icupkg: 1: /var/jenkins_home/jobs/Node/workspace/out/Release/icupkg: Syntax error: word unexpected (expecting ")")
tools/icu/icudata.target.mk:13: recipe for target '/var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icudt57l.dat' failed

when I specify small-icu, the error changes

'../../deps/icu-small/source/data/in/icudt57l.dat', 'locales': 'en,root', 'endian': 'little', 'tmpdir': '/var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icutmp'}
icu_small.json: icutrim.py config: Trim down ICU to just a certain locale set, needed for node.js use.
FAILED: /var/jenkins_home/jobs/Node/workspace/out/Release/icupkg -tl ../../deps/icu-small/source/data/in/icudt57l.dat /var/jenkins_home/jobs/Node/workspace/out/Release/obj/gen/icutmp/icudt57l.dat

in terms of the variables that I'm settings before I run the .configure script, they are the following

#!/bin/sh
export AR=~/timesys/bin/armv7l-timesys-linux-gnueabi-ar
export CC=~/timesys/bin/armv7l-timesys-linux-gnueabi-gcc
export CXX=~/timesys/bin/armv7l-timesys-linux-gnueabi-g++
export LINK=~/timesys/bin/armv7l-timesys-linux-gnueabi-g++

export TARGET_ARCH="-march=armv7l"
export TARGET_TUNE="-mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp -mthumb-interwork -mno-thumb"

#DEFINE FLAGS
export CXX_TARGET_ARCH="-march-armv7l"
export V8_TARGET_ARCH="-march-armv7l"

to compile on the host, I think you need to be able to set up timesys distribution on your machine,, i think bin folder the bin folder for me takes 37MB. I'll check if that's available from timesys website ( i doubt though), those were given to me.

[srl295]

I (maybe) got setup for cross compiling on raspberry pi (set AR/CC/CXX/LINK), and was able to reproduce:

./out/Release/icupkg:                                 ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=364920e9400e9bb597ff0829c9fc56b9026a1ab5, not stripped

So what gives here? Why was icupkg compiled for the target when it's clearly toolset: host?
Looking…

[huseyinkozan]

Hello,
I tried to build Node package for OpenWRT MIPS arch for Tessel. I searched what ICU stands for, and I thing we cannot disable it. And also, we should not increase the package size (if possible) due to limitted hardware resources [0].

[0] https://tessel.io/blog/113259439202/tessel-2-hardware-overview

[srl295]

@huseyinkozan You can disable with --without-intl passed to configure, but not sure why you mean you can't disable it.

[srl295]

//cc @nodejs/build

[rvagg]

@joaocgreis I figure you might be an ideal person to have a look here since you're our resident cross-compiling expert

[joaocgreis]

Host arch detection is done using whatever is in the CXX variable, so it's detecting both the target and the host to be arm.

@mrvini can you add CXX_host and CC_host variables? V8 only supports building for arm in arm and x86 (32-bit) hosts, so the variables need -m32 appended. Something like CXX_host="g++ -m32".

This works well with the Raspberry Pi tools for armv7-a, but I've heard of linking problems with other toolchains. I can't dig into that, but if you get it and can figure it out, it would be great to know.

[huseyinkozan]

@srl295 If I disable it, will the APIs depends that work ?

[mrvini]

Thanks @joaocgreis , so I"ve tried to add the flag that you've asked for

export CXX_host="g++ -m32"

getting an error

In file included from /usr/include/stdint.h:25:0,
                 from /usr/lib/gcc/x86_64-linux-gnu/4.9/include/stdint.h:9,
                 from ../deps/icu/source/common/unicode/ptypes.h:66,
                 from ../deps/icu/source/common/unicode/umachine.h:44,
                 from ../deps/icu/source/common/unicode/utypes.h:36,
                 from ../deps/icu/source/tools/toolutil/ppucd.cpp:15:
/usr/include/features.h:374:25: fatal error: sys/cdefs.h: No such file or directory
 #  include <sys/cdefs.h>
                         ^
compilation terminated.
tools/icu/icutools.host.mk:530: recipe for target '/var/jenkins_home/jobs/Node/workspace/out/Release/obj.host/icutools/deps/icu/source/tools/toolutil/ppucd.o' failed

[srl295]

@joaocgreis thanks!

@mrvini ok, so now we're getting somewhere, I suppose. You may be missing some of the m32 header files Anyway - I'll test when I get a chance

@huseyinkozan if you disable it then some APIs such as Intl won't work, correct. But perhaps try it, it doesn't add too much space.

[huseyinkozan]

@srl295 Firstly, I tried to package icu as an extarnal library and build Node with --with-intl=system-icu. But couldnt pass the required parameters to OpenWRT package (I am not an expert).

I will try to build with --without-intl.

[srl295]

@huseyinkozan try this - pkg-config icu-uc --modversion - it should show a version. If it doesn't work, system-icu won't work. You can install the icu libraries in your system.

[huseyinkozan]

@srl295 I called the command :

$ ./host/pkg-config-0.28/pkg-config icu-uc --modversion
57.1

But I couldnt find [0] how to use pkg-config at this OpenWRT Makefile :
https://github.com/huseyinkozan/openwrt-tessel/blob/master/package/node/node/Makefile

[0] https://wiki.openwrt.org/doc/devel/packages

[huseyinkozan]

Building with --without-intl parameter success. Thank you @srl295.

[srl295]

@huseyinkozan — welcome.
@joaocgreis @mrvini - adding export CC_host=gcc CXX_host=g++ LINK_host=g++ _seemed_ to work for me for the rpi cross compile. I say _seemed_ because my rPi isn't actually online now.

Note: This isn't really a new requirement — but it seems that only the ICU build process actually requires these to be set properly.

[mrvini]

@srl295

I've tried your suggestion and set additional flags

export TARGET_ARCH="-march=armv7l"
export TARGET_TUNE="-mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp -mthumb-interwork -mno-thumb"
export CXX_TARGET_ARCH="-march-armv7l"
export V8_TARGET_ARCH="-march-armv7l"
export CXX_host="g++ -m32"
#export CXX_host="g++" 
export CC_host="gcc" 
export LINK_host="g++"

i got the error when I set export CXX_host="g++ -m32"
and got the following error

MMD -MF /var/jenkins_home/jobs/Node/workspace/out/Release/.deps//var/jenkins_home/jobs/Node/workspace/out/Release/obj.host/icutools/deps/icu/source/tools/toolutil/ppucd.o.d.raw   -c -o /var/jenkins_home/jobs/Node/workspace/out/Release/obj.host/icutools/deps/icu/source/tools/toolutil/ppucd.o ../deps/icu/source/tools/toolutil/ppucd.cpp
In file included from /usr/include/stdint.h:25:0,
                 from /usr/lib/gcc/x86_64-linux-gnu/4.9/include/stdint.h:9,
                 from ../deps/icu/source/common/unicode/ptypes.h:66,
                 from ../deps/icu/source/common/unicode/umachine.h:44,
                 from ../deps/icu/source/common/unicode/utypes.h:36,
                 from ../deps/icu/source/tools/toolutil/ppucd.cpp:15:
/usr/include/features.h:374:25: fatal error: sys/cdefs.h: No such file or directory
 #  include <sys/cdefs.h>
                         ^
compilation terminated.

when I set CXX_host="g++" instead, i am getting the following error

"/var/jenkins_home/jobs/Node/workspace/out/Release/obj.target/deps/v8/tools/gyp/libv8_nosnapshot.a" "/var/jenkins_home/jobs/Node/workspace/out/Release/libv8_nosnapshot.a")
  g++ -pthread -rdynamic  -o /var/jenkins_home/jobs/Node/workspace/out/Release/genccode -Wl,--start-group /var/jenkins_home/jobs/Node/workspace/out/Release/obj.host/genccode/deps/icu/source/tools/genccode/genccode.o /var/jenkins_home/jobs/Node/workspace/out/Release/obj.host/genccode/tools/icu/no-op.o /var/jenkins_home/jobs/Node/workspace/out/Release/obj.host/tools/icu/libicutools.a -Wl,--end-group 
/var/jenkins_home/jobs/Node/workspace/out/Release/obj.host/tools/icu/libicutools.a: error adding symbols: Archive has no index; run ranlib to add one
collect2: error: ld returned 1 exit status

do the flags looks ok? or I am missing some?

[joaocgreis]

@mrvini as @srl295 pointed above, you need to install support files to compile with -m32. On Ubuntu this is just sudo apt-get install gcc-multilib g++-multilib.

You should also have all the host variables with -m32:

export CXX_host="g++ -m32"
export CC_host="gcc -m32"
export LINK_host="g++ -m32"

[mrvini]

@joaocgreis , @srl295 thanks it make sense now and it works :) :+1:

just in case anyone might be interested in the future, as @joaocgreis mentioned, install multilib first

sudo dpkg --add-architecture i386
sudo apt-get install gcc-multilib g++-multilib

and the flags are the following

export TARGET_ARCH="-march=armv7l"
export TARGET_TUNE="-mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp -mthumb-interwork -mno-thumb"
export CXX_TARGET_ARCH="-march-armv7l"
export V8_TARGET_ARCH="-march-armv7l"
export CXX_host="g++ -m32"
export CC_host="gcc -m32" 
export LINK_host="g++ -m32"

./configure --without-snapshot --dest-cpu=arm --dest-os=linux --with-arm-float-abi=softfp --with-intl=full-icu --download=all;

[joaocgreis]

@mrvini Good to know, thanks for the summary!

[srl295]

@mrvini you have --with-intl=full-icu --download=all - that's great, gives you the full i18n data. You can also just omit those flags and you'll get the default for node, English only.

[mrvini]

thanks @srl295 , got u

[nappy]

@joaocgreis I am trying to get the Intl feature working on android with the suggestions above.

sudo dpkg --add-architecture i386
sudo apt-get install gcc-multilib g++-multilib

export CXX_host="g++ -m32"
export CC_host="gcc -m32"
export LINK_host="g++ -m32"

But I get the following error on the ICU part:

g++ -m32 -rdynamic -fPIE -pie  -o /home/nodejs/node/out/Release/iculslocs /home/nodejs/node/out/Release/obj.host/iculslocs/tools/icu/iculslocs.o /home/nodejs/node/out/Release/obj.host/iculslocs/tools/icu/no-op.o /home/nodejs/node/out/Release/obj.host/tools/icu/libicutools.a -llog
/usr/bin/ld: cannot find -llog

To my knowledge -llog is a android specific library, so how to ensure the flag is not provided to the host toolchain? I found it being set in common.gypi, but I have no clue how to make the switch.

[srl295]

@nappy this seems like a separate issue.

common.gypi has

      [ 'OS=="android"', {
        'defines': ['_GLIBCXX_USE_C99_MATH'],
        'libraries': [ '-llog' ],
      }],

seems like it may need a conditional there? Perhaps it's yet another latent bug that building with intl exposes?

[huseyinkozan]

I tried to compile by removing the --without-intl option,

CONFIGURE_ARGS = --dest-cpu=$(DEST_CPU) --dest-os=linux \
  --shared-openssl --shared-openssl-includes="$(STAGING_DIR)/usr/include" --shared-openssl-libpath="$(STAGING_DIR)/usr/lib" \
  --shared-zlib --shared-zlib-includes="$(STAGING_DIR)/usr/include" --shared-zlib-libpath="$(STAGING_DIR)/usr/lib" \
  --v8-options="--max_old_space_size=20 --initial_old_space_size=4 --max_semi_space_size=2 --max_executable_size=5" \
  --without-snapshot --with-mips-arch-variant=r2 --with-mips-fpu-mode=fp32 --with-mips-float-abi=soft

installed 32bit libs with,

sudo dpkg --add-architecture i386
sudo apt-get install gcc-multilib g++-multilib

set *_host flags as noted above,

export CXX_host="g++ -m32"
export CC_host="gcc -m32"
export LINK_host="g++ -m32"

and got this error :

g++ -m32 -pthread -rdynamic  -o /home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/out/Release/genccode -Wl,--start-group /home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/out/Release/obj.host/genccode/deps/icu-small/source/tools/genccode/genccode.o /home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/out/Release/obj.host/genccode/tools/icu/no-op.o /home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/out/Release/obj.host/tools/icu/libicutools.a -Wl,--end-group -L/home/hk/p/openwrt-tessel/openwrt/staging_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/usr/lib -lz -lssl -lcrypto
/usr/bin/ld: skipping incompatible /home/hk/p/openwrt-tessel/openwrt/staging_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/usr/lib/libz.so when searching for -lz
/usr/bin/ld: skipping incompatible /home/hk/p/openwrt-tessel/openwrt/staging_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/usr/lib/libz.a when searching for -lz
/usr/bin/ld: cannot find -lz
/usr/bin/ld: skipping incompatible /home/hk/p/openwrt-tessel/openwrt/staging_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/usr/lib/libssl.so when searching for -lssl
/usr/bin/ld: skipping incompatible /home/hk/p/openwrt-tessel/openwrt/staging_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/usr/lib/libssl.a when searching for -lssl
/usr/bin/ld: cannot find -lssl
/usr/bin/ld: skipping incompatible /home/hk/p/openwrt-tessel/openwrt/staging_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/usr/lib/libcrypto.so when searching for -lcrypto
/usr/bin/ld: skipping incompatible /home/hk/p/openwrt-tessel/openwrt/staging_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/usr/lib/libcrypto.a when searching for -lcrypto
/usr/bin/ld: cannot find -lcrypto
collect2: error: ld returned 1 exit status
tools/icu/genccode.host.mk:136: recipe for target '/home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/out/Release/genccode' failed
make[4]: *** [/home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/out/Release/genccode] Error 1
make[4]: Leaving directory '/home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/out'
Makefile:66: recipe for target 'node' failed
make[3]: *** [node] Error 2
make[3]: Leaving directory '/home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0'
Makefile:67: recipe for target '/home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/.built' failed
make[2]: *** [/home/hk/p/openwrt-tessel/openwrt/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/node-v6.5.0/.built] Error 2
make[2]: Leaving directory '/home/hk/p/openwrt-tessel/package/node/node'
package/Makefile:191: recipe for target 'package/feeds/tessel/node/compile' failed
make[1]: *** [package/feeds/tessel/node/compile] Error 2
make[1]: Leaving directory '/home/hk/p/openwrt-tessel/openwrt'
/home/hk/p/openwrt-tessel/openwrt/include/toplevel.mk:181: recipe for target 'package/node/compile' failed
make: *** [package/node/compile] Error 2

Any idea ?

[bnoordhuis]

@huseyinkozan Either install those libraries or omit --shared-openssl, --shared-zlib, etc. if you don't have them?

[huseyinkozan]

@bnoordhuis I did:

CONFIGURE_ARGS = --dest-cpu=$(DEST_CPU) --dest-os=linux \
  --shared-openssl --shared-openssl-includes="$(STAGING_DIR)/usr/include" --shared-openssl-libpath="$(STAGING_DIR)/usr/lib" \
  --shared-zlib --shared-zlib-includes="$(STAGING_DIR)/usr/include" --shared-zlib-libpath="$(STAGING_DIR)/usr/lib" \
  --v8-options="--max_old_space_size=20 --initial_old_space_size=4 --max_semi_space_size=2 --max_executable_size=5" \
  --without-snapshot --with-mips-arch-variant=r2 --with-mips-fpu-mode=fp32 --with-mips-float-abi=soft

[huseyinkozan]

Hmm, sorry. Miss understand.
They are built, but I thing they built against target, not host (or vice versa).

[srl295]

ICU needs to link and run code on both host and target. Can you make sure the libraries are available on both?

[huseyinkozan]

Yes, I have both at host and target.

I tried two different ways; --with-intl=sysyem-icu, and exporting *_host variables with default icu options. One of them fails at genccode, other fails at execinfo.h. Dont remember which one is.

[huseyinkozan]

Found execinfo.h issue : https://github.com/nodejs/node/issues/8233. Trying ...

[srl295]

Ok so the execinfo issue is not icu related right?

[huseyinkozan]

Yes. I give up with icu. I built with --without-intl option. My trials are here :
https://github.com/huseyinkozan/openwrt-tessel/tree/node-6.5.0