Frr: FRR 4.0 Cross Compilation Issue

Created on 27 Apr 2018 · 17Comments · Source: FRRouting/frr

I have been trying to cross compile the FRR 4.0 (d90b2b7), and it ends up with the following error; the issue is similar as reported in #1344 and #958

Attached are the config.log files

make[3]: Entering directory /home/rajaram/frr-frr-4.0-builddir/hosttools' make[3]: *** No rule to make targetlib/route_types.h'. Stop.
make[3]: Leaving directory /home/rajaram/frr-frr-4.0-builddir/hosttools' make[2]: *** [hosttools/lib/clippy] Error 2 make[2]: Leaving directory/home/rajaram/frr-frr-4.0-builddir'
make[1]: * [all-recursive] Error 1
make[1]: Leaving directory `/home/rajaram/frr-frr-4.0-builddir'
make: * [all] Error 2
frr_config.log.txt
frr_hosttools_config.log.txt

question

Source

rajaram-ghosh

All 17 comments

for #958 I solved every problem, in what environment are you trying to do this ?
this is your problem

configure:4201: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.

you need to pass more flags, look at the configure for openwrt, I'll update the PR in openwrt routing section, but replace the variables with ones that suits you

define Build/Configure
    ( cd $(PKG_BUILD_DIR)/ ; \
    mkdir build/ ; \
    cd build ; \
    ../configure \
        --host="$(GNU_TARGET_NAME)" \
        --build="$(GNU_HOST_NAME)" \
        CFLAGS="$(TARGET_CFLAGS) $(EXTRA_CFLAGS)" \
        CXXFLAGS="$(TARGET_CFLAGS) $(EXTRA_CFLAGS)" \
        LDFLAGS="$(TARGET_LDFLAGS) $(EXTRA_LDFLAGS)" \
        HOST_CFLAGS="$(HOST_CFLAGS)" \
        HOST_LDFLAGS="$(HOST_LDFLAGS)" \
        BUILD_CPPFLAGS="$(TARGET_CPPLAGS)" \
        BUILD_CFLAGS="$(TARGET_CFLAGS)" \
        BUILD_LDFLAGS="$(TARGET_LDFLAGS)" \
        --localstatedir=/var/run/frr \
        --sysconfdir=/etc/frr/ \
        --enable-shared \
        --disable-static \
        --enable-user=network \
        --enable-group=network \
        --enable-multipath=8 \
        --disable-capabilities \
        --disable-ospfclient \
        --disable-doc \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-babeld,babeld) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-bgpd,bgpd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-eigrpd,eigrpd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-isisd,isisd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-ldpd,ldpd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-nhrpd,nhrpd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-ospfd,ospfd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-ospf6d,ospf6d) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-pimd,pimd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-ripd,ripd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-ripngd,ripngd) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-vtysh,vtysh) \
        $(call autoconf_bool,CONFIG_PACKAGE_frr-libfrr,zebra) \
)
endef

lucize on 27 Apr 2018

I am using Yocto Cross Compiler Toolchain to build the FRR from command line, is there a Yocto recipe available for FRR?

Before running 'configure' the environment variables are set using the following script
environment-setup-ppce5500-fsl-linux.txt

Do you see any issue in there?

The --host=powerpc-fsl-linux is also set while running 'configure'

$ /home/rajaram/Downloads/FRRouting/frr-frr-4.0/configure --host=powerpc-fsl-linux --prefix=/home/rajaram/frrouter/frr --sysconfdir=/home/rajaram/frrouter/frr_conf --localstatedir=/var/run --enable-user=root --enable-group=root

rajaram-ghosh on 28 Apr 2018

when cross compile, you have to configure from a sub-folder (for clippy)
like mkdir build/ ; cd build ; ../configure --args and after that do a make from the sub-folder
is your FRRouting the source and frr-frr-4.0 an empty folder ?
anyway you'll need the host build flags and target build flags that I have specified and your environment setup is not setting all of them

lucize on 28 Apr 2018

these are your BUILD_* flags, you have to adjust them, also you need the HOST_* flags

export CC="powerpc-fsl-linux-gcc  -m32 -mhard-float -mcpu=e5500 --sysroot=$SDKTARGETSYSROOT"
export CXX="powerpc-fsl-linux-g++  -m32 -mhard-float -mcpu=e5500 --sysroot=$SDKTARGETSYSROOT"
export CPP="powerpc-fsl-linux-gcc -E  -m32 -mhard-float -mcpu=e5500 --sysroot=$SDKTARGETSYSROOT"
export AS="powerpc-fsl-linux-as "
export LD="powerpc-fsl-linux-ld  --sysroot=$SDKTARGETSYSROOT"
export GDB=powerpc-fsl-linux-gdb
export STRIP=powerpc-fsl-linux-strip
export RANLIB=powerpc-fsl-linux-ranlib
export OBJCOPY=powerpc-fsl-linux-objcopy
export OBJDUMP=powerpc-fsl-linux-objdump
export AR=powerpc-fsl-linux-ar
export NM=powerpc-fsl-linux-nm

lucize on 28 Apr 2018

I am not sure if I understood what changes you are asking me to do;

What should be the value for HOST_CFLAGS, HOST_LDFLAGS?

I have set values for BUILD_CFLAGS and BUILD_LDFLAGS same as CFLAGS and LDFLAGS respectively, BUILD_CPPFLAGS is empty.

What other changes I need to make.

Here are the steps I followed, and the result are same
frr_config.log
frr_hosttools_config.log

$ cd /home/rajaram/git

$ git clone https://github.com/FRRouting/frr.git

$ cd frr/

$ ./bootstrap.sh

$ mkdir build

$ cd build/

$ . /opt/fsl-networking/QorIQ-SDK-V1.7/environment-setup-ppce5500-fsl-linux

$ ../configure
--host=powerpc-fsl-linux
--build=x86_64-linux
CFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types"
CXXFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types"
LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
HOST_CFLAGS=""
HOST_LDFLAGS=""
BUILD_CPPFLAGS=""
BUILD_CFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types"
BUILD_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
--localstatedir=/var/run/frr
--sysconfdir=/home/rajaram/frrouter/frr_conf/
--enable-shared
--disable-static
--enable-user=root
--enable-group=root
--enable-multipath=8
--disable-capabilities
--disable-ospfclient
--disable-doc

rajaram-ghosh on 28 Apr 2018

it can be that SDK is building is own version of host compiler or libs, not only for crosscompile, but for using it on the system, maybe it is not available on the installed machine
I uploaded a working hosttools/config.log and a diff between FLAGS used in host and in target
cat config.log | grep FLAGS

FLAGS.diff.txt
config.log.txt

from your log uname -m = x86_64 so in hosttools you should have x86_64 stuff but you have powerpc

configure:3481: found /opt/fsl-networking/QorIQ-SDK-V1.7/sysroots/x86_64-fslsdk-linux/usr/bin/perl
configure:3493: result: /opt/fsl-networking/QorIQ-SDK-V1.7/sysroots/x86_64-fslsdk-linux/usr/bin/perl
configure:3512: checking for x86_64-pc-linux-gnu-pkg-config
configure:3545: result: no
configure:3555: checking for pkg-config
configure:3573: found /opt/fsl-networking/QorIQ-SDK-V1.7/sysroots/x86_64-fslsdk-linux/usr/bin/pkg-config
configure:3585: result: /opt/fsl-networking/QorIQ-SDK-V1.7/sysroots/x86_64-fslsdk-linux/usr/bin/pkg-config
configure:3610: checking pkg-config is at least version 0.9.0
configure:3613: result: yes
configure:3693: checking for x86_64-pc-linux-gnu-gcc
configure:3720: result: powerpc-fsl-linux-gcc  -m32 -mhard-float -mcpu=e5500 --sysroot=/home/rajaram/QorIQ-SDK-V1.7-20141218-yocto/build_t1040rdb_release/tmp/sysroots/t1040rdb
configure:3989: checking for C compiler version
configure:3998: powerpc-fsl-linux-gcc  -m32 -mhard-float -mcpu=e5500 --sysroot=/home/rajaram/QorIQ-SDK-V1.7-20141218-yocto/build_t1040rdb_release/tmp/sysroots/t1040rdb --version >&5
powerpc-fsl-linux-gcc (GCC) 4.8.1

powerpc-fsl-linux-gcc is not ok, you need to use some x86_64 here, pass the location of your system gcc or some manual built gcc that can run on your host

my cat hosttools/config log | grep CPP is

ac_cv_prog_CPP='gcc -E'
CPP='gcc -E'

you have powerpc-fsl-linux-gcc ...
CC as well
LD again
correct should be like
LD='/usr/bin/ld -m elf_x86_64'

lucize on 29 Apr 2018

maybe my example for a crosscompile on x86_64 to x86_64 (with other libs) is not so obvious, I can show you later a diff between an x86_64 and arm, but I think you got the idea of the problem

lucize on 29 Apr 2018

I get that, the C compiler used is powerpc-fsl-linux-gcc, and so the executable produced cannot be run in the native x86_64 build PC and hence the below error

configure:4201: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.

But why does it selects the cross gcc when it should be selecting the the native gcc?

rajaram-ghosh on 30 Apr 2018

don't run environment-setup-ppce5500-fsl-linux and manually add

        --host="$(GNU_TARGET_NAME)" \
        --build="$(GNU_HOST_NAME)" \
        CFLAGS="$(TARGET_CFLAGS) $(EXTRA_CFLAGS)" \
        CXXFLAGS="$(TARGET_CFLAGS) $(EXTRA_CFLAGS)" \
        LDFLAGS="$(TARGET_LDFLAGS) $(EXTRA_LDFLAGS)" \
        HOST_CFLAGS="$(HOST_CFLAGS)" \
        HOST_LDFLAGS="$(HOST_LDFLAGS)" \
        BUILD_CPPFLAGS="$(TARGET_CPPLAGS)" \
        BUILD_CFLAGS="$(TARGET_CFLAGS)" \
        BUILD_LDFLAGS="$(TARGET_LDFLAGS)" \

from that file so it suits your needs, maybe you only need to pass the $TARGET flags manually

lucize on 30 Apr 2018

If I don't run the environment-setup-ppce5500-fsl-linux script then the cross toolchain path won't get set, and make compiles everything with the native gcc for the i386:x86-64 platform as shown in the objdump output

$ cd /home/rajaram/git

$ git clone https://github.com/FRRouting/frr.git

$ cd frr/

$ ./bootstrap.sh 

$ mkdir build

$ cd build/

$ ../configure \
        --host=powerpc-fsl-linux \
        --build=x86_64-linux \
        CFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types" \
        CXXFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types" \
        LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed" \
        HOST_CFLAGS="" \
        HOST_LDFLAGS="" \
        BUILD_CPPFLAGS="" \
        BUILD_CFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types" \
        BUILD_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed" \
        --localstatedir=/var/run/frr \
        --sysconfdir=/home/rajaram/frrouter/frr_conf/ \
        --enable-shared \
        --disable-static \
        --enable-user=root \
        --enable-group=root \
        --enable-multipath=8 \
        --disable-capabilities \
        --disable-ospfclient \
        --disable-doc \

$ make

$ objdump -f isisd/isis_te.o 

isisd/isis_te.o:     file format elf64-x86-64
architecture: i386:x86-64, flags 0x00000011:
HAS_RELOC, HAS_SYMS
start address 0x0000000000000000

If I export the toolchain path before running configure, then it ends up in error checking whether the C compiler works... no

$ export PATH=/home/rajaram/QorIQ-SDK-V1.7-20141218-yocto/build_t1040rdb_release/tmp/sysroots/x86_64-linux/usr/bin:/home/rajaram/QorIQ-SDK-V1.7-20141218-yocto/build_t1040rdb_release/tmp/sysroots/x86_64-linux/usr/bin/ppce5500-fsl-linux:$PATH

$ ../configure \
        --host=powerpc-fsl-linux \
        --build=x86_64-linux \
        CFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types" \
        CXXFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types" \
        LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed" \
        HOST_CFLAGS="" \
        HOST_LDFLAGS="" \
        BUILD_CPPFLAGS="" \
        BUILD_CFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types" \
        BUILD_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed" \
        --localstatedir=/var/run/frr \
        --sysconfdir=/home/rajaram/frrouter/frr_conf/ \
        --enable-shared \
        --disable-static \
        --enable-user=root \
        --enable-group=root \
        --enable-multipath=8 \
        --disable-capabilities \
        --disable-ospfclient \
        --disable-doc \

configure: WARNING: using cross tools not prefixed with host triplet
checking pkg-config is at least version 0.9.0... yes
checking for powerpc-fsl-linux-gcc... powerpc-fsl-linux-gcc
checking whether the C compiler works... no
configure: error: in `/home/rajaram/Downloads/FRRouting/frr-master/build':
configure: error: C compiler cannot create executables
See `config.log' for more details

rajaram-ghosh on 1 May 2018

you're not doing it right, the TARGET_* flags are not just -O2 -pipe -g -feliminate-unused-debug-types or -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed
look into the environment setup and add the full path to gcc, cpp, ld, etc..
the lines should be quite long

lucize on 1 May 2018

From your setup can you give me the value of TARGET_CFLAGS EXTRA_CFLAGS TARGET_LDFLAGS EXTRA_LDFLAGS HOST_CFLAGS HOST_LDFLAGS

        CFLAGS="$(TARGET_CFLAGS) $(EXTRA_CFLAGS)" \
        CXXFLAGS="$(TARGET_CFLAGS) $(EXTRA_CFLAGS)" \
        LDFLAGS="$(TARGET_LDFLAGS) $(EXTRA_LDFLAGS)" \
        HOST_CFLAGS="$(HOST_CFLAGS)" \
        HOST_LDFLAGS="$(HOST_LDFLAGS)" \

rajaram-ghosh on 1 May 2018

here is the whole configure line for an arm target built on x86_64
config_variables_arm_openwrt.txt

lucize on 1 May 2018

I ran in to this exact error as well. The problem turned out to be a missing dependency, python-dev.

If I've understood the cross-compilation process correctly, configure will automatically run another configure in a separate hosttools directory where it will build only clippy. However, if the sub-configure fails for any reason that error does not propagate to the upper configure.

I think it would be helpful to wrap the sub-configure in some AC_CHECK_BLACK_MAGIC that aborts configure in this case.