Apollo: Can we build each module independently in Apollo?

[hongkedavid]

Hi Apollo developers,

I am wondering if we can build each module independently in Apollo code base. If possible, what is the correct command to build each module?

Thanks very much for your reply!

[xiaoxq]

Yes. You can study bazel a bit, and run command like

bazel build modules/monitor:all

to build a single module.

But in general we have some compiling and linking options which should be set. So it's easier to do apollo.sh to build.

[hongkedavid]

@xiaoxq Thanks very much for your reply!
I am using clang to do some LLVM-based flow analysis of Apollo modules (by setting CC environment as clang), but I encountered some linker errors with _bash apollo.sh build_, for example

ERROR: /apollo/modules/perception/obstacle/onboard/BUILD:199:1: Linking of rule '//modules/perception/obstacle/onboard:perception_obstacle_lidar_process_test' failed (Exit 1)

To debug the error, I am thinking about building each module independently with clang to understand the cause of the error (while I am not sure this is a good direction to go). Is there any instruction you can provide for building Apollo using clang? If not, could you provide further information for the compiling and linking options that should be set for per-module build?

FYI, the detailed output of the linker error with clang is as followed. I have little idea how to further proceed to resolve this error since the apollo build system is based on Bazel and wrapped by some high-level scripts (e.g., apollo.sh).

ERROR: /apollo/modules/perception/obstacle/onboard/BUILD:199:1: Linking of rule '//modules/perception/obstacle/onboard:perception_obstacle_lidar_process_test' failed (Exit 1)
bazel-out/local-dbg/bin/modules/perception/obstacle/onboard/_objs/perception_obstacle_lidar_process_test/modules/perception/obstacle/onboard/lidar_process_test.pic.o: In function `__cxx_global_var_init18':
/proc/self/cwd/modules/perception/obstacle/onboard/lidar_process_test.cc:60: undefined reference to `testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, void const*, void (*)(), void (*)(), testing::internal::TestFactoryBase*)'
bazel-out/local-dbg/bin/modules/perception/obstacle/onboard/_objs/perception_obstacle_lidar_process_test/modules/perception/obstacle/onboard/lidar_process_test.pic.o: In function `__cxx_global_var_init35':
/proc/self/cwd/modules/perception/obstacle/onboard/lidar_process_test.cc:95: undefined reference to `testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, void const*, void (*)(), void (*)(), testing::internal::TestFactoryBase*)'
bazel-out/local-dbg/bin/modules/perception/obstacle/onboard/_objs/perception_obstacle_lidar_process_test/modules/perception/obstacle/onboard/lidar_process_test.pic.o: In function `__cxx_global_var_init39':
/proc/self/cwd/modules/perception/obstacle/onboard/lidar_process_test.cc:119: undefined reference to `testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, void const*, void (*)(), void (*)(), testing::internal::TestFactoryBase*)'
bazel-out/local-dbg/bin/modules/perception/obstacle/onboard/_objs/perception_obstacle_lidar_process_test/modules/perception/obstacle/onboard/lidar_process_test.pic.o: In function `testing::AssertionResult testing::internal::CmpHelperEQ<apollo::perception::PerceptionObstacle_Type, apollo::perception::PerceptionObstacle_Type>(char const*, char const*, apollo::perception::PerceptionObstacle_Type const&, apollo::perception::PerceptionObstacle_Type const&)':
/proc/self/cwd/external/glew/gtest/gtest.h:1333: undefined reference to `testing::internal::EqFailure(char const*, char const*, testing::internal::String const&, testing::internal::String const&, bool)'
bazel-out/local-dbg/bin/modules/perception/obstacle/onboard/_objs/perception_obstacle_lidar_process_test/modules/perception/obstacle/onboard/lidar_process_test.pic.o: In function `testing::AssertionResult testing::internal::CmpHelperEQ<int, int>(char const*, char const*, int const&, int const&)':
/proc/self/cwd/external/glew/gtest/gtest.h:1333: undefined reference to `testing::internal::EqFailure(char const*, char const*, testing::internal::String const&, testing::internal::String const&, bool)'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[lianglia-apollo]

@hongkedavid I am also having this linking issue when using clang to build. Do let us know if you find a solution. Thanks!

[zhxt]

This kind of error is also happened to me when I build Apollo outside of docker for some test module targets(it' odd just for test targets) . It seems bazel can not resolve the needed dependencies correctly for some unknown reason. My workaround for this is adding a extra "linkopts" rule to BUILD files to specify the missing libraries.

[hongkedavid]

@lianglia-apollo @zhxt Thanks guys for your information and suggestion! I will try with the "linkopts" suggestion and see whether it works. Will keep you posted :)

[hongkedavid]

@lianglia-apollo @zhxt I tried to build each module independently (command listed as follows) based on your suggested build command.

bazel build modules/monitor:all
bazel build modules/control:all
bazel build modules/canbus:all
bazel build modules/planning:all
bazel build modules/prediction:all
bazel build modules/localization:all
bazel build modules/routing:all
bazel build modules/third_party_perception:all
bazel build modules/drivers/conti_radar:all
bazel build modules/calibration/lidar_ex_checker:all
bazel build modules/calibration/republish_msg:all
bazel build modules/perception:all
bazel build modules/dreamview:all

All modules can be built successfully except perception and dreamview (error details as follows). With a quick glimpse of the header file _modules/perception/traffic_light/util/color_space.h_, it seems like the error is caused by clang unable to recognize the header file _immintrin.h_ and _x86intrin.h_. Any thought on how to fix this? Thanks!

ERROR: /apollo/modules/perception/traffic_light/util/BUILD:5:1: C++ compilation of rule '//modules/perception/traffic_light/util:perception_traffic_light_util' failed (Exit 1)
warning: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Wunknown-warning-option]
warning: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Wunknown-warning-option]
In file included from modules/perception/traffic_light/util/color_space.cc:36:
./modules/perception/traffic_light/util/color_space.h:162:25: error: use of undeclared identifier '__m256i'
const size_t A = sizeof(__m256i);
                        ^
./modules/perception/traffic_light/util/color_space.h:168:7: error: unknown type name '__m256i'
const __m256i K_ZERO = SIMD_MM256_SET1_EPI8(0);
      ^
./modules/perception/traffic_light/util/color_space.h:169:7: error: unknown type name '__m256i'
const __m256i K_INV_ZERO = SIMD_MM256_SET1_EPI8(0xFF);
      ^
./modules/perception/traffic_light/util/color_space.h:171:7: error: unknown type name '__m256i'
const __m256i K8_01 = SIMD_MM256_SET1_EPI8(0x01);
      ^
./modules/perception/traffic_light/util/color_space.h:172:7: error: unknown type name '__m256i'
const __m256i K8_02 = SIMD_MM256_SET1_EPI8(0x02);
      ^
./modules/perception/traffic_light/util/color_space.h:173:7: error: unknown type name '__m256i'
const __m256i K8_04 = SIMD_MM256_SET1_EPI8(0x04);
      ^
./modules/perception/traffic_light/util/color_space.h:174:7: error: unknown type name '__m256i'
const __m256i K8_08 = SIMD_MM256_SET1_EPI8(0x08);
      ^
./modules/perception/traffic_light/util/color_space.h:175:7: error: unknown type name '__m256i'
const __m256i K8_10 = SIMD_MM256_SET1_EPI8(0x10);
      ^
./modules/perception/traffic_light/util/color_space.h:176:7: error: unknown type name '__m256i'
const __m256i K8_20 = SIMD_MM256_SET1_EPI8(0x20);
      ^
./modules/perception/traffic_light/util/color_space.h:177:7: error: unknown type name '__m256i'
const __m256i K8_40 = SIMD_MM256_SET1_EPI8(0x40);
      ^
./modules/perception/traffic_light/util/color_space.h:178:7: error: unknown type name '__m256i'
const __m256i K8_80 = SIMD_MM256_SET1_EPI8(0x80);
      ^
./modules/perception/traffic_light/util/color_space.h:180:7: error: unknown type name '__m256i'
const __m256i K8_01_FF = SIMD_MM256_SET2_EPI8(0x01, 0xFF);
      ^
./modules/perception/traffic_light/util/color_space.h:182:7: error: unknown type name '__m256i'
const __m256i K16_0001 = SIMD_MM256_SET1_EPI16(0x0001);
      ^
./modules/perception/traffic_light/util/color_space.h:183:7: error: unknown type name '__m256i'
const __m256i K16_0002 = SIMD_MM256_SET1_EPI16(0x0002);
      ^
./modules/perception/traffic_light/util/color_space.h:184:7: error: unknown type name '__m256i'
const __m256i K16_0003 = SIMD_MM256_SET1_EPI16(0x0003);
      ^
./modules/perception/traffic_light/util/color_space.h:185:7: error: unknown type name '__m256i'
const __m256i K16_0004 = SIMD_MM256_SET1_EPI16(0x0004);
      ^
./modules/perception/traffic_light/util/color_space.h:186:7: error: unknown type name '__m256i'
const __m256i K16_0005 = SIMD_MM256_SET1_EPI16(0x0005);
      ^
./modules/perception/traffic_light/util/color_space.h:187:7: error: unknown type name '__m256i'
const __m256i K16_0006 = SIMD_MM256_SET1_EPI16(0x0006);
      ^
./modules/perception/traffic_light/util/color_space.h:188:7: error: unknown type name '__m256i'
const __m256i K16_0008 = SIMD_MM256_SET1_EPI16(0x0008);
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 warnings and 20 errors generated.

clang lib should already contain the header file _immintrin.h_ and _x86intrin.h_. But I am not sure how to configure it in the perception module.

[earlysleepearlyup]

@hongkedavid Hello! I met the same error when I build the perception module. Had you solved this problem?

[hongkedavid]

@earlysleepearlyup I haven't. Please let me know if you find a fix. Thanks!

[layssi]

I have the same issue

[natashadsouza]

We recommend that you build Apollo using our Docker environment as described in our documentation. Unfortunately, building Apollo outside docker is very time consuming and not at all efficient as Apollo relies on several drivers and third-party scripts. Thank you for sharing your experience of building Apollo modules outside Docker, feel free to continue testing it out and let us know if you encounter any other issues.

[hongkedavid]

@natashadsouza Just to clarify, I was not building Apollo outside docker. I was trying to build a single module (i.e., perception) inside Apollo's docker container and encountered that error.

[zhxt]

@hongkedavid Perception module use AVX2 instruction set for optimization, so you may need to add an "--copt=-mavx2 " option to the bazel build command as the script does:
https://github.com/ApolloAuto/apollo/blob/9408323f2c65702a6d194d23a83fe393f2e4d536/apollo.sh#L738

[hongkedavid]

@zhxt Thanks for your information. I will try your suggestion.
@earlysleepearlyup @layssi @Peter9606 You may try this as well.

[natashadsouza]

@hongkedavid thank you for the clarification. If you do encounter any further issues, please reach out to us.
@zhxt thank you for suggesting a fix.

[zhxt]

@hongkedavid Hope that helps.
@natashadsouza My pleasure :-)

[natashadsouza]

@hongkedavid if your issue is resolved by the suggested fix, kindly close the issue.
Thanks!

[jiwonjoung]

Note: for cpu_only build add DCPU_ONLY.
bazel build --copt=-mavx2 --cxxopt=-DCPU_ONLY modules/perception:all

This allowed me to successfully build both dreamview and the perception modules.

[shifakhan4892]

Hi,
I am trying to build modules independently just like how @xiaoxq @hongkedavid suggested using "bazel build ", however i am getting the following error :

ERROR: missing input file '@ros//:include/kdl_conversions/kdl_msg.h'.
ERROR: /home/shifa/.cache/bazel/_bazel_shifa/540135163923dd7d5820f3ee4b306b32/external/ros/BUILD.bazel:5:1: @ros//:ros_common: missing input file '@ros//:include/kdl_conversions/kdl_msg.h'.
ERROR: /home/shifa/.cache/bazel/_bazel_shifa/540135163923dd7d5820f3ee4b306b32/external/ros/BUILD.bazel:5:1 1 input file(s) do not exist.
INFO: Elapsed time: 232.921s, Critical Path: 12.12s

Can anyone please guide me of why it has this dependency issue ? Also to use Clang++ as the compiler for this should I set CC and CXX variables or is there a better method ?

[shifakhan4892]

@lianglia-apollo @zhxt I tried to build each module independently (command listed as follows) based on your suggested build command.

bazel build modules/monitor:all
bazel build modules/control:all
bazel build modules/canbus:all
bazel build modules/planning:all
bazel build modules/prediction:all
bazel build modules/localization:all
bazel build modules/routing:all
bazel build modules/third_party_perception:all
bazel build modules/drivers/conti_radar:all
bazel build modules/calibration/lidar_ex_checker:all
bazel build modules/calibration/republish_msg:all
bazel build modules/perception:all
bazel build modules/dreamview:all

All modules can be built successfully except perception and dreamview (error details as follows). With a quick glimpse of the header file _modules/perception/traffic_light/util/color_space.h_, it seems like the error is caused by clang unable to recognize the header file _immintrin.h_ and _x86intrin.h_. Any thought on how to fix this? Thanks!

ERROR: /apollo/modules/perception/traffic_light/util/BUILD:5:1: C++ compilation of rule '//modules/perception/traffic_light/util:perception_traffic_light_util' failed (Exit 1)
warning: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Wunknown-warning-option]
warning: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Wunknown-warning-option]
In file included from modules/perception/traffic_light/util/color_space.cc:36:
./modules/perception/traffic_light/util/color_space.h:162:25: error: use of undeclared identifier '__m256i'
const size_t A = sizeof(__m256i);
                        ^
./modules/perception/traffic_light/util/color_space.h:168:7: error: unknown type name '__m256i'
const __m256i K_ZERO = SIMD_MM256_SET1_EPI8(0);
      ^
./modules/perception/traffic_light/util/color_space.h:169:7: error: unknown type name '__m256i'
const __m256i K_INV_ZERO = SIMD_MM256_SET1_EPI8(0xFF);
      ^
./modules/perception/traffic_light/util/color_space.h:171:7: error: unknown type name '__m256i'
const __m256i K8_01 = SIMD_MM256_SET1_EPI8(0x01);
      ^
./modules/perception/traffic_light/util/color_space.h:172:7: error: unknown type name '__m256i'
const __m256i K8_02 = SIMD_MM256_SET1_EPI8(0x02);
      ^
./modules/perception/traffic_light/util/color_space.h:173:7: error: unknown type name '__m256i'
const __m256i K8_04 = SIMD_MM256_SET1_EPI8(0x04);
      ^
./modules/perception/traffic_light/util/color_space.h:174:7: error: unknown type name '__m256i'
const __m256i K8_08 = SIMD_MM256_SET1_EPI8(0x08);
      ^
./modules/perception/traffic_light/util/color_space.h:175:7: error: unknown type name '__m256i'
const __m256i K8_10 = SIMD_MM256_SET1_EPI8(0x10);
      ^
./modules/perception/traffic_light/util/color_space.h:176:7: error: unknown type name '__m256i'
const __m256i K8_20 = SIMD_MM256_SET1_EPI8(0x20);
      ^
./modules/perception/traffic_light/util/color_space.h:177:7: error: unknown type name '__m256i'
const __m256i K8_40 = SIMD_MM256_SET1_EPI8(0x40);
      ^
./modules/perception/traffic_light/util/color_space.h:178:7: error: unknown type name '__m256i'
const __m256i K8_80 = SIMD_MM256_SET1_EPI8(0x80);
      ^
./modules/perception/traffic_light/util/color_space.h:180:7: error: unknown type name '__m256i'
const __m256i K8_01_FF = SIMD_MM256_SET2_EPI8(0x01, 0xFF);
      ^
./modules/perception/traffic_light/util/color_space.h:182:7: error: unknown type name '__m256i'
const __m256i K16_0001 = SIMD_MM256_SET1_EPI16(0x0001);
      ^
./modules/perception/traffic_light/util/color_space.h:183:7: error: unknown type name '__m256i'
const __m256i K16_0002 = SIMD_MM256_SET1_EPI16(0x0002);
      ^
./modules/perception/traffic_light/util/color_space.h:184:7: error: unknown type name '__m256i'
const __m256i K16_0003 = SIMD_MM256_SET1_EPI16(0x0003);
      ^
./modules/perception/traffic_light/util/color_space.h:185:7: error: unknown type name '__m256i'
const __m256i K16_0004 = SIMD_MM256_SET1_EPI16(0x0004);
      ^
./modules/perception/traffic_light/util/color_space.h:186:7: error: unknown type name '__m256i'
const __m256i K16_0005 = SIMD_MM256_SET1_EPI16(0x0005);
      ^
./modules/perception/traffic_light/util/color_space.h:187:7: error: unknown type name '__m256i'
const __m256i K16_0006 = SIMD_MM256_SET1_EPI16(0x0006);
      ^
./modules/perception/traffic_light/util/color_space.h:188:7: error: unknown type name '__m256i'
const __m256i K16_0008 = SIMD_MM256_SET1_EPI16(0x0008);
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 warnings and 20 errors generated.

clang lib should already contain the header file _immintrin.h_ and _x86intrin.h_. But I am not sure how to configure it in the perception module.

Did you use the linkopts option for the successful independent builds ? @hongkedavid

[Martin20]

Hello,

I wanted to build the localization module but I got the error below. Any solutions?

sudo bazel build modules/localization:all
(15:06:02) INFO: Current date is 2020-02-07
(15:06:02) ERROR: error loading package '': Encountered error while reading extension file 'bazel/grpc_deps.bzl': no such package '@com_github_grpc_grpc//bazel': Error downloading [file:/home/tmp/v1.14.2.tar.gz] to /home/common/.cache/bazel/_bazel_root/ab3edc26bb1068250376a50283c5876d/external/com_github_grpc_grpc/v1.14.2.tar.gz: /home/tmp/v1.14.2.tar.gz (No such file or directory)
(15:06:02) ERROR: error loading package '': Encountered error while reading extension file 'bazel/grpc_deps.bzl': no such package '@com_github_grpc_grpc//bazel': Error downloading [file:/home/tmp/v1.14.2.tar.gz] to /home/common/.cache/bazel/_bazel_root/ab3edc26bb1068250376a50283c5876d/external/com_github_grpc_grpc/v1.14.2.tar.gz: /home/tmp/v1.14.2.tar.gz (No such file or directory)
(15:06:02) INFO: Elapsed time: 0.232s
(15:06:02) FAILED: Build did NOT complete successfully (0 packages loaded)