Envoy: Quiche build failure under gcc

[dmitri-d]

Quiche fails to build under gcc (9.2.x) with the following error:

FAIL: //test/extensions/grpc_credentials/aws_iam:aws_iam_grpc_credentials_test (see /build/tmp/_bazel_bazel/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-opt/testlogs/test/extensions/grpc_credentials/aws_iam/aws_iam_grpc_credentials_test/test.log)
INFO: From Testing //test/extensions/grpc_credentials/aws_iam:aws_iam_grpc_credentials_test:
ERROR: /build/tmp/_bazel_bazel/b570b5ccd0454dc9af9f65ab1833764d/external/com_googlesource_quiche/BUILD.bazel:1988:1: C++ compilation of rule '@com_googlesource_quiche//:quic_core_framer_lib' failed (Exit 1)
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/quic_framer.cc: In member function 'bool quic::QuicFramer::AppendIetfAckFrameAndTypeByte(const quic::QuicAckFrame&, quic::QuicDataWriter*)':
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/quic_framer.cc:5420:40: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
         writer->remaining() - ecn_size <
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
             QuicDataWriter::GetVarInt62Len(gap) +
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                 QuicDataWriter::GetVarInt62Len(ack_range)) {
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[dmitri-d]

Line numbers have changed since I reported the issue:

bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/quic_framer.cc: In member function 'bool quic::QuicFramer::AppendIetfAckFrameAndTypeByte(const quic::QuicAckFrame&, quic::QuicDataWriter*)':
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/quic_framer.cc:5434:40: error: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'int' [-Werror=sign-compare]
 5434 |         writer->remaining() - ecn_size <
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
 5435 |             QuicDataWriter::GetVarInt62Len(gap) +
      |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 5436 |                 QuicDataWriter::GetVarInt62Len(ack_range)) {
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[danzh2010]

/assign @danzh2010

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.

[kkHAIKE]

same issus after https://github.com/envoyproxy/envoy/pull/13066

[wrowe]

See the same, specific details I encountered with ubuntu 18.04 GCC as documented in this comment; https://github.com/envoyproxy/envoy/pull/12898#issuecomment-696914168

[lazy-always]

We are also getting the same build failure

ERROR: /root/.cache/bazel/_bazel_root/0bedc29ce814c783e9af4c990605c934/external/com_googlesource_quiche/BUILD.bazel:2170:17: C++ compilation of rule '@com_googlesource_quiche//:quic_core_framer_lib' failed (Exit 1) gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 272 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
bazel-out/k8-fastbuild/bin/external/com_googlesource_quiche/quiche/quic/core/quic_framer.cc: In member function 'bool quic::QuicFramer::AppendIetfAckFrameAndTypeByte(const quic::QuicAckFrame&, quic::QuicDataWriter*)':
bazel-out/k8-fastbuild/bin/external/com_googlesource_quiche/quiche/quic/core/quic_framer.cc:5546:61: error: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'int' [-Werror=sign-compare]
5546 | static_cast(writer->remaining() - ecn_size) <
| ~~~~~~~~~~^
5547 | QuicDataWriter::GetVarInt62Len(gap) +
| ~~~~~~~~~
5548 | QuicDataWriter::GetVarInt62Len(ack_range)) {
| ~~~~~~~~~~~

[NomadXD]

Hi I am getting the same build failure. I'm trying to build on ubuntu 20.04. My specs are as follows.

• bazel 3.4.1
• gcc 9.3.0
• g++ 9.3.0

When I try to run bazel build //:envoy I get the following error.
ERROR: /home/khadija/.cache/bazel/_bazel_khadija/67b71b3527ed4854635b017e6b6cd642/external/com_googlesource_quiche/BUILD.bazel:2170:17: C++ compilation of rule '@com_googlesource_quiche//:quic_core_framer_lib' failed (Exit 1): gcc failed: error executing command (cd /home/khadija/.cache/bazel/_bazel_khadija/67b71b3527ed4854635b017e6b6cd642/sandbox/linux-sandbox/2634/execroot/envoy_filter_example && \ exec env - \ BAZEL_LINKLIBS=-l%:libstdc++.a \ BAZEL_LINKOPTS=-lm \ CC=gcc \ CXX=g++ \ PATH=/home/khadija/.cache/bazelisk/downloads/bazelbuild/bazel-3.4.1-linux-x86_64/bin:/usr/lib/llvm-8/bin:/home/khadija/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin \ PWD=/proc/self/cwd \

And it fails the build giving the following stacktrace.
[-Werror=sign-compare] 5546 | static_cast<size_t>(writer->remaining() - ecn_size) < | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ 5547 | QuicDataWriter::GetVarInt62Len(gap) + | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 5548 | QuicDataWriter::GetVarInt62Len(ack_range)) { | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ At global scope: cc1plus: error: unrecognized command line option '-Wno-range-loop-analysis' [-Werror] cc1plus: all warnings being treated as errors Target //:envoy failed to build

Any solution ?

[thomas1984]

@NomadXD
Hello! Please check the Readme document and perform some operations on the Linux system:
Linux
On Linux, we recommend using the prebuilt Clang+LLVM package from LLVM official site. Extract the tar.xz and run the following:

bazel/setup_clang.sh
This will setup a clang.bazelrc file in Envoy source root. If you want to make clang as default, run the following:

echo "build --config=clang" >> user.bazelrc
Note: Either libc++ or libstdc++-7-dev (or higher) must be installed.

I didn’t execute the setup_clang.sh command before, and I got the same error as yours. I downloaded a copy of clang+llvm-11.0.0-x86_64-linux-gnu-ubuntu-20.04.tar.xz. After decompression and execution, the compilation succeeded. , I hope this can help you!

[NomadXD]

@thomas1984 Thanks for the reply and sorry I didn't have time to check for couple of weeks. I tried that but still no luck. I did the following steps.

git clone https://github.com/envoyproxy/envoy-filter-example.git
cd envoy-filter-example
git submodule update --init
cd envoy
git checkout v1.16.0
bazel/setup_clang.sh ~/Downloads/clang+llvm-11.0.0-x86_64-linux-gnu-ubuntu-20.04
cd ..
bazel build //http-filter-example:envoy --config=clang

I am getting the following error.

ERROR: /home/khadija/.cache/bazel/_bazel_khadija/b8963e6f21a84c9e75cda0cff04a3eb5/external/envoy/source/common/common/BUILD:321:23: C++ compilation of rule '@envoy//source/common/common:thread_impl_lib_posix' failed (Exit 1) gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 70 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
gcc: error: unrecognized command line option '-fno-limit-debug-info'
gcc: error: unrecognized command line option '-Wgnu-conditional-omitted-operand'
gcc: error: unrecognized command line option '-Wc++2a-extensions'; did you mean '-fms-extensions'?
gcc: error: unrecognized command line option '-Wrange-loop-analysis'
Target //http-filter-example:envoy failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.718s, Critical Path: 0.40s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

I think clang doesn't get configured and thats why there's a gcc error there. Any idea ?

[cooperkuo]

Same issue when using envoyproxy image with gcc9 inside.
The reason I have to use gcc is that our project links some libraries compiled by gcc instead of clang, it seems that it's as difficult to re-compile them as compiling newer envoyproxy using gcc.
It is much appreciate to get an official gcc-way to update our envoyproxy :)

[danzh2010]

This issue should have been fixed in QUICHE and merged into Envoy in #13949.

[NomadXD]

@cooperkuo yeah it's a valid issue i think. lot of people had the same issue and noone gave a proper answer or a way to fix. I tried using clang also but still no luck. :-(

[dmitri-d]

Use --sandbox_debug to see verbose messages from the sandbox
gcc: error: unrecognized command line option '-fno-limit-debug-info'
gcc: error: unrecognized command line option '-Wgnu-conditional-omitted-operand'
gcc: error: unrecognized command line option '-Wc++2a-extensions'; did you mean '-fms-extensions'?
gcc: error: unrecognized command line option '-Wrange-loop-analysis'

clang config generated in envoy submodule isn't being picked up by bazel when building from http-filter-example's main dir. https://github.com/envoyproxy/envoy-filter-example/pull/139 should fix the issue.

[NomadXD]

@dmitri-d Thanks a lot for the tip. Was able to build successfully. Can you build it successfully using gcc as well ? I'm kinda new to envoy and it's ecosystem, so just trying explore everything.

[NomadXD]

@danzh2010 Thanks a lot for the info. Earlier I was trying to build v1.16.0. But now since the quiche build issue is fixed after that , I tried to build the master. Now I am not getting a Quiche build failure but getting the following error.

gcc: fatal error: cannot execute '/usr/lib/gcc/x86_64-linux-gnu/9/cc1plus': execv: Argument list too long

I googled it and seems like it's not something specific to envoy. I think it's related to ARG_MAX value and stack size allocated for params. I'm referring to this https://www.linuxquestions.org/questions/linux-kernel-70/argument-list-too-long-922171/

In the error stack trace I have this part,

gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 1392 argument(s) skipped)

which I guess related to this stack size issue.