Bazel: wrong link order of system libraries in cc rules

[ali5h]

Description of the problem / feature request:

protoc (protobuf v3.8.0 and v3.9.1) build fails with bazel 0.27.1, 0.28.1, 0.29.1 and 1.0.0 on centos 7 with devtoolset-{7,8}. We don't have any CROSSTOOL defined

bazel 0.26.1 works with no issues

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

$ bazel build @com_google_protobuf//:protoc
.....
Execution platform: @bazel_tools//platforms:host_platform
bazel-out/k8-fastbuild/bin/external/com_google_protobuf/_objs/protoc_lib/command_line_interface.pic.o:command_line_interface.cc:function google::protobuf::compiler::CommandLineInterface::ExpandArgumentFile(std::string const&, std::vector<std::string, std::allocator<std::string> >*): error: undefined reference to 'std::basic_ios<char, std::char_traits<char> >::operator bool() const'
bazel-out/k8-fastbuild/bin/external/com_google_protobuf/_objs/protoc_lib/java_helpers.pic.o:java_helpers.cc:function google::protobuf::compiler::java::SortFieldsByNumber(google::protobuf::Descriptor const*): error: undefined reference to '__cxa_throw_bad_array_new_length'
bazel-out/k8-fastbuild/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_message.pic.o:objectivec_message.cc:function google::protobuf::compiler::objectivec::(anonymous namespace)::SortFieldsByNumber(google::protobuf::Descriptor const*): error: undefined reference to '__cxa_throw_bad_array_new_length'
bazel-out/k8-fastbuild/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_message.pic.o:objectivec_message.cc:function google::protobuf::compiler::objectivec::(anonymous namespace)::SortFieldsByStorageSize(google::protobuf::Descriptor const*): error: undefined reference to '__cxa_throw_bad_array_new_length'
bazel-out/k8-fastbuild/bin/external/com_google_protobuf/_objs/protobuf/dynamic_message.pic.o:dynamic_message.cc:function google::protobuf::DynamicMessageFactory::GetPrototypeNoLock(google::protobuf::Descriptor const*): error: undefined reference to '__cxa_throw_bad_array_new_length'
collect2: error: ld returned 1 exit status

What operating system are you running Bazel on?

Centos 7

What's the output of bazel info release?

release 1.0.0 (also tried 0.29.1, 0.28.1 and 0.27.1)

[ali5h]

also seen with devtoolset-8 and bazel-0.29.1

[ali5h]

in bazel 0.27+ stdc++ is being added before other libraries in the params file and linker is dropping unused symbols. in bazel 0.26.1 stdc++ was at the end of the generated params file. This change of order is causing this issue

[ali5h]

cc @iirina @philwo

[ali5h]

friendly ping, this is blocking bazel upgrade for us!

[iirina]

@hlopko can you take a look?

[hlopko]

I guess this is the incompatible flag reponsible for that: https://github.com/bazelbuild/bazel/issues/7687

@oquenchil could you take a look please?

[ali5h]

that flag is switched in 0.26, but 0.26 works fine

[hlopko]

I believe it was flipped in 0.27 (https://github.com/bazelbuild/bazel/commit/28c8a8c91f704e15e28f931347a789d5a1e8fdef is the commit that did that)

[ali5h]

is there any hacky solution for this that can unblock us?

[oquenchil]

Are you using a custom toolchain or doing anything to the command line?

If you are using a custom toolchain, you must make sure that the features appear in the correct order. Is -lstdc++ a toolchain flag, a flag that you pass through an linkopt in a cc_library or a flag that you add in some other way?

See the order of the features for the default toolchain:
https://github.com/bazelbuild/rules_cc/blob/0e66ef31d611b9fe5b762bc7b363e15134c53c85/cc/private/toolchain/unix_cc_toolchain_config.bzl#L1124

Are you passing -lstdc++ with the environment variable BAZEL_LINKOPTS? That's definitely the right way to pass it. For a hacky workaround you may try having a cc_library with a linkopts=["-lstdc++"], then that will be in user_link_flags and therefore below the other libraries. Not ideal but perhaps it works.

[ali5h]

we are using devtoolset-8 on centos. it is activated prior to running the bazel build command which sets up PATH and other env variables. There is no other customization. Also the issue is with building @com_google_protobuf//:protoc so we don't have access to the cc_library rules. I tried running this but still face the same issue:

$ bazel build --action_env=BAZEL_LINKOPTS=-lstdc++ @com_google_protobuf//:protoc

[oquenchil]

Hey, I don't know much about CentOS and devtoolset-8, can you please give me step by step command-lines how to reproduce this using a fresh CentOS 7 VM or using Docker?

We don't test that on our CI, if I can reproduce maybe I can see which changes are necessary on the toolchain.

Thanks!

[philwo]

@ali5h In addition, if you can let us know how to setup this, we can add this platform to our CI and test Bazel on it in the future. :)

[ali5h]

i will setup a repo to reproduce this

[ali5h]

https://github.com/ali5h/bazel_devtoolset

[ali5h]

@oquenchil did u manage to check the sample repo?

[kasper0406]

I hit this issue as well. Currently stuck on version 0.26.1 while there is no known work around or fix.
I tried flipping some of the flags flipped between 0.26.1 and 0.27.0 (ex --incompatible_do_not_split_linking_cmdline), but flipping all of them back to their previous value didn't accomplish anything. My knowledge of Bazel is quite limited, so I didn't dig deeper.

[ali5h]

adding proper toolchain definition fixes the issue, plz check my sample repo

[ali5h]

feel free to close this, if this is the right solution

[oquenchil]

We were discussing inside the team how we could solve this with a new feature but if that works for you I will close the issue.

[alex-xnor]

I'm not sure this should be closed. It's probably true that adding a toolchain definition should make it build successfully (I'm not sure since I haven't tried, but it seems likely that it would work; tomorrow I might have time to try it). But, it still seems like a bug in Bazel's toolchain auto-configuration logic if this doesn't work out of the box.

Here's a scenario that you would expect to work out of the box: Clone a fresh copy of Protobuf (git clone https://github.com/protocolbuffers/protobuf.git). Using the latest released version of Bazel (1.1.0 at the time of writing), run bazel build //:protoc. Ought to work, no?

On most distributions (or, truthfully, I've only tested this on Ubuntu 16.04), this seems to work just fine. But, on Red Hat Enterprise Linux 7 with devtoolset-4, it doesn't work, failing with the error mentioned in the issue description.

At the very least, if the conclusion is that people should be expected to add an explicit toolchain definition, then that should probably be in the documentation. But that kind of defeats the purpose of having toolchain auto-detection.

[oquenchil]

Yes, that doesn't sound right. It seems we need to do more work here. First, I'd like to understand where the flag -lstd is coming from. Do you know this?

[alex-xnor]

Yeah, I think it's coming from here:

https://github.com/bazelbuild/bazel/blob/8075057af6108ebc23c146f18eecec911d4b8c00/tools/cpp/unix_cc_configure.bzl#L370-L375

This ends up going into link_flags here:

https://github.com/bazelbuild/bazel/blob/8075057af6108ebc23c146f18eecec911d4b8c00/tools/cpp/unix_cc_configure.bzl#L519

which then shows up in the resulting cc_toolchain_config's link_flags, visible if you run bazel query --output=build @local_config_cc//:local:

# /extra/build/6aab7f06e75516922b5adde0d3721294/external/local_config_cc/BUILD:72:1
cc_toolchain_config(
  name = "local",
  cpu = "k8",
  compiler = "compiler",
  toolchain_identifier = "local",
  host_system_name = "local",
  target_system_name = "local",
  target_libc = "local",
  abi_version = "local",
  abi_libc_version = "local",
  cxx_builtin_include_directories = ["/opt/rh/devtoolset-4/root/usr/lib/gcc/x86_64-redhat-linux/5.3.1/include", "/usr/local/include", "/opt/rh/devtoolset-4/root/usr/include", "/usr/include", "/opt/rh/devtoolset-4/root/usr/include/c++/5.3.1", "/opt/rh/devtoolset-4/root/usr/include/c++/5.3.1/x86_64-redhat-linux", "/opt/rh/devtoolset-4/root/usr/include/c++/5.3.1/backward"],
  tool_paths = {"ar": "/usr/bin/ar", "ld": "/usr/bin/ld", "cpp": "/opt/rh/devtoolset-4/root/usr/bin/cpp", "gcc": "/opt/rh/devtoolset-4/root/usr/bin/gcc", "dwp": "/usr/bin/dwp", "gcov": "/opt/rh/devtoolset-4/root/usr/bin/gcov", "nm": "/usr/bin/nm", "objcopy": "/usr/bin/objcopy", "objdump": "/usr/bin/objdump", "strip": "/usr/bin/strip"},
  compile_flags = ["-U_FORTIFY_SOURCE", "-fstack-protector", "-Wall", "-Wunused-but-set-parameter", "-Wno-free-nonheap-object", "-fno-omit-frame-pointer"],
  dbg_compile_flags = ["-g"],
  opt_compile_flags = ["-g0", "-O2", "-D_FORTIFY_SOURCE=1", "-DNDEBUG", "-ffunction-sections", "-fdata-sections"],
  cxx_flags = ["-std=c++0x"],
  link_flags = ["-fuse-ld=gold", "-Wl,-no-as-needed", "-Wl,-z,relro,-z,now", "-B/opt/rh/devtoolset-4/root/usr/bin", "-pass-exit-codes", "-lstdc++", "-lm"],
  link_libs = [],
  opt_link_flags = ["-Wl,--gc-sections"],
  unfiltered_compile_flags = ["-fno-canonical-system-headers", "-Wno-builtin-macro-redefined", "-D__DATE__=\"redacted\"", "-D__TIMESTAMP__=\"redacted\"", "-D__TIME__=\"redacted\""],
  coverage_compile_flags = ["--coverage"],
  coverage_link_flags = ["--coverage"],
  supports_start_end_lib = True,
)

And I assume given the name that link_flags somehow makes it into the resulting link command.

[alex-xnor]

I can confirm that using @ali5h's gcc8 directory and --crosstool_top=//gcc8:toolchain works, after adjusting some of the paths in cc_toolchain_config.bzl (devtoolset-8 ⇒ devtoolset-4, 8 ⇒ 5.3.1).

[authurguo]

Maybe configurated-less bazel (1.2.1) could not works with centos scl utils. Install scl git tool rh-git218-git-all on centos 7, it break when running git clone by bazel.

[rodgarrison]

Re: bazel 1.2.0 build from dist source
Thank goodness I found this issue and read Alex's identification to bazel/tools/cpp/unix_cc_configure.bzl. I was able to build bazel from dist after running into this exact same issue by changing bazel/tools/cpp/unix_cc_configure.bzl line 379 from the empty string "" to "-lstdc++". Look for the assignment link_libs =. Hopefully this is the end of this C++ toolchain nonsense. I'll be using Bazel with Golang (Golang is trivial to install) which avoids all of C/C++ complexities.

[oquenchil]

This change https://github.com/bazelbuild/bazel/commit/906f5f1c233ca18c67a9cedd2c7182f33b88c44b introduces the fix so that the workaround is no longer needed.

The tracking issue for the flag flip is in https://github.com/bazelbuild/bazel/issues/10905

[dechaoliu]

This change 906f5f1 introduces the fix so that the workaround is no longer needed.

The tracking issue for the flag flip is in #10905

Hi, oquenchil

I'm sorry that even I use bazel 3.1.0, this error still exist.

bazel 3.1.0, protobuf 3.8.0

error: undefined reference to '__cxa_throw_bad_array_new_length'

Tis error occurred only when I used centos devtoolset-7:

/opt/rh/devtoolset-7/root/usr/bin/gcc @bazel-out/host/bin/external/com_google_protobuf/protoc-2.params
error: undefined reference to '__cxa_throw_bad_array_new_length'

/opt/rh/devtoolset-7/root/usr/bin/gcc  -lstdc++ @bazel-out/host/bin/external/com_google_protobuf/protoc-2.params
still error: undefined reference to '__cxa_throw_bad_array_new_length'

If I change '/opt/rh/devtoolset-7/root/usr/bin/gcc' to '/opt/rh/devtoolset-7/root/usr/bin/g++', this error will not exist.

If I install gcc7 by source code, this error will not exist, I can successfully compile protobuf by bazel.

protoc-2.params

-o
bazel-out/host/bin/external/com_google_protobuf/protoc
-Wl,-S
-fuse-ld=gold
-Wl,-no-as-needed
-Wl,-z,relro,-z,now
-B/opt/rh/devtoolset-7/root/usr/bin
-pass-exit-codes
-lstdc++
-lm
-Wl,--gc-sections
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc/main.o
-Wl,--start-lib
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/code_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/command_line_interface.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_enum.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_enum_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_extension.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_file.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_helpers.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_map_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_message.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_message_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_padding_optimizer.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_primitive_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_service.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/cpp_string_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_doc_comment.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_enum.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_enum_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_field_base.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_helpers.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_map_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_message.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_message_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_primitive_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_reflection_class.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_repeated_enum_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_repeated_message_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_repeated_primitive_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_source_generator_base.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/csharp_wrapper_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_context.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_doc_comment.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_enum.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_enum_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_enum_field_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_enum_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_extension.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_extension_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_file.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_generator_factory.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_helpers.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_map_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_map_field_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_message.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_message_builder.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_message_builder_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_message_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_message_field_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_message_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_name_resolver.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_primitive_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_primitive_field_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_service.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_shared_code_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_string_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_string_field_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/js_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/well_known_types_embed.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_enum.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_enum_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_extension.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_file.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_helpers.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_map_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_message.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_message_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_oneof.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_primitive_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/php_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/plugin.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/plugin.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/python_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/ruby_generator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/subprocess.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/zip_writer.o
-Wl,--end-lib
-Wl,--start-lib
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/any.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/any.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/api.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/importer.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/parser.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/descriptor.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/descriptor.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/descriptor_database.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/duration.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/dynamic_message.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/empty.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/extension_set_heavy.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/field_mask.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/generated_message_reflection.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/generated_message_table_driven.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/gzip_stream.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/printer.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/tokenizer.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/zero_copy_stream_impl.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/map_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/message.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/reflection_ops.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/service.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/source_context.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/struct.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/mathlimits.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/substitute.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/text_format.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/timestamp.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/type.pb.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/unknown_field_set.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/delimited_message_util.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/field_comparator.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/field_mask_util.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/datapiece.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/default_value_objectwriter.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/error_listener.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/field_mask_utility.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/json_escaping.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/json_objectwriter.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/json_stream_parser.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/object_writer.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/proto_writer.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/protostream_objectsource.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/protostream_objectwriter.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/type_info.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/type_info_test_helper.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/utility.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/json_util.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/message_differencer.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/time_util.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/type_resolver_util.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/wire_format.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/wrappers.pb.o
-Wl,--end-lib
-Wl,--start-lib
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/any_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/arena.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/extension_set.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/generated_message_table_driven_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/generated_message_util.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/implicit_weak_message.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/coded_stream.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/strtod.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/io_win32.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/zero_copy_stream.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/zero_copy_stream_impl_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/message_lite.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/parse_context.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/repeated_field.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/bytestream.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/common.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/int128.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/status.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/statusor.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/stringpiece.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/stringprintf.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/structurally_valid.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/strutil.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/time.o
bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/wire_format_lite.o
-Wl,--end-lib
-Wl,--start-lib
bazel-out/host/bin/external/zlib/_objs/zlib/adler32.o
bazel-out/host/bin/external/zlib/_objs/zlib/compress.o
bazel-out/host/bin/external/zlib/_objs/zlib/crc32.o
bazel-out/host/bin/external/zlib/_objs/zlib/deflate.o
bazel-out/host/bin/external/zlib/_objs/zlib/gzclose.o
bazel-out/host/bin/external/zlib/_objs/zlib/gzlib.o
bazel-out/host/bin/external/zlib/_objs/zlib/gzread.o
bazel-out/host/bin/external/zlib/_objs/zlib/gzwrite.o
bazel-out/host/bin/external/zlib/_objs/zlib/infback.o
bazel-out/host/bin/external/zlib/_objs/zlib/inffast.o
bazel-out/host/bin/external/zlib/_objs/zlib/inflate.o
bazel-out/host/bin/external/zlib/_objs/zlib/inftrees.o
bazel-out/host/bin/external/zlib/_objs/zlib/trees.o
bazel-out/host/bin/external/zlib/_objs/zlib/uncompr.o
bazel-out/host/bin/external/zlib/_objs/zlib/zutil.o
-Wl,--end-lib
-lpthread
-lm

[oquenchil]

Are you using the incompatible flag?

[kste]

I can confirm the same issue. Using the Docker image quay.io/pypa/manylinux2014_x86_64, with Bazel 3.1.0 and running:

git clone https://github.com/protocolbuffers/protobuf.git
bazel build //:protoc

[oquenchil]

Please use --incompatible_linkopts_to_linklibs.