~/tensorflowNoLinux Ubuntu 18.0.43.6/3.77.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)source2.10.29.1V10.1.243/7.6.1GeForce GTX 1080 TX 12Gbbazel build --linkopt='-lrt' --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-mavx --copt=-msse4.1 --copt=-msse4.2 --config=noaws --config=nohdfs --config=nonccl --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --config=cuda -k //tensorflow/tools/pip_package:build_pip_package --verbose_failures 2>&1 | tee bazel.log
Failed to build from source TF2.1 with CUDA (though able to builf TF2.0 without a problem)
Config (all default options except enabling CUDA):
(base) sergey@sergey-Bionic:~/tensorflow$ ./configure
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.29.1 installed.
Please specify the location of python. [Default is /home/sergey/anaconda3/bin/python]:
Found possible Python library paths:
/home/sergey/anaconda3/lib/python3.6/site-packages
Please input the desired Python library path to use. Default is [/home/sergey/anaconda3/lib/python3.6/site-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]:
XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
Do you wish to build TensorFlow with TensorRT support? [y/N]:
No TensorRT support will be enabled for TensorFlow.
Found CUDA 10.1 in:
/usr/local/cuda/lib64
/usr/local/cuda/include
Found cuDNN 7 in:
/usr/lib64
/usr/include
Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 6.1]:
Do you want to use clang as CUDA compiler? [y/N]:
nvcc will be used as CUDA compiler.
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.
bazel-out/host/bin/tensorflow/core/protobuf/tpu/libtpu_embedding_output_layout_proto_cc.lo(tpu_embedding_output_layout.pb.o):(.data.rel.ro._ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE[_ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE]+0x60): undefined reference to google::protobuf::Message::CheckTypeAndMergeFrom(google::protobuf::MessageLite const&)'
bazel-out/host/bin/tensorflow/core/protobuf/tpu/libtpu_embedding_output_layout_proto_cc.lo(tpu_embedding_output_layout.pb.o):(.data.rel.ro._ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE[_ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE]+0x80): undefined reference togoogle::protobuf::MessageLite::SerializeWithCachedSizesToArray(unsigned char*) const'
bazel-out/host/bin/tensorflow/core/protobuf/tpu/libtpu_embedding_output_layout_proto_cc.lo(tpu_embedding_output_layout.pb.o):(.data.rel.ro._ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE[_ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE]+0xb0): undefined reference to google::protobuf::Message::DiscardUnknownFields()'
bazel-out/host/bin/tensorflow/core/protobuf/tpu/libtpu_embedding_output_layout_proto_cc.lo(tpu_embedding_output_layout.pb.o):(.data.rel.ro._ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE[_ZTVN10tensorflow3tpu24TPUEmbeddingOutputLayoutE]+0xb8): undefined reference togoogle::protobuf::Message::SpaceUsedLong() const'
collect2: error: ld returned 1 exit status
bazel-out/host/bin/tensorflow/core/libcheckpoint_ops_op_lib.lo(checkpoint_ops.o): In function __static_initialization_and_destruction_0(int, int) [clone .constprop.45]':
checkpoint_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.45+0x19b): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
checkpoint_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.45+0x3ff): undefined reference to tensorflow::OpDefBuilder::SetIsStateful()'
checkpoint_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.45+0x424): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
collect2: error: ld returned 1 exit status
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x140): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x250): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x386): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x562): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
collect2: error: ld returned 1 exit status
@davidmochen David, how is the status of TF 2.1 rc build?
@bfontain Bruce the error is somehow related to TPUEmbedding?
@sbushmanov Could you also raise this issue in tensorflow/tensorflow repository?
We will also give update if we have.
Thanks
@saberkun Done.
https://github.com/tensorflow/tensorflow/issues/34191
TF 2.1 rc build from last night looks good:
https://cnsviewer2.corp.google.com/cns/tp-d/home/tpu-perf-team/tf-performance-reports/ttl=30d/snapshots/report_1573579248014/regression_r2.1.html
On 11/12/19, sbushmanov notifications@github.com wrote:
@saberkun Done.
https://github.com/tensorflow/tensorflow/issues/34191--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/tensorflow/models/issues/7793#issuecomment-552798794
On 11/12/19, David Chen dmchen@google.com wrote:
TF 2.1 rc build from last night looks good:
https://cnsviewer2.corp.google.com/cns/tp-d/home/tpu-perf-team/tf-performance-reports/ttl=30d/snapshots/report_1573579248014/regression_r2.1.htmlOn 11/12/19, sbushmanov notifications@github.com wrote:
@saberkun Done.
https://github.com/tensorflow/tensorflow/issues/34191--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/tensorflow/models/issues/7793#issuecomment-552798794
Still the same build errors on newly downloaded branch r2.1
Example of errors:
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla37WhileLoopBackendConfig_KnownTripCountE[_ZTVN3xla37WhileLoopBackendConfig_KnownTripCountE]+0x80): undefined reference to google::protobuf::MessageLite::SerializeWithCachedSizesToArray(unsigned char*) const'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla37WhileLoopBackendConfig_KnownTripCountE[_ZTVN3xla37WhileLoopBackendConfig_KnownTripCountE]+0xb0): undefined reference togoogle::protobuf::Message::DiscardUnknownFields()'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla37WhileLoopBackendConfig_KnownTripCountE[_ZTVN3xla37WhileLoopBackendConfig_KnownTripCountE]+0xb8): undefined reference to google::protobuf::Message::SpaceUsedLong() const'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla22WhileLoopBackendConfigE[_ZTVN3xla22WhileLoopBackendConfigE]+0x20): undefined reference togoogle::protobuf::Message::GetTypeNameabi:cxx11 const'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla22WhileLoopBackendConfigE[_ZTVN3xla22WhileLoopBackendConfigE]+0x58): undefined reference to google::protobuf::Message::InitializationErrorString[abi:cxx11]() const'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla22WhileLoopBackendConfigE[_ZTVN3xla22WhileLoopBackendConfigE]+0x60): undefined reference togoogle::protobuf::Message::CheckTypeAndMergeFrom(google::protobuf::MessageLite const&)'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla22WhileLoopBackendConfigE[_ZTVN3xla22WhileLoopBackendConfigE]+0x80): undefined reference to google::protobuf::MessageLite::SerializeWithCachedSizesToArray(unsigned char*) const'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla22WhileLoopBackendConfigE[_ZTVN3xla22WhileLoopBackendConfigE]+0xb0): undefined reference togoogle::protobuf::Message::DiscardUnknownFields()'
bazel-out/host/bin/tensorflow/compiler/xla/libxla_data_proto_cc.lo(xla_data.pb.o):(.data.rel.ro._ZTVN3xla22WhileLoopBackendConfigE[_ZTVN3xla22WhileLoopBackendConfigE]+0xb8): undefined reference to `google::protobuf::Message::SpaceUsedLong() const'
collect2: error: ld returned 1 exit status
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x2f5b): undefined reference to tensorflow::OpDefBuilder::SetIsStateful()'
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x2f80): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x3339): undefined reference to tensorflow::OpDefBuilder::SetIsStateful()'
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x335e): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x3636): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x3882): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x3a9e): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x3c9e): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
bazel-out/host/bin/tensorflow/core/libimage_ops_op_lib.lo(image_ops.o):image_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.128+0x3e3e): more undefined references to `tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
collect2: error: ld returned 1 exit status
audio_ops.cc:(.text._ZN10tensorflow12_GLOBAL__N_118SpectrogramShapeFnEPNS_15shape_inference16InferenceContextE+0xab): undefined reference to tensorflow::GetNodeAttr(tensorflow::AttrSlice const&, absl::string_view, int*)'
audio_ops.cc:(.text._ZN10tensorflow12_GLOBAL__N_118SpectrogramShapeFnEPNS_15shape_inference16InferenceContextE+0x1c8): undefined reference totensorflow::shape_inference::InferenceContext::MakeShape(std::initializer_list
bazel-out/host/bin/tensorflow/core/libaudio_ops_op_lib.lo(audio_ops.o): In function tensorflow::OpDefBuilder::~OpDefBuilder()':
audio_ops.cc:(.text._ZN10tensorflow12OpDefBuilderD2Ev[_ZN10tensorflow12OpDefBuilderD5Ev]+0x1c4): undefined reference totensorflow::OpDef::~OpDef()'
bazel-out/host/bin/tensorflow/core/libaudio_ops_op_lib.lo(audio_ops.o): In function __static_initialization_and_destruction_0(int, int) [clone .constprop.63]':
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x140): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x250): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x386): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
audio_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.63+0x562): undefined reference to `tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
collect2: error: ld returned 1 exit status
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x12e4): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x14cd): undefined reference totensorflow::shape_inference::UnknownShape(tensorflow::shape_inference::InferenceContext)'
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x14e4): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x16fd): undefined reference totensorflow::shape_inference::UnknownShape(tensorflow::shape_inference::InferenceContext)'
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x1714): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x1951): undefined reference totensorflow::OpDefBuilder::SetIsStateful()'
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x195f): undefined reference to tensorflow::shape_inference::UnknownShape(tensorflow::shape_inference::InferenceContext*)'
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x1976): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
functional_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.70+0x1ac7): undefined reference to `tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
collect2: error: ld returned 1 exit status
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x19c0): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x1aad): undefined reference totensorflow::shape_inference::UnchangedShape(tensorflow::shape_inference::InferenceContext*)'
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x1ac4): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x1c46): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x1e2f): undefined reference to tensorflow::shape_inference::UnchangedShape(tensorflow::shape_inference::InferenceContext*)'
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x1e46): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x2076): undefined reference to tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper<true> const&)'
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x22fc): undefined reference totensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
string_ops.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.145+0x2608): undefined reference to `tensorflow::register_op::OpDefBuilderReceiver::OpDefBuilderReceiver(tensorflow::register_op::OpDefBuilderWrapper
collect2: error: ld returned 1 exit status
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x314): undefined reference to tensorflow::OpDef::~OpDef()'
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x323): undefined reference totensorflow::KernelDefBuilder::KernelDefBuilder(char const)'
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x32a): undefined reference to tensorflow::DEVICE_CPU'
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x332): undefined reference totensorflow::KernelDefBuilder::Device(char const)'
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x33a): undefined reference to tensorflow::KernelDefBuilder::Build()'
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x353): undefined reference tovtable for tensorflow::kernel_factory::OpKernelRegistrar::PtrOpKernelFactory'
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x38a): undefined reference to tensorflow::kernel_factory::OpKernelRegistrar::InitInternal(tensorflow::KernelDef const*, absl::string_view, std::unique_ptr<tensorflow::kernel_factory::OpKernelFactory, std::default_delete<tensorflow::kernel_factory::OpKernelFactory> >)'
fact.cc:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.41+0x3a4): undefined reference totensorflow::KernelDefBuilder::~KernelDefBuilder()'
bazel-out/host/bin/tensorflow/core/libuser_ops_op_lib.lo(fact.o):(.data.rel.ro._ZTI6FactOp[_ZTI6FactOp]+0x10): undefined reference to `typeinfo for tensorflow::OpKernel'
collect2: error: ld returned 1 exit status
some one can success build?