Serving: Not able to build tensorflow serving on ubuntu 18.04 with cuda 10.0, cudnn7.3.1, tensorrt 5.0.2

Created on 19 Nov 2018 · 8Comments · Source: tensorflow/serving

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
TensorFlow Serving installed from (source or binary): Source
TensorFlow Serving version: latest

Describe the problem

Not able to build tensorflow serving on ubuntu 18.04 with cuda 10.0, cudnn7.3.1, tensorrt 5.0.2

Exact Steps to Reproduce

export NCCL_VERSION=2.3.7
export CUDNN_VERSION=7.3.1
export TF_TENSORRT_VERSION=5.0.2
export BAZEL_VERSION=0.17.2
export CI_BUILD_PYTHON=python
export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH
export TF_NEED_CUDA=1
export TF_NEED_TENSORRT=1
export TENSORRT_INSTALL_PATH=/usr/lib/x86_64-linux-gnu
export TF_CUDA_COMPUTE_CAPABILITIES=6.1,6.1,6.1,6.1
export TF_CUDA_VERSION=10.0
export TF_CUDNN_VERSION=7
export TF_NCCL_VERSION=2
export NCCL_INSTALL_PATH=/usr/local/nccl2/lib
export NCCL_HDR_PATH=/usr/local/nccl2/include
export TMP="/tmp"
export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:${LD_LIBRARY_PATH}

sudo mkdir /usr/lib/x86_64-linux-gnu/include/ && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/include/cudnn.h /usr/lib/x86_64-linux-gnu/include/cudnn.h && \
  sudo ln -s /usr/include/cudnn.h /usr/local/cuda/include/cudnn.h && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libcudnn.so /usr/local/cuda/lib64/libcudnn.so && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libcudnn.so.${TF_CUDNN_VERSION} /usr/local/cuda/lib64/libcudnn.so.${TF_CUDNN_VERSION}

sudo mkdir -p ${NCCL_INSTALL_PATH} && \
  sudo mkdir -p ${NCCL_HDR_PATH} && \
  sudo ln -s /usr/include/nccl.h ${NCCL_HDR_PATH}/nccl.h && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libnccl.so ${NCCL_INSTALL_PATH}/libnccl.so && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libnccl.so.${TF_NCCL_VERSION} ${NCCL_INSTALL_PATH}/libnccl.so.${TF_NCCL_VERSION


sudo ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/libcuda.so.1 && \
    LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:${LD_LIBRARY_PATH} \
    bazel build --color=yes --curses=yes --config=cuda\
    --verbose_failures \
    --output_filter=DONT_MATCH_ANYTHING \
    tensorflow_serving/model_servers:tensorflow_model_server

Source code / logs

INFO: Analysed target //tensorflow_serving/model_servers:tensorflow_model_server (0 packages loaded).
INFO: Found 1 target...
ERROR: /home/mia/.cache/bazel/_bazel_mia/92738da4087940669a9536fcf791c3ca/external/org_tensorflow/tensorflow/contrib/tensorrt/BUILD:263:1: C++ compilation of rule '@org_tensorflow//tensorflow/contrib/tensorrt:trt_conversion' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/mia/.cache/bazel/_bazel_mia/92738da4087940669a9536fcf791c3ca/execroot/tf_serving && \
  exec env - \
    LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/usr/local/cuda/lib64/stubs:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda-10.0/lib64 \
    PATH=/usr/local/cuda-10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/mia/bin:/home/mia/.fzf/bin \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/home/mia/tf-trt/bin/python \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/contrib/tensorrt/_objs/trt_conversion/convert_nodes.d '-frandom-seed=bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/contrib/tensorrt/_objs/trt_conversion/convert_nodes.o' -D__CLANG_SUPPORT_DYN_ANNOTATION__ -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' '-DEIGEN_HAS_TYPE_TRAITS=0' -DTF_USE_SNAPPY -iquote external/org_tensorflow -iquote bazel-out/k8-opt/genfiles/external/org_tensorflow -iquote bazel-out/k8-opt/bin/external/org_tensorflow -iquote external/com_google_absl -iquote bazel-out/k8-opt/genfiles/external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/bazel_tools -iquote bazel-out/k8-opt/genfiles/external/bazel_tools -iquote bazel-out/k8-opt/bin/external/bazel_tools -iquote external/eigen_archive -iquote bazel-out/k8-opt/genfiles/external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/k8-opt/genfiles/external/local_config_sycl -iquote bazel-out/k8-opt/bin/external/local_config_sycl -iquote external/nsync -iquote bazel-out/k8-opt/genfiles/external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/gif_archive -iquote bazel-out/k8-opt/genfiles/external/gif_archive -iquote bazel-out/k8-opt/bin/external/gif_archive -iquote external/jpeg -iquote bazel-out/k8-opt/genfiles/external/jpeg -iquote bazel-out/k8-opt/bin/external/jpeg -iquote external/protobuf_archive -iquote bazel-out/k8-opt/genfiles/external/protobuf_archive -iquote bazel-out/k8-opt/bin/external/protobuf_archive -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/genfiles/external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/k8-opt/genfiles/external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/k8-opt/genfiles/external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/k8-opt/genfiles/external/highwayhash -iquote bazel-out/k8-opt/bin/external/highwayhash -iquote external/zlib_archive -iquote bazel-out/k8-opt/genfiles/external/zlib_archive -iquote bazel-out/k8-opt/bin/external/zlib_archive -iquote external/double_conversion -iquote bazel-out/k8-opt/genfiles/external/double_conversion -iquote bazel-out/k8-opt/bin/external/double_conversion -iquote external/snappy -iquote bazel-out/k8-opt/genfiles/external/snappy -iquote bazel-out/k8-opt/bin/external/snappy -iquote external/local_config_cuda -iquote bazel-out/k8-opt/genfiles/external/local_config_cuda -iquote bazel-out/k8-opt/bin/external/local_config_cuda -iquote external/local_config_tensorrt -iquote bazel-out/k8-opt/genfiles/external/local_config_tensorrt -iquote bazel-out/k8-opt/bin/external/local_config_tensorrt -isystem external/eigen_archive -isystem bazel-out/k8-opt/genfiles/external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/nsync/public -isystem bazel-out/k8-opt/genfiles/external/nsync/public -isystem bazel-out/k8-opt/bin/external/nsync/public -isystem external/gif_archive/lib -isystem bazel-out/k8-opt/genfiles/external/gif_archive/lib -isystem bazel-out/k8-opt/bin/external/gif_archive/lib -isystem external/protobuf_archive/src -isystem bazel-out/k8-opt/genfiles/external/protobuf_archive/src -isystem bazel-out/k8-opt/bin/external/protobuf_archive/src -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/genfiles/external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/zlib_archive -isystem bazel-out/k8-opt/genfiles/external/zlib_archive -isystem bazel-out/k8-opt/bin/external/zlib_archive -isystem external/double_conversion -isystem bazel-out/k8-opt/genfiles/external/double_conversion -isystem bazel-out/k8-opt/bin/external/double_conversion -isystem external/local_config_cuda/cuda -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda -isystem external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include -isystem external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include/crt -isystem external/local_config_tensorrt/include -isystem bazel-out/k8-opt/genfiles/external/local_config_tensorrt/include -isystem bazel-out/k8-opt/bin/external/local_config_tensorrt/include '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIE -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections '-D_GLIBCXX_USE_CXX11_ABI=0' -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare -fno-exceptions '-ftemplate-depth=900' '-DGOOGLE_CUDA=1' '-DGOOGLE_TENSORRT=1' -msse3 -DTENSORFLOW_MONOLITHIC_BUILD -pthread '-DGOOGLE_CUDA=1' '-DGOOGLE_TENSORRT=1' -c external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc -o bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/contrib/tensorrt/_objs/trt_conversion/convert_nodes.o)
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc: In member function 'virtual bool tensorflow::tensorrt::convert::TRT_TensorOrWeights::SimpleITensor::setDynamicRange(float, float)':
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:428:56: warning: no return statement in function returning non-void [-Wreturn-type]
   bool setDynamicRange(float min, float max) override {}
                                                        ^
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc: In constructor 'tensorflow::tensorrt::convert::TRT_TensorOrWeights::TRT_TensorOrWeights(nvinfer1::DataType, const nvinfer1::Dims&, int)':
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:446:60: error: invalid new-expression of abstract class type 'tensorflow::tensorrt::convert::TRT_TensorOrWeights::SimpleITensor'
     : simple_itensor_(new SimpleITensor(trt_dtype, trt_dims)),
                                                            ^
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:391:28: note:   because the following virtual functions are pure within 'tensorflow::tensorrt::convert::TRT_TensorOrWeights::SimpleITensor':
 class TRT_TensorOrWeights::SimpleITensor : public nvinfer1::ITensor {
                            ^~~~~~~~~~~~~
In file included from external/org_tensorflow/tensorflow/contrib/tensorrt/log/trt_logger.h:23:0,
                 from external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.h:26,
                 from external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:16:
bazel-out/k8-opt/genfiles/external/local_config_tensorrt/tensorrt/include/NvInfer.h:694:19: note:   virtual float nvinfer1::ITensor::getDynamicRange() const
     virtual float getDynamicRange() const = 0;
                   ^~~~~~~~~~~~~~~
Target //tensorflow_serving/model_servers:tensorflow_model_server failed to build
INFO: Elapsed time: 709.674s, Critical Path: 205.19s
INFO: 3943 processes: 3943 local.
FAILED: Build did NOT complete successfully

builinstall

Source

srihari-humbarwadi

Most helpful comment

Technically yes it does support TRT. We will be working on a blog post that will better describe how to do this easily.

gautamvasudevan on 10 Dec 2018

👍3

All 8 comments

Cuda 10.0 and TensorRT 5 are not currently supported with TensorFlow Serving. You're welcome to open a pull request with this feature working though! Thanks!

gautamvasudevan on 20 Nov 2018

@gautamvasudevan Does the tf serving now support Tensorrt for inference. Or Do I have to build it from the source code? Thanks

Chris19920210 on 10 Dec 2018

Technically yes it does support TRT. We will be working on a blog post that will better describe how to do this easily.

gautamvasudevan on 10 Dec 2018

👍3

@gautamvasudevan Thanks a lot. It will be really appreciated if you inform me when the blog gets done.

Chris19920210 on 11 Dec 2018

Are there any documents showing how tensroflow serving supports tensorRT now?

elvys-zhang on 25 Dec 2018

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
TensorFlow Serving installed from (source or binary): Source
TensorFlow Serving version: latest

Describe the problem

Not able to build tensorflow serving on ubuntu 18.04 with cuda 10.0, cudnn7.3.1, tensorrt 5.0.2

Exact Steps to Reproduce

export NCCL_VERSION=2.3.7
export CUDNN_VERSION=7.3.1
export TF_TENSORRT_VERSION=5.0.2
export BAZEL_VERSION=0.17.2
export CI_BUILD_PYTHON=python
export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH
export TF_NEED_CUDA=1
export TF_NEED_TENSORRT=1
export TENSORRT_INSTALL_PATH=/usr/lib/x86_64-linux-gnu
export TF_CUDA_COMPUTE_CAPABILITIES=6.1,6.1,6.1,6.1
export TF_CUDA_VERSION=10.0
export TF_CUDNN_VERSION=7
export TF_NCCL_VERSION=2
export NCCL_INSTALL_PATH=/usr/local/nccl2/lib
export NCCL_HDR_PATH=/usr/local/nccl2/include
export TMP="/tmp"
export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:${LD_LIBRARY_PATH}

sudo mkdir /usr/lib/x86_64-linux-gnu/include/ && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/include/cudnn.h /usr/lib/x86_64-linux-gnu/include/cudnn.h && \
  sudo ln -s /usr/include/cudnn.h /usr/local/cuda/include/cudnn.h && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libcudnn.so /usr/local/cuda/lib64/libcudnn.so && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libcudnn.so.${TF_CUDNN_VERSION} /usr/local/cuda/lib64/libcudnn.so.${TF_CUDNN_VERSION}

sudo mkdir -p ${NCCL_INSTALL_PATH} && \
  sudo mkdir -p ${NCCL_HDR_PATH} && \
  sudo ln -s /usr/include/nccl.h ${NCCL_HDR_PATH}/nccl.h && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libnccl.so ${NCCL_INSTALL_PATH}/libnccl.so && \
  sudo ln -s /usr/lib/x86_64-linux-gnu/libnccl.so.${TF_NCCL_VERSION} ${NCCL_INSTALL_PATH}/libnccl.so.${TF_NCCL_VERSION


sudo ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/libcuda.so.1 && \
    LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:${LD_LIBRARY_PATH} \
    bazel build --color=yes --curses=yes --config=cuda\
    --verbose_failures \
    --output_filter=DONT_MATCH_ANYTHING \
    tensorflow_serving/model_servers:tensorflow_model_server

Source code / logs

INFO: Analysed target //tensorflow_serving/model_servers:tensorflow_model_server (0 packages loaded).
INFO: Found 1 target...
ERROR: /home/mia/.cache/bazel/_bazel_mia/92738da4087940669a9536fcf791c3ca/external/org_tensorflow/tensorflow/contrib/tensorrt/BUILD:263:1: C++ compilation of rule '@org_tensorflow//tensorflow/contrib/tensorrt:trt_conversion' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/mia/.cache/bazel/_bazel_mia/92738da4087940669a9536fcf791c3ca/execroot/tf_serving && \
  exec env - \
    LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/usr/local/cuda/lib64/stubs:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda-10.0/lib64 \
    PATH=/usr/local/cuda-10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/mia/bin:/home/mia/.fzf/bin \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/home/mia/tf-trt/bin/python \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/contrib/tensorrt/_objs/trt_conversion/convert_nodes.d '-frandom-seed=bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/contrib/tensorrt/_objs/trt_conversion/convert_nodes.o' -D__CLANG_SUPPORT_DYN_ANNOTATION__ -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' '-DEIGEN_HAS_TYPE_TRAITS=0' -DTF_USE_SNAPPY -iquote external/org_tensorflow -iquote bazel-out/k8-opt/genfiles/external/org_tensorflow -iquote bazel-out/k8-opt/bin/external/org_tensorflow -iquote external/com_google_absl -iquote bazel-out/k8-opt/genfiles/external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/bazel_tools -iquote bazel-out/k8-opt/genfiles/external/bazel_tools -iquote bazel-out/k8-opt/bin/external/bazel_tools -iquote external/eigen_archive -iquote bazel-out/k8-opt/genfiles/external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/k8-opt/genfiles/external/local_config_sycl -iquote bazel-out/k8-opt/bin/external/local_config_sycl -iquote external/nsync -iquote bazel-out/k8-opt/genfiles/external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/gif_archive -iquote bazel-out/k8-opt/genfiles/external/gif_archive -iquote bazel-out/k8-opt/bin/external/gif_archive -iquote external/jpeg -iquote bazel-out/k8-opt/genfiles/external/jpeg -iquote bazel-out/k8-opt/bin/external/jpeg -iquote external/protobuf_archive -iquote bazel-out/k8-opt/genfiles/external/protobuf_archive -iquote bazel-out/k8-opt/bin/external/protobuf_archive -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/genfiles/external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/k8-opt/genfiles/external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/k8-opt/genfiles/external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/k8-opt/genfiles/external/highwayhash -iquote bazel-out/k8-opt/bin/external/highwayhash -iquote external/zlib_archive -iquote bazel-out/k8-opt/genfiles/external/zlib_archive -iquote bazel-out/k8-opt/bin/external/zlib_archive -iquote external/double_conversion -iquote bazel-out/k8-opt/genfiles/external/double_conversion -iquote bazel-out/k8-opt/bin/external/double_conversion -iquote external/snappy -iquote bazel-out/k8-opt/genfiles/external/snappy -iquote bazel-out/k8-opt/bin/external/snappy -iquote external/local_config_cuda -iquote bazel-out/k8-opt/genfiles/external/local_config_cuda -iquote bazel-out/k8-opt/bin/external/local_config_cuda -iquote external/local_config_tensorrt -iquote bazel-out/k8-opt/genfiles/external/local_config_tensorrt -iquote bazel-out/k8-opt/bin/external/local_config_tensorrt -isystem external/eigen_archive -isystem bazel-out/k8-opt/genfiles/external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/nsync/public -isystem bazel-out/k8-opt/genfiles/external/nsync/public -isystem bazel-out/k8-opt/bin/external/nsync/public -isystem external/gif_archive/lib -isystem bazel-out/k8-opt/genfiles/external/gif_archive/lib -isystem bazel-out/k8-opt/bin/external/gif_archive/lib -isystem external/protobuf_archive/src -isystem bazel-out/k8-opt/genfiles/external/protobuf_archive/src -isystem bazel-out/k8-opt/bin/external/protobuf_archive/src -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/genfiles/external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/zlib_archive -isystem bazel-out/k8-opt/genfiles/external/zlib_archive -isystem bazel-out/k8-opt/bin/external/zlib_archive -isystem external/double_conversion -isystem bazel-out/k8-opt/genfiles/external/double_conversion -isystem bazel-out/k8-opt/bin/external/double_conversion -isystem external/local_config_cuda/cuda -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda -isystem external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include -isystem external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include/crt -isystem external/local_config_tensorrt/include -isystem bazel-out/k8-opt/genfiles/external/local_config_tensorrt/include -isystem bazel-out/k8-opt/bin/external/local_config_tensorrt/include '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIE -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections '-D_GLIBCXX_USE_CXX11_ABI=0' -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare -fno-exceptions '-ftemplate-depth=900' '-DGOOGLE_CUDA=1' '-DGOOGLE_TENSORRT=1' -msse3 -DTENSORFLOW_MONOLITHIC_BUILD -pthread '-DGOOGLE_CUDA=1' '-DGOOGLE_TENSORRT=1' -c external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc -o bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/contrib/tensorrt/_objs/trt_conversion/convert_nodes.o)
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc: In member function 'virtual bool tensorflow::tensorrt::convert::TRT_TensorOrWeights::SimpleITensor::setDynamicRange(float, float)':
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:428:56: warning: no return statement in function returning non-void [-Wreturn-type]
   bool setDynamicRange(float min, float max) override {}
                                                        ^
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc: In constructor 'tensorflow::tensorrt::convert::TRT_TensorOrWeights::TRT_TensorOrWeights(nvinfer1::DataType, const nvinfer1::Dims&, int)':
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:446:60: error: invalid new-expression of abstract class type 'tensorflow::tensorrt::convert::TRT_TensorOrWeights::SimpleITensor'
     : simple_itensor_(new SimpleITensor(trt_dtype, trt_dims)),
                                                            ^
external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:391:28: note:   because the following virtual functions are pure within 'tensorflow::tensorrt::convert::TRT_TensorOrWeights::SimpleITensor':
 class TRT_TensorOrWeights::SimpleITensor : public nvinfer1::ITensor {
                            ^~~~~~~~~~~~~
In file included from external/org_tensorflow/tensorflow/contrib/tensorrt/log/trt_logger.h:23:0,
                 from external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.h:26,
                 from external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc:16:
bazel-out/k8-opt/genfiles/external/local_config_tensorrt/tensorrt/include/NvInfer.h:694:19: note:     virtual float nvinfer1::ITensor::getDynamicRange() const
     virtual float getDynamicRange() const = 0;
                   ^~~~~~~~~~~~~~~
Target //tensorflow_serving/model_servers:tensorflow_model_server failed to build
INFO: Elapsed time: 709.674s, Critical Path: 205.19s
INFO: 3943 processes: 3943 local.
FAILED: Build did NOT complete successfully

same question
ubuntu 18.04 with cuda 10.0, cudnn7.3.1, tensorrt 5.0.2

berli on 4 Jan 2019

I find a way to work around.

The reason is the class "nvinfer1::ITensor", which is a pure virtual class, added some new interfaces in newer tensorrt version like(5.0.2 or 5.1.5), but in tensorflow(1.13.1) class "TRT_TensorOrWeights::SimpleITensor" does not implement these interfaces, which means can not "new SimpleITensor".

The work around method is pretty simple:
(1) Find your compileing source code
Usually it is /external/org_tensorflow/tensorflow/contrib/tensorrt/convert/convert_nodes.cc

(2) Implement the missing interfaces in class TRT_TensorOrWeights::SimpleITensor
bool dynamicRangeIsSet() const override { return true; }
void resetDynamicRange() override {}
float getDynamicRangeMin() const override { return 0.f; }
float getDynamicRangeMax() const override { return 0.f; }

Then rebuild and everything works fine.

dcmrlee on 12 Dec 2019

👍1

when comes to TensorRT6.0.1
in xxxxxx/tensorflow/compiler/tf2tensorrt/convert/convert_nodes.cc,
the folowing interface should be implemented.

bool isShapeTensor() const override { return false; }
bool isExecutionTensor() const override {return false; }

notice tensorflow 1.14 master 00fad90125b18b80fe054de1055770cfb8fe4ba3 do not fix this issue.

CLIsVeryOK on 13 Feb 2020

👍1

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Tensorflow Serving docker failed compilation

demiladef · 4Comments

Op type not registered 'ClipByValue' in binary running on 229d61c80ffd

cchung100m · 4Comments

inception-client error with tensorflow-serving-apis, but works well with bazel built server

TonyChouZJU · 4Comments

tensorflow_model_server ignores visible gpu device

vikeshkhanna · 3Comments

Serve Multiple Versions simultaneously Error

brianschardt · 3Comments