Onnx-tensorrt: Could NOT find TENSORRT (missing: TENSORRT_LIBRARY)

[omeralierdemir]

Hi, I try convert onnx model to tensortRT C++ API but I couldn't.
My system:
I have a jetson tx2, tensorRT6 (and tensorRT 5.1.6 on different tx2)
I tried to this commend cmake . -DCUDA_INCLUDE_DIRS=/usr/local/cuda/include -DTENSORRT_ROOT=/usr/src/tensorrt -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc in onnx_tensorRT directory. I got this output:

`
CMake Error at CMakeLists.txt:100 (add_subdirectory):
The source directory

/home/omer/onnx-tensorrt/third_party/onnx

does not contain a CMakeLists.txt file.

-- Found TensorRT headers at /usr/include/aarch64-linux-gnu
-- Find TensorRT libs at /usr/lib/aarch64-linux-gnu/libnvinfer.so;/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so;TENSORRT_LIBRARY_MYELIN-NOTFOUND
-- Could NOT find TENSORRT (missing: TENSORRT_LIBRARY)
ERRORCannot find TensorRT library.
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
TENSORRT_LIBRARY_MYELIN
linked by target "nvonnxparser_static" in directory /home/omer/onnx-tensorrt
linked by target "nvonnxparser" in directory /home/omer/onnx-tensorrt

-- Configuring incomplete, errors occurred

`
I have few quastions. is this repo supported tensorRT 6.0 or 5.1.6 version and tx2? because : https://devtalk.nvidia.com/default/topic/1069634/how-to-upgrade-tensorrt-to-7-0-version/?offset=10 How can convert onnx model to tensorRT C++ API ?

[rmccorm4]

Hi @omeralierdemir,

Please set them or make sure they are set and tested correctly in the CMake files:
TENSORRT_LIBRARY_MYELIN

Myelin was introduced in TensorRT 7, which implies that you're building from either the 7.0 branch or from master. Please try building from the 6.0 branch for TensorRT 6.0.

See: https://github.com/onnx/onnx-tensorrt/issues/355#issuecomment-568831810

[omeralierdemir]

Hi @rmccorm4
Firstly, thanks for answering my questions quickly. I did what your said and I cloned 6.0 branch. But I couldn't build. I got this output.
git clone -b 6.0 --single-branch https://github.com/onnx/onnx-tensorrt.git
mkdir build && cd build &&cmake .. -DTENSORRT_ROOT=/usr/src/tensorrt

`
-- The CXX compiler identification is GNU 7.4.0
-- The C compiler identification is GNU 7.4.0
-- The CUDA compiler identification is NVIDIA 10.0.326
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CUDA compiler: /usr/local/cuda-10.0/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda-10.0/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Found Protobuf: /usr/local/lib/libprotobuf.so;-lpthread (found version "3.11.2")
CMake Error at CMakeLists.txt:121 (add_subdirectory):
The source directory

/home/omer/Desktop/onnx-tensorrt/third_party/onnx

does not contain a CMakeLists.txt file.

-- GPU_ARCH is not defined. Generating CUDA code for default SMs.
-- Found CUDNN: /usr/include
-- Found TensorRT headers at /usr/include/aarch64-linux-gnu
-- Find TensorRT libs at /usr/lib/aarch64-linux-gnu/libnvinfer.so;/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so
-- Found TENSORRT: /usr/include/aarch64-linux-gnu
-- Configuring incomplete, errors occurred!
See also "/home/omer/Desktop/onnx-tensorrt/build/CMakeFiles/CMakeOutput.log".
See also "/home/omer/Desktop/onnx-tensorrt/build/CMakeFiles/CMakeError.log".

`

I tried this issue #354 but it didn't work

Also I tried these commands too but they didn't work

cmake .. -DTENSORRT_ROOT=/usr/src/tensorrt -DGPU_ARCHS="62"
cmake .. -DCUDA_INCLUDE_DIRS=/usr/local/cuda/include -DTENSORRT_ROOT=/usr/src/tensorrt -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc

[rmccorm4]

Hi @omeralierdemir,

I tried this issue #354 but it didn't work

What specifically did you try and what result did you get?

CMake Error at CMakeLists.txt:121 (add_subdirectory):
The source directory /home/omer/Desktop/onnx-tensorrt/third_party/onnx
does not contain a CMakeLists.txt file.

I agree that initializing the submodules should fix this error as per #354.

This worked for me just now on an x86 machine with a P4 GPU (CC 6.1) -- using nvcr.io/nvidia/tensorrt:19.12-py3 for reproducibility:

git clone -b 6.0 --single-branch https://github.com/onnx/onnx-tensorrt.git
cd onnx-tensorrt/
# Should resolve your "does not contain a CMakeLists.txt" error
git submodule update --init --recursive
# Skip this if you already have a valid protobuf dependency installed
apt update -y && apt install -y libprotobuf-dev protobuf-compiler
# Build
mkdir build && cd build
cmake .. -DTENSORRT_ROOT=/usr/src/tensorrt -DCUDA_INCLUDE_DIRS=/usr/local/cuda/include

cmake result:

root@8913d12915f0:/workspace/onnx-tensorrt# mkdir build && cd build
root@8913d12915f0:/workspace/onnx-tensorrt# cmake .. -DTENSORRT_ROOT=/usr/src/tensorrt -DCUDA_INCLUDE_DIRS=/usr/local/cuda/include
-- The CXX compiler identification is GNU 7.4.0
-- The C compiler identification is GNU 7.4.0
-- The CUDA compiler identification is NVIDIA 10.2.89
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread (found version "3.0.0")
-- Build type not set - defaulting to Release
--
-- ******** Summary ********
--   CMake version         : 3.14.4
--   CMake command         : /usr/local/bin/cmake
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler version  : 7.4.0
--   CXX flags             :  -Wall -Wno-deprecated-declarations -Wno-unused-function -Wno-unused-but-set-variable -Wnon-virtual-dtor
--   Build type            : Release
--   Compile definitions   : ONNX_NAMESPACE=onnx2trt_onnx
--   CMAKE_PREFIX_PATH     :
--   CMAKE_INSTALL_PREFIX  : /usr/local
--   CMAKE_MODULE_PATH     :
--
--   ONNX version          : 1.3.0
--   ONNX NAMESPACE        : onnx2trt_onnx
--   ONNX_BUILD_TESTS      : OFF
--   ONNX_BUILD_BENCHMARKS : OFF
--   ONNX_USE_LITE_PROTO   : OFF
--   ONNXIFI_DUMMY_BACKEND : OFF
--
--   Protobuf compiler     : /usr/bin/protoc
--   Protobuf includes     : /usr/include
--   Protobuf libraries    : /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread
--   BUILD_ONNX_PYTHON     : OFF
-- GPU_ARCH is not defined. Generating CUDA code for default SMs.
-- Found CUDNN: /usr/include
-- Found TensorRT headers at /usr/include/x86_64-linux-gnu
-- Find TensorRT libs at /usr/lib/x86_64-linux-gnu/libnvinfer.so;/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so
-- Found TENSORRT: /usr/include/x86_64-linux-gnu
-- Configuring done
-- Generating done
-- Build files have been written to: /workspace/onnx-tensorrt/build

make result:

root@8913d12915f0:/workspace/onnx-tensorrt/build# make -j8
Scanning dependencies of target gen_onnx_proto
Scanning dependencies of target nvonnxparser_plugin
[  2%] Running gen_proto.py on onnx/onnx.in.proto
Processing /workspace/onnx-tensorrt/third_party/onnx/onnx/onnx.in.proto
Writing /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx_onnx2trt_onnx.proto
Writing /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx_onnx2trt_onnx.proto3
Writing /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx.pb.h
generating /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx_pb.py
[  4%] Running C++ protocol buffer compiler on /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx_onnx2trt_onnx.proto
[  4%] Built target gen_onnx_proto
[  6%] Running gen_proto.py on onnx/onnx-operators.in.proto
Processing /workspace/onnx-tensorrt/third_party/onnx/onnx/onnx-operators.in.proto
Writing /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx-operators_onnx2trt_onnx.proto
Writing /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx-operators_onnx2trt_onnx.proto3
Writing /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx-operators.pb.h
generating /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx_operators_pb.py
[  8%] Running C++ protocol buffer compiler on /workspace/onnx-tensorrt/build/third_party/onnx/onnx/onnx-operators_onnx2trt_onnx.proto
Scanning dependencies of target onnx_proto
[ 10%] Building CUDA object CMakeFiles/nvonnxparser_plugin.dir/FancyActivation.cu.o
[ 12%] Building CXX object third_party/onnx/CMakeFiles/onnx_proto.dir/onnx/onnx_onnx2trt_onnx.pb.cc.o
[ 14%] Building CXX object third_party/onnx/CMakeFiles/onnx_proto.dir/onnx/onnx-operators_onnx2trt_onnx.pb.cc.o
[ 18%] Building CUDA object CMakeFiles/nvonnxparser_plugin.dir/ResizeNearest.cu.o
[ 18%] Building CXX object CMakeFiles/nvonnxparser_plugin.dir/plugin.cpp.o
[ 22%] Building CUDA object CMakeFiles/nvonnxparser_plugin.dir/Split.cu.o
[ 22%] Building CXX object CMakeFiles/nvonnxparser_plugin.dir/InstanceNormalization.cpp.o
[ 24%] Linking CXX static library libonnx_proto.a
[ 28%] Built target onnx_proto
[ 30%] Linking CXX static library libnvonnxparser_plugin.a
[ 30%] Built target nvonnxparser_plugin
Scanning dependencies of target nvonnxparser_static
Scanning dependencies of target nvonnxparser
Scanning dependencies of target nvonnxparser_runtime_static
Scanning dependencies of target nvonnxparser_runtime
[ 32%] Building CXX object CMakeFiles/nvonnxparser_runtime_static.dir/NvOnnxParserRuntime.cpp.o
[ 34%] Building CXX object CMakeFiles/nvonnxparser_runtime_static.dir/PluginFactory.cpp.o
[ 36%] Building CXX object CMakeFiles/nvonnxparser_static.dir/ModelImporter.cpp.o
[ 38%] Building CXX object CMakeFiles/nvonnxparser_static.dir/NvOnnxParser.cpp.o
[ 40%] Building CXX object CMakeFiles/nvonnxparser.dir/NvOnnxParser.cpp.o
[ 42%] Building CXX object CMakeFiles/nvonnxparser.dir/ModelImporter.cpp.o
[ 44%] Building CXX object CMakeFiles/nvonnxparser_static.dir/builtin_op_importers.cpp.o
[ 46%] Building CXX object CMakeFiles/nvonnxparser_runtime.dir/NvOnnxParserRuntime.cpp.o
[ 48%] Building CXX object CMakeFiles/nvonnxparser_runtime.dir/PluginFactory.cpp.o
[ 51%] Building CXX object CMakeFiles/nvonnxparser_static.dir/onnx2trt_utils.cpp.o
[ 53%] Building CXX object CMakeFiles/nvonnxparser_runtime_static.dir/builtin_plugins.cpp.o
[ 55%] Building CXX object CMakeFiles/nvonnxparser_runtime.dir/builtin_plugins.cpp.o
[ 57%] Building CXX object CMakeFiles/nvonnxparser.dir/builtin_op_importers.cpp.o
[ 59%] Building CXX object CMakeFiles/nvonnxparser_static.dir/ShapedWeights.cpp.o
[ 61%] Building CXX object CMakeFiles/nvonnxparser_static.dir/OnnxAttrs.cpp.o
[ 63%] Linking CXX static library libnvonnxparser_runtime_static.a
[ 63%] Built target nvonnxparser_runtime_static
[ 65%] Building CXX object CMakeFiles/nvonnxparser.dir/onnx2trt_utils.cpp.o
[ 67%] Building CXX object CMakeFiles/nvonnxparser.dir/ShapedWeights.cpp.o
[ 69%] Linking CUDA device code CMakeFiles/nvonnxparser_runtime.dir/cmake_device_link.o
[ 71%] Linking CXX shared library libnvonnxparser_runtime.so
[ 73%] Building CXX object CMakeFiles/nvonnxparser.dir/OnnxAttrs.cpp.o
[ 73%] Built target nvonnxparser_runtime
[ 75%] Linking CXX static library libnvonnxparser_static.a
[ 75%] Built target nvonnxparser_static
Scanning dependencies of target getSupportedAPITest
Scanning dependencies of target onnx2trt
Scanning dependencies of target trt_onnxify
[ 77%] Building CXX object CMakeFiles/onnx2trt.dir/main.cpp.o
[ 79%] Building CXX object CMakeFiles/getSupportedAPITest.dir/getSupportedAPITest.cpp.o
[ 81%] Building CXX object CMakeFiles/getSupportedAPITest.dir/ModelImporter.cpp.o
[ 83%] Building CXX object CMakeFiles/trt_onnxify.dir/onnx_trt_backend.cpp.o
[ 85%] Linking CUDA device code CMakeFiles/onnx2trt.dir/cmake_device_link.o
[ 87%] Linking CXX executable onnx2trt
[ 89%] Linking CUDA device code CMakeFiles/nvonnxparser.dir/cmake_device_link.o
[ 91%] Linking CXX shared library libnvonnxparser.so
[ 91%] Built target onnx2trt
[ 91%] Built target nvonnxparser
[ 93%] Linking CUDA device code CMakeFiles/trt_onnxify.dir/cmake_device_link.o
[ 95%] Linking CXX shared library libtrt_onnxify.so
[ 95%] Built target trt_onnxify
[ 97%] Linking CUDA device code CMakeFiles/getSupportedAPITest.dir/cmake_device_link.o
[100%] Linking CXX executable getSupportedAPITest
[100%] Built target getSupportedAPITest

onnx2trt:

root@8913d12915f0:/workspace/onnx-tensorrt/build# ./onnx2trt
ONNX to TensorRT model parser
Usage: onnx2trt onnx_model.pb
                [-o engine_file.trt]  (output TensorRT engine)
                [-t onnx_model.pbtxt] (output ONNX text file without weights)
                [-T onnx_model.pbtxt] (output ONNX text file with weights)
                [-b max_batch_size (default 32)]
                [-w max_workspace_size_bytes (default 1 GiB)]
                [-d model_data_type_bit_depth] (32 => float32, 16 => float16)
                [-l] (list layers and their shapes)
                [-g] (debug mode)
                [-v] (increase verbosity)
                [-q] (decrease verbosity)
                [-V] (show version information)
                [-h] (show help)

[omeralierdemir]

Hi @rmccorm4,
These steps worked. thank you very, very much. Now I have a .trt file. I searched README.md and last issues for sample test used with C++ api. But I couldn't find. This website (https://elinux.org/TensorRT/YoloV3) used this command for test:
./trtexec --onnx=yolov3.onnx --output=layer106-conv --int8 --batch=$BATCH --device=$DEVICE // BATCH = 1, 2, 6 … 32
there is probably something I omited

tahnks again

[rmccorm4]

Hi @omeralierdemir,

This repo is for the ONNX parser for TensorRT, which is just a subset of TensorRT (you could've also built the ONNX parser, like you did above, by following the steps in the TensorRT GitHub repo).

See https://github.com/NVIDIA/TensorRT for open source C++ samples. There should also be some samples that come with the TensorRT release that you installed in /usr/src/tensorrt/samples for both C++ and Python.

trtexec generally comes with the TensorRT release in /usr/src/tensorrt/bin. Other than trtexec, you can use the C++/Python APIs to test your TensorRT engines.

See https://github.com/dusty-nv/jetson-inference for TensorRT samples specifically for Jetson.

[omeralierdemir]

thanks for your help.
I close this issue...