Tensorrt: Support TensorFlow Object Detection API

[marvision-ai]

Hello,

I see TensorRT now supports Tensorflow Faster RCNN models. Do the Tensorflow object detection api models work with this?

If so, what would be a sample config file in order for the uff processor to work properly?

Thanks!

[nikever]

up

[rajeevsrao]

We have a UFF model used for sampleFasterRCNN but that model is trained with NVidia TransferLearningTootlkit, and not with TensorFlow object detection APIs. The implementation of TLT and Tensorflow objection detection APIs are different according to the author of those plugins/sample, so I believe the plugins (with exception of CropAndResize) may not be useful in present form for TF models

[rmccorm4]

@mbufi, If you find anything helpful or implement plugins to work with the Tensorflow Object Detection API, please feel free to share for those who run into a similar issue.

Closing, but feel free to re-open if you have any more issues.

[marvision-ai]

We have a UFF model used for sampleFasterRCNN but that model is trained with NVidia TransferLearningTootlkit, and not with TensorFlow object detection APIs. The implementation of TLT and Tensorflow objection detection APIs are different according to the author of those plugins/sample, so I believe the plugins (with exception of CropAndResize) may not be useful in present form for TF models

I would like to reopen this issue now. I used TLT to train a Resnet50 Faster RCNN model. I now have both .etlt and .engine files for this model. How would I go about using TensorRT python API to utilize this model?

My thinking is that this should be no problem since both of the libraries are internal Nvidia libraries which will play nicely together.

Please let me know how to do this.

Thank you so much for the effort!

[rmccorm4]

Hi @mbufi,

I believe the .etlt file is meant to be used for Deepstream, and the .engine file can be used like any other TensorRT engine.

You can checkout the python samples that come with the TRT release from devzone/NGC container, and look at how they deserialize the engine file and perform inference on it.

I think /use/src/tensorrt/samples/python/common.py has most of what you're looking for.

[marvision-ai]

Are there specific plugins I need to use? I ask this because the TLT documentation tells me I need to use specific pligins and patches to run the conversion and to run on deepstream.
Maybe @rajeevsrao can comment on this to see if there is python api code that supports the FasterRCNN model from TLT.

Thanks!

[UndeadBlow]

Is there any chance to convert TF OD API Rcnn to TRT engine?
Did someone find a way?

[marvision-ai]

No, still no way. It is best to us TLT instead to train your model.

[UndeadBlow]

@mbufi Thanks for answer.
Heh, we all understand that goal of NVIDIA is to push us use their frameworks only, but it can not be even compared to Tensorflow code base =\

[rajeevsrao]

The goal for TensorRT is to support the TF-OD models through the ONNX workflow. We are in the process of improving our ONNX operator coverage and TF2ONNX converter support to support this.