Tensorrt: [TRT] UffParser: Validator error: mrcnn_detection_bboxes: Unsupported operation _SpecialSlice_TRT &&&& FAILED TensorRT.sample_maskrcnn # ./build/out/sample_uff_maskRCNN -d TensorRT/data

[MLcoder1]

I get an error when I try to run MaskRCNN as below:

I have installed libnvinfer 6 also:

$ apt list --installed | grep -i -e nvinfer -e nvparser
It gives the following output:

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libnvinfer-dev/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
libnvinfer-plugin-dev/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
libnvinfer-plugin6/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
libnvinfer5/unknown,unknown,now 5.1.5-1+cuda10.1 amd64 [installed]
libnvinfer6/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
libnvparsers-dev/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
libnvparsers6/unknown,now 6.0.1-1+cuda10.1 amd64 [installed,automatic]
python-libnvinfer/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
python-libnvinfer-dev/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
python3-libnvinfer/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]
python3-libnvinfer-dev/unknown,now 6.0.1-1+cuda10.1 amd64 [installed]

[SongXia84]

Got same error!
ubuntu14.04+cuda9.0+tensorRT6.0.1.5 +tensorflow1.12.0.

Another issue is:
When use tensorflow 1.14.0, I can not convert the h5 model to the UFF model.

File_ "/usr/local/lib/python3.6/dist-packages/uff/converters/tensorflow/converter_functions.py", line 40, in convert_add
    uff_graph.binary(inputs[0], inputs[1], 'add', name)
IndexError: list index out of range

[fuman5000]

I get the same error as SongXia84, index out of range.

[dssdyx]

I think it's the problem that exe cannot link libnvinfer_plugin.so correctly.

[dssdyx]

I mean we have to cmake and make TensorRT source as what tutorial says, otherwise it will result in some strange errors...

[MLcoder1]

The problem persists even after cmake and make. :(

[dssdyx]

The problem persists even after cmake and make. :(

If you have install TensorRT before you try this sample , you may fail to build and run it. Because in your computer there has been a library called libnvinfer_plugin.so without the needed plugins in this sample.So, I think maybe you should reinstall TensorRT. At least, don't let the exe link to wrong .so

[rtrobin]

I downgraded keras to 2.1.6. It could convert model to uff successfully. Hope it could help anyone here. @SongXia84 @fuman5000

[ShawnNew]

I downgraded keras to 2.1.6. It could convert model to uff successfully. Hope it could help anyone here. @SongXia84 @fuman5000

I have Keras version of 2.1.3 as shown in the following picture.

and tensorflow version < 2.0.

But I still got the problem of list index out of range when I issued python3 mrcnn_to_trt_single.py command. Surely the parameters were correctly assigned to the command.
Please refer to the error log:

Anyone here can help? Thanks in advance!!

[rtrobin]

@ShawnNew please refer this post.

TensorRT: 6.0.1.5
CUDA: 9.0
CuDNN: 7.6.3
python: 3.6.5
Tensorflow-gpu==1.9.0
Keras: 2.1.3

[ShawnNew]

Is it true that this problem stems from version mismatching?

[rtrobin]

@ShawnNew what problem are you referring to?

For your post, yes it is about mismatching of the version of tf or keras or both. For my experience, the different version of keras does lead to converting problem.

For the original post, it is about lacking the plugin in runtime. It could be fixed by loading custom op plugin library dynamically, or rebuild and reinstall whole trt plugin.

[ShawnNew]

@rtrobin Thank you very much. I can link the plugin correctly, while still get the list index out of range problem I posted even though I installed the right keras version. The only difference in my setting is I have tensorflow-gpu==1.14.0, I don't know if that will make things go south.
Will try older tf version and report the result soon.

[gemmit]

I downgraded keras to 2.1.6. It could convert model to uff successfully. Hope it could help anyone here. @SongXia84 @fuman5000

I have Keras version of 2.1.3 as shown in the following picture.

and tensorflow version < 2.0.

But I still got the problem of list index out of range when I issued python3 mrcnn_to_trt_single.py command. Surely the parameters were correctly assigned to the command.
Please refer to the error log:

Anyone here can help? Thanks in advance!!

@ShawnNew I don't know if this helps, but did you make the changes in converter_functions.py as mentioned in the Readme here?

[ShawnNew]

I downgraded keras to 2.1.6. It could convert model to uff successfully. Hope it could help anyone here. @SongXia84 @fuman5000

I have Keras version of 2.1.3 as shown in the following picture.

and tensorflow version < 2.0.

But I still got the problem of list index out of range when I issued python3 mrcnn_to_trt_single.py command. Surely the parameters were correctly assigned to the command.
Please refer to the error log:

Anyone here can help? Thanks in advance!!

@ShawnNew I don't know if this helps, but did you make the changes in converter_functions.py as mentioned in the Readme here?

@gemmit I have already changed the index in 'converter_functions.py'.

[Tyler-D]

@ShawnNew I think you are using tensorflow in cpu version. mrcnn_mask is just a convolution, it should not have transpose node unless in cpu version (there is only NHWC convolution in tensorflow cpu version, so it adds transpose node automatically for NCHW data)

So maybe you could try pip uninstall tensorflow==1.14.0.

[ShawnNew]

@Tyler-D I have already uninstall tensorflow (CPU version). But still got no luck.
My development environment depends on cuda 10.1, so I cannot install tensorflow==1.9.0.

[rtrobin]

@ShawnNew You can pull a tensorflow(==1.9.0) docker image do the conversion job. Once you get the uff model, you can use it in your development environment for deployment.

[Tyler-D]

@ShawnNew OK. That's the point. Look at this https://www.tensorflow.org/install/gpu
I think tensorflow is not compatible with CUDA 10.1 ---- That's why it turns to CPU-version convolution.

[rmccorm4]

Re: TF compatability with CUDA 10.1, it looks like it's pretty easy to do if building from source: https://github.com/tensorflow/tensorflow/issues/26150#issuecomment-506807444

I'm not sure how it works with the current pip package.

I generally work with/in containers for simplicity, similar to @rtrobin's suggestion.

FWIW, Nvidia's latest NGC container (19.10) comes built with CUDA 10.1 + TF 1.14: https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/rel_19.10.html#rel_19.10

[Tyler-D]

Yes. But if users use pip install to install tensorflow, a library of tensorflow compiled with CUDA10.0 will be downloaded.

[ShawnNew]

@rtrobin I succeeded to transform the keras model into trt by using docker container. Below is a quick record:

• docker pull tensorflow/tensorflow:1.9.0-gpu-py3 pull the tensorflow(==1.9.0) docker image from dockerHub.
• nvidia-docker run --rm -v /path/to/trt/release:/tensorrt -v /path/to/trt/source:/workspace/TensorRt -it tensorflow/tensorflow:1.9.0-gpu-py3 /bin/bash start a container.
• Inside the container, install related package into python3.
  # install tensorrt packages cd /tensorrt/graphsurgeon pip3 install graphsurgeon-xxx-py2.py3-none-any.whl && cd ../uff pip3 install uff-xxx-py2.py3-none-any.whl && cd ../python pip3 install tensorrt-6.0.1.5-cp35-none-linux_x86_64.whl # install depend pip packages cd /workspace/TensorRT/samples/opensource/sampleUffMaskRCNN pip3 install keras==2.1.3 scikit-image # run the conversion cd converted && python3 mrcnn_to_trt_single.py -w /path/to/keras/model -o /path/to/output -p ./config.py

Unfortunately, I cannot run inference in my development environment outside docker. Got the following error:

I think the inference process does not depend on tf, so the problem isn't caused by tf. And I also tried copying libnvinfer_plugin.so.6.0.1 into /usr/lib/x84_64-linux-gnu but no had luck.
What else can I try to this ./sample_uff_maskRCNN example?

[ShawnNew]

Yes. But if users use pip install to install tensorflow, a library of tensorflow compiled with CUDA10.0 will be downloaded.

@Tyler-D Indeed, with CUDA version 10.1, pip will download tf version (>1.9.0). So we can try the docker approach, refer https://github.com/NVIDIA/TensorRT/issues/132#issuecomment-550120904.

[rtrobin]

@ShawnNew Check this post.

[ShawnNew]

@rtrobin Hi, I have tried link the .so library to /usr/lib/x86_64-linux-gnu directory, but still failed.

[rmccorm4]

Hi all,

Hopefully this post can help any still struggling: https://github.com/NVIDIA/TensorRT/issues/123#issuecomment-551269792

[ShawnNew]

Thanks @rmccorm4 , but I have already passed the conversion step by using tensorflow/tensorflow:1.9.0-gpu-py3 image.
My problem now is I cannot proceed to the next step, which means inference failed.

This is the screenshot of the error.

[rmccorm4]

@ShawnNew What environment are you trying to convert the UFF -> TRT like in your screenshot? That still just looks something like auto conversion from add->addV2, which I think happens in TF >= 1.15.

I think you need to use TF <= 1.14 for the whole process TF -> UFF -> TRT, but you only did the (TF -> UFF) part in your container.

Once you've converted UFF -> TRT, your tensorflow version shouldn't matter anymore because you should strictly be using the TensorRT API at that point to do inference on your TRT engine.

You might be able to also convert UFF -> TRT in that tensorflow/tensorflow:1.9.0-gpu-py3 container, but I'm not sure. If you're going to try using a container for parsing UFF -> TRT, I would suggest going with nvcr.io/nvidia/tensorflow:19.10-py3 at the moment.

[ShawnNew]

Thanks @rmccorm4 , will try using tensorrt API to convert UFF model to TRT within the container.

[ShawnNew]

I finally succeed in the sampleUffMaskRCNN example by using https://github.com/NVIDIA/TensorRT/issues/123#issuecomment-551269792. Many thanks to @rmccorm4.

[rmccorm4]

Woohoo!

I think there's now enough information in this thread and #123 to make this work. Along with a couple tweaks I made in #208 to hopefully avoid some pitfalls.

I'm going to close this now. Feel free to ask to have this one re-opened if you're still unable to get it working with the above information.

[ApoorvaSuresh]

Hello all,

I successfully converted my model to UFF.

When I run the sample, I have this error:

&&&& RUNNING TensorRT.sample_maskrcnn # ./sample_maskRCNN -d /mnt/DTAA_data/DTAA/code/z662728/optimization/data/
[05/04/2020-09:24:19] [I] Building and running a GPU inference engine for Mask RCNN
[05/04/2020-09:24:21] [E] [TRT] UffParser: Validator error: mrcnn_detection_bboxes: Unsupported operation _SpecialSlice_TRT
&&&& FAILED TensorRT.sample_maskrcnn # ./sample_maskRCNN -d /mnt/DTAA_data/DTAA/code/z662728/optimization/data/

How did you solve it? @rmccorm4 @ShawnNew I would be very grateful if you could please help me :)

Config:
TF-gpu 1.14.0
CUDA 10.0
TensorRT: 6.0.1.5
Keras 2.1.3

Thanks!