TensorRT 7 for Jetson / Jetpack

[j-o-d-o]

While the TensorRT is now at v7, the cross compile for the Jetson Platforms is still at v6. I had issues on the ONNX parser on v6 which are fixed on the new release and now my code only works on the host machine but not on the Jetson.

Is it planned to update the TensorRT version on the Docker images anytime soon? In my book it would be important to always have the same versions on all the platforms available, cause otherwise you can run into problems that your code only works on one side. But then again, I don't know all the system restrictions.

[rmccorm4]

Hi @j-o-d-o,

TRT7 for Jetpack hasn't been released, sorry but you'll have to stay tuned 🤞

[un-knight]

Hi @j-o-d-o,

TRT7 for Jetpack hasn't been released, sorry but you'll have to stay tuned

How long will it takes for Jetpack TRT7 to release? I am really hoping to use TRT7 on TX2 as early as possible.

[maaft]

TRT7 would be very useful indeed!

[PythonImageDeveloper]

Is it possible to convert the onnx model to TensorRT 7 on the host PC computer and then use the converted model on jetson nano with TensorRT 6?

[depthwise]

@PythonImageDeveloper apparently not:

[TensorRT] ERROR: ../rtSafe/coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)
[TensorRT] ERROR: INVALID_STATE: std::exception
[TensorRT] ERROR: INVALID_CONFIG: Deserialize the cuda engine failed.

[PythonImageDeveloper]

@depthwise
you converted with TensorRT 7 on host PC and run on jetson nano with TensorRT 6, you get this errors?

[depthwise]

I tried it on Jetson Xavier, but yes, this is an attempt to run TRT7 model on TRT6.

[rmccorm4]

TensorRT engines are currently not portable across versions, even on the same machine. A TensorRT engine built with one version should expect to see this error Serialization Error in verifyHeader: 0 (Version tag does not match) if used in another version, like above https://github.com/NVIDIA/TensorRT/issues/292#issuecomment-580566182.

[hyperfraise]

I'm very interested in this too. Trying to deploy tensorrt 3D conv models on jetsons.

[j-o-d-o]

@rmccorm4 Any updates on the timeline when we might seen TensorRT 7 ready on the Jetson Nano?

[rmccorm4]

@j-o-d-o Unfortunately, I can't share that information 😕

[jinfagang]

@rmccorm4 So it's 2020, tensorrt 7 will ported to jetson nano?

[mirecta]

so nvidia ignore it , they not need to sell jetson boards

[jinfagang]

@mirecta No, they just sold jetson boards and stop software upgrading then you have to buy next generation.

[hyperfraise]

@mirecta No, they just sold jetson boards and stop software upgrading then you have to buy next generation.

Such cynicism. I don't think that's the matter here. TensorRT 7.0 was only just released. I do think it'll be compatible in the future. And you know that currently, no "next generation" jetson supports it, right ? So they don't have anything to push you toward...

[glenosborne]

I would like an indication of a release date, approximate it if you have to, we are waiting on this release and would help if we could schedule work around this release. Please give a month or months range so we can at least plan our development. Thanks

[Smankusors]

I saw that there's JetPack 4.4 Developer Preview which have TensorRT 7.1.0 Developer Preview, might you guys want to try it out.

[glenosborne]

Hi Antony,

Thank you very much for letting me know,
How can i get into the developer preview or where do i download
the Jetpack 4.4 developer preview.

This is fantastic, thank you for the email i am really looking forward to
this.

Regards Glen

Glen Osborne

Web Developer

Head Office: PO Box 12-236, Penrose, Auckland 1642, New Zealand

T: [phone] | M: [phone] | E: [email protected][e-mail] | W: www.asgtechnologies.cohttp://www.asgtechnologies.co/

[cid:00b603ea-2b46-4790-8aa4-9592ebaeff6e]

---

From: Antony Kurniawan [e-mail]
Sent: Saturday, 25 April 2020 5:05 PM
To: NVIDIA/TensorRT [e-mail]
Cc: Glen Osborne | AST [e-mail]; Comment [e-mail]
Subject: Re: [NVIDIA/TensorRT] TensorRT 7 for Jetson / Jetpack (#292)

I saw that there's JetPack 4.4 Developer Previewhttps://docs.nvidia.com/jetson/jetpack/release-notes/ which have TensorRT 7.1.0 Developer Preview, might you guys want to try it out.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA/TensorRT/issues/292#issuecomment-619321981, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOB2VVB3JWFYUBN5RT44WZTROJVSBANCNFSM4J6RZRGQ.

[Smankusors]

You can download the Jetpack developer preview via this page :slightly_smiling_face:

[gasparramoa]

I already test _TensorRT 7.1_ in Jetson Nano and it's not compatible with _TensorRT 7.0.0.11_ for the desktop computer.

I tried to use a _TensorRT_ model in Jetson Nano and the same error appeared when I built the engine:

[TensorRT] ERROR: ../rtSafe/coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)
[TensorRT] ERROR: INVALID_STATE: std::exception
[TensorRT] ERROR: INVALID_CONFIG: Deserialize the cuda engine failed

For the context, I was trying to convert a _Pytorch_ model into a _TensorRT_ model.
I first converted the _Pytorch_ to an _ONNX_ model and then converted it to _TensorRT_ in the desktop computer.

[Torch Model] -> [ONNX model] -> [TensorRT model]

The solution was to convert the _ONNX_ model in the Jetson Nano instead of using the _TensorRT_ model converted in the desktop computer. If you convert the _ONNX_ model in Jetson Nano to _TensorRT_, the model will be a _TensorRT 7.1_ and you will be able to built the engine.

[depthwise]

@gasparramoa IIRC that's by design. When TRT launches a model it first profiles it directly on the device to establish which algorithms have the best performance. This can't be done on another device.

[PythonImageDeveloper]

@gasparramoa,
You told, TensorRT 7.1 in Jetson Nano, which jetpack version? 4.4?
Is it possible to share you codes for converting pytorch model to onnx model and then TensorRT model?

[gasparramoa]

I used the JetPack 4.4.
To convert the _Torch model_ to _ONNX model_ I used this function of Pytorch: Check here

torch.onnx.export()

I used the _opset_version 11_ in the _ONNX_ model.

To convert the ONNX to TensorRT I used the _onnx2trt_: Here

[PythonImageDeveloper]

Thanks,
I've used onnx2trt for ssd before, but I didn't got correct predection. of course, with Tensorrt 5 and 6.
Whta's exactly opset_version 11? for each model this should be changable?
How much did you achieve improve performace?

[beizhengren]

Hi @j-o-d-o,
TRT7 for Jetpack hasn't been released, sorry but you'll have to stay tuned

How long will it takes for Jetpack TRT7 to release? I am really hoping to use TRT7 on TX2 as early as possible.

@rmccorm4 @un-knight Hi, does TRT7 support with TX2?
Thanks!

[rajeevsrao]

Folks, please check the Jetpack 4.4 SDK with TensorRT 7.1.3 and support for TX2: https://developer.nvidia.com/embedded/jetpack