Onnx-tensorrt: "ONNX parser only supports networks with an explicit batch dimension"

[dontloo]

Hi, I built with the latest TensorRT master branch and got the following error message when running python onnx_backend_test.py OnnxBackendRealModelTest and the sample code

[4] Assertion failed: !_importer_ctx.network()->hasImplicitBatchDimension() && "This version of the ONNX parser only supports networks with an explicit batch dimension"

---

Ran 18 tests in 5.622s

FAILED (errors=9, skipped=9)

How should I solve this?

My versions are tensorrt-6.0.1.5 and cuda 10.1

Thanks!

[sungsooo]

Hi, I encountered the same problem.
Did you solve this problem?
Then, could you explain this problem?

Thanks!

[creisner]

Hi,
Same problem here.

Thanks.

[santimarro]

Im having the same problem!
Thanks!

[PaginDm]

Im having this problem too!

Thanks

[nlqq]

well, it means ur onnx version is too old, the batch size in the sample is dynamic, correct onnx version is the newest , not the 0.0.3

[xonobo]

I'm having the same problem!

[Hellcat1005]

Same problem. How to make an explicit batch dimension?

[crouchggj]

@Hellcat1005
using following steps to create INetworkDefinition instance works for me:
(1) const auto explicitBatch = 1U << static_cast(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
(2) auto network = GhUniquePtr(builder->createNetworkV2(explicitBatch));

[j-o-d-o]

@crouchggj That does the trick

[KNaudin]

Could you please specify where you should use this code ?

[ChienYiChi]

Could you please specify where you should use this code ?

Environments:
TensorRT 7.0.0.11
ONNX IR version: 0.0.4
Opset version: 11
Producer name: pytorch
Producer version: 1.3
Domain:
Model version: 0

I use C++ to convert onnx (exported from pytorch) to tensorrt engine.
Here is the code snippet of how I deal with this ExplicitBatch problem
nvinfer1::IBuilder *builder = nvinfer1::createInferBuilder(gLogger);
const auto explicitBatch = 1U << static_cast<uint32_t>(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
nvinfer1::INetworkDefinition *network = builder->createNetworkV2(explicitBatch);

Reference:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-700/tensorrt-developer-guide/index.html

[vegetable09]

@crouchggj it works for me,

Ubuntu16.04
TensoRT7,
cuda10.0,
tensorrt-OSS release/7.0

[fanweiya]

I'm having the same problem!

[cardboardcode]

@crouchggj @ChienYiChi
Thank you for the solutions you have provided. It worked for me as well.

[Solution]

@KNaudin
Steps to resolve this issue:

1. Find the following code statement in your code that is parsing your selected .onnx model file into a TensorRT .engine file.

nvinfer1::IBuilder *builder = nvinfer1::createInferBuilder(gLogger);

1. Add the line after the above code statement.

   const auto explicitBatch = 1U << static_cast(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);

2. Find the LINE 1 below and replace with LINE 2.

[LINE 1]

trt_network_ = tensorrt_common::infer_object(trt_builder_->createNetwork());

[LINE 2]

nvinfer1::INetworkDefinition *network = builder->createNetworkV2(explicitBatch);

Hope this helps.

[kevinch-nv]

Thanks to those in the thread who have provided answers. Post TRT 6.0 you must build your INetwork object with the createNetworkV2(explicitBatch) call in order for onnx-tensorrt to successfully parse any ONNX model. This is documented here in the README.

Closing this issue.