Detectron2: onnx model exportable support

[jinfagang]

Since onnx provides almost all ops needs by maskrcnn, it would be great if model can exported to onnx and would be benefit more from TensorRT acceleration for these large models.

[sampepose]

The Portal team at Facebook is already using ONNX to deploy Detectron2 models to device.

I added ROIAlign to ONNX, but we're still missing:

• GenerateProposals
• BBoxTransform
• BoxWithNMSLimit

We splice up the model, using ONNX where possible, and using caffe2.python.core.CreateOperator for these unsupported ops. I will see if we can release our conversion script.

[jinfagang]

@sampepose Thanks for this wonderful work. AFAIK, original maskrcnn-benmark which detectron2 based mainly already exported to onnx by community, and ONNX opset 11 have all ops needed by maskrcnn.onnx.

For what we want final is, exported to onnx if possible and finally convert onnx model to TensorRT engine to gain the massive accelerations.

From this aspect, importing too many ops that not in range of onnx standard is not really necessary I think.

[mlinxiang]

onnx export mode has been supported

any docs or examples?

[dselivanov]

No, it's our internal support.

why is this closed then :-D ? Adding ONNX support to your proprietary ~facebook~ pipeline is not equal to adding it to detectron2 opensource.

[beniz]

@jiajunshen so Google, internally, made the onnx work, and keeps it closed-source, is that correct ? Confusing :)

[jillemash]

Hi,
Shouldn't this issue stay open as long as the conversion script is not available ?
In https://github.com/facebookresearch/detectron2/issues/243#issuecomment-557795036 and https://github.com/facebookresearch/detectron2/issues/46#issuecomment-541446343 Yuxin Wu
(ppwwyyxx) says that this issue will be updated to follow progress on this.
I'm interested to subscribe to this one to be notified when converting models to ONNX will be available.

Thanks.

[jinfagang]

I have to say generates onnx is quite simple. The question is make it runnable on a acceleration framwork such as TensorRT.

things becomes different when it comes to such a complicated model. Wish community can start the next step thing though.

[fernandorovai]

Hi,
@jinfagang can you please provide a direction to convert the model to ONNX?
I wanna use it with OpenVINO

[cryptSky]

@jinfagang Could you share with us how to convert this model to onnx? Would highly appreciate that. Thanks in advance

[Musbell]

@jinfagang I would love @fernandorovai question to be answered because i am having some issues with the conversion.

[jinfagang]

official repo now supports export. using pytorch master source code to build

[nattavitk]

official repo now supports export. using pytorch master source code to build

I built PyTorch from master source code, and successfully export FASTER_RCNN_R50_FPN_3x to onnx model. However, I found that onnxruntime did not know AliasWithName operator.

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: AliasWithName is not a registered function/op

Have you experienced this issue?

[ppwwyyxx]

Answered in https://detectron2.readthedocs.io/modules/export.html#detectron2.export.export_onnx_model

[nattavitk]

Answered in https://detectron2.readthedocs.io/modules/export.html#detectron2.export.export_onnx_model

Thank you

[kotomiDu]

Hi,
@jinfagang can you please provide a direction to convert the model to ONNX?
I wanna use it with OpenVINO

@fernandorovai Hi, Does it work converting detectron2 model into OpenVINO IR?

[LeonNerd]

官方仓库现在支持出口。使用pytorch master源代码进行构建

我从主源代码构建了PyTorch，并成功导出FASTER_RCNN_R50_FPN_3x到onnx模型。但是，我发现onnxruntime不知道AliasWithName运算符。

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: AliasWithName is not a registered function/op

I had the same problem

[justingyym]

在https://detectron2.readthedocs.io/modules/export.html#detectron2.export.export_onnx_model中回答
Can I add these operators(such as AliasWithName) to onnx myself and use onnxruntime for inference. How difficult is the work?

[JavierClearImageAI]

From the comments I assume that exporting detectron2 to caffe2 or ONNX is simple. Any hint about how to convert them to TensorRT or any other way to predict using GPU power?

[avagreeen]

Hi,
I can successfully export the onnx model with MODEL.DEVICE cuda, but when I try to run it with caffe2.python.onnx.backend, with device = 'CUDA', it reports the error:

RuntimeError: [enforce fail at operator.cc:275] op. Cannot create operator of type 'BBoxTransform' on the device 'CUDA'. Verify that implementation for the corresponding device exist. It might also happen if the binary is not linked with the operator implementation code. If Python frontend is used it might happen if dyndep.InitOpsLibrary call is missing.

[rs9899]

@jinfagang @ppwwyyxx

Can you provide some examples for https://detectron2.readthedocs.io/modules/export.html#detectron2.export.export_onnx_model and how to use this to convert DensePose project models to ONNX model that can run in onnx.js or onnxruntime?

Thanks

[jinfagang]

@rs9899 how's DensePose runs speed on pytorch side?

[rs9899]

@jinfagang , personally I am just using to generate IUV_Map of a single image at a time so it runs in about 5 seconds, but that includes loading of the model and the setup. So, it is not the best benchmark.
But do you know how to get similar output on JS and browser because that's where I want to deploy the code?

[nassarofficial]

official repo now supports export. using pytorch master source code to build

I built PyTorch from master source code, and successfully export FASTER_RCNN_R50_FPN_3x to onnx model. However, I found that onnxruntime did not know AliasWithName operator.

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: AliasWithName is not a registered function/op

Have you experienced this issue?

Did you find a solution for this issue?

[xsidneib]

Hi Guys, have anybody sorted this issue by adding a custom plugin? could you share please?

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: AliasWithName is not a registered function/op

[chenynCV]

Hi Guys, have anybody sorted this issue by adding a custom plugin? could you share please?

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: AliasWithName is not a registered function/op

I came across the same issue. Have anyone tried torchvision?

[chkda]

Hi Guys, have anybody sorted this issue by adding a custom plugin? could you share please?

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: AliasWithName is not a registered function/op

Facing the same issue.

[PhamDuyHieutb]

Answered in https://detectron2.readthedocs.io/modules/export.html#detectron2.export.export_onnx_model

I still don't know where it is? Would you please point me?

[PhamDuyHieutb]

I use Caffe2Tracer to convert to onnx model. But when I ran inference, this throwed an exception:

[ONNXRuntimeError] : 1 : FAIL : Fatal error: AliasWithName is not a registered function/op

[zhoujinhai]

Hi,
@jinfagang can you please provide a direction to convert the model to ONNX?
I wanna use it with OpenVINO

Have you success used it with OpenVINO?

[zhoujinhai]

Hi,
@jinfagang can you please provide a direction to convert the model to ONNX?
I wanna use it with OpenVINO

@fernandorovai Hi, Does it work converting detectron2 model into OpenVINO IR?

I have the same problem, Does it work with OpenVINO?

[solarflarefx]

@jinfagang So it's a bit unclear to me. Can detectron2 models now be exported to .onnx and then converted into a gpu-optimized TensorRT engine for inference?

[hasansalimkanmaz]

What I understand from this link, we couldn't run the model via onnx. It is only feasible with caffe2. And, we don't have any clue about post-processing to run it with onnx. Am I missing something?