Yet-another-efficientdet-pytorch: how to convert onnx

[you-old]

hi , how to convert onnw , can you provide the convert file?

[AlexandrGraschenkov]

Hi, @zylo117
That good that you do this job. Whole world can easy experiment with detection task.

It's interesting how fast it can be on mobile. First I try to convert to ONNX without success. Inside code I've changed onnx_export=True. But still get error:

[ CPUFloatType{1,49104,4} ]) of traced region did not have observable data dependence with trace inputs; this probably indicates your program cannot be understood by the tracer.

Do you know why is so?

[421psh]

Hi, @zylo117
That good that you do this job. Whole world can easy experiment with detection task.

It's interesting how fast it can be on mobile. First I try to convert to ONNX without success. Inside code I've changed onnx_export=True. But still get error:

[ CPUFloatType{1,49104,4} ]) of traced region did not have observable data dependence with trace inputs; this probably indicates your program cannot be understood by the tracer.

Do you know why is so?

Same issue after pulling the latest commit, previously I didn't have such problem. Any ideas?

[HtutLynn]

I was able to convert to ONNX with a few modifications. _MemoryEfficientSwish_ function, child of _SwishImplementation_ is causing the problem in ONNX Conversion. But author has already implemented _onnx_export_ flags in most of the modules of efficientdet so you just need to carefully read the code and change all the flags to _True_ accordingly. Here are the steps that I modified to be able to convert to ONNX.

Step 1 :

Add a new parameter _onnx_export_ to EfficientBackbone in efficient.backbone.py

class EfficientDetBackbone(nn.Module):
    def __init__(self, num_classes=80, compound_coef=0, load_weights=False, onnx_export=False, **kwargs):
        super(EfficientDetBackbone, self).__init__()
        self.compound_coef = compound_coef
        self.onnx_export = onnx_export

Step 2 :

Change the _onnx_export_ parameter to _True_ for some of the modules in efficient/backbone.py

For BiFPN

self.bifpn = nn.Sequential(
            *[BiFPN(self.fpn_num_filters[self.compound_coef],
                    conv_channel_coef[compound_coef],
                    True if _ == 0 else False,
                    attention=True if compound_coef < 6 else False,
                    onnx_export=self.onnx_export)
              for _ in range(self.fpn_cell_repeats[compound_coef])])

For regressor

self.regressor = Regressor(in_channels=self.fpn_num_filters[self.compound_coef], num_anchors=num_anchors,
                                   num_layers=self.box_class_repeats[self.compound_coef], onnx_export=self.onnx_export)

For classifier

self.classifier = Classifier(in_channels=self.fpn_num_filters[self.compound_coef], num_anchors=num_anchors,
                                     num_classes=num_classes,
                                     num_layers=self.box_class_repeats[self.compound_coef],
                                     onnx_export=self.onnx_export)

Step 3 :

_MemoryEfficientSwish_ function is also used in EfficientNet feature extractor backbone so we need to change it too.

Add a new parameter _onnx_export_ to EfficientNet in efficientdet/model.py and EfficientNet instance creation in efficient/backbone.py

In efficientdet/model.py

class EfficientNet(nn.Module):
    """
    modified by Zylo117
    """

    def __init__(self, compound_coef, load_weights=False, onnx_export=False):
        super(EfficientNet, self).__init__()
        model = EffNet.from_pretrained(f'efficientnet-b{compound_coef}', load_weights)
        del model._conv_head
        del model._bn1
        del model._avg_pooling
        del model._dropout
        del model._fc
        self.model = model
        self.model.set_swish(memory_efficient=not onnx_export)

In efficient/backbone.py

self.backbone_net = EfficientNet(self.backbone_compound_coef[compound_coef],load_weights, 
                                         onnx_export=self.onnx_export)

Step 4 :

For Onnx conversion, Onnx expects strides to be _int_ so strides should be _int_ instead of list. But sometimes, strides are lists in author's implementation so we need to convert it.
issue

modify efficientnet/model.py line 50:

Depthwise convolution phase

 k = self._block_args.kernel_size
    s = self._block_args.stride
    if isinstance(s, list):
        s = s[0]
    # print(s)
    self._depthwise_conv = Conv2d(
        in_channels=oup, out_channels=oup, groups=oup,  # groups makes it depthwise
        kernel_size=k, stride=s, bias=False)

If we pass a single int as stride to Conv2dStaticSamePadding, the type of depthwise_conv.stride will be tuple instead of list. Therefore, we need to modify line 415 in efficientnet/model.py.
issue

if block._depthwise_conv.stride == [2, 2]: change to
if block._depthwise_conv.stride == (2, 2):

I was able to convert to ONNX model with these modifications. As for TensorRT, I still haven't tested it yet so I can't say for sure whether it is possible or not.

[kanybekasanbekov]

In addition to @HtutLynn 's answer: get rid of the anchors from _EfficientDetBackbone()_ class in backbone.py file. Anchors should not be inside the model if you want to convert a model to onnx.