Yet-another-efficientdet-pytorch: Different from offical implementation

Created on 24 Apr 2020 · 14Comments · Source: zylo117/Yet-Another-EfficientDet-Pytorch

I noticed your implementaiton is different from the offical's in two places:

if self.first_time:
            p3, p4, p5 = inputs

            p6_in = self.p5_to_p6(p5)
            p7_in = self.p6_to_p7(p6_in)

            p3_in = self.p3_down_channel(p3)
            p4_in = self.p4_down_channel(p4)
            p5_in = self.p5_down_channel(p5)

1) You apply conv on the encoder outputs and the p6,p7 are downsampled version of the original.

2) You apply a separate convolution for the residual, not using the above.

if self.first_time:
            p4_in = self.p4_down_channel_2(p4)
            p5_in = self.p5_down_channel_2(p5)

Can you please justify or explain why you made these changes?

Source

bluesky314

Most helpful comment

it's 3, I extract 5 features from the backbone, but only last 3 of them go to BiFPN.

zylo117 on 25 Apr 2020

👍3

All 14 comments

No, it's the same as the official's. You must have missed it.
https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/issues/19#issuecomment-611983918

zylo117 on 24 Apr 2020

👍2

Sorry @zylo117, I still don't get it. In the official paper the authors say that they take level 3-7 features (P3-P7) why in the other issue you say that for the first time there are only 3 features, P3-P5? Maybe is obvious but I really don't get it

mnslarcher on 25 Apr 2020

👍1

Yes, it's really confusing because the paper didn't mention this part. The truth is, there are only three features from the backbone are forwarding to bifpn, P3&P5&P7. But for the sake of consistency, the names were changed to P3/P4/P5. And then P6 is generated from P5, and P7 from P6.

Why? Because any pyramid feature's width and height must be half of the last feature's. There are stages whose features' sizes are not pyramid like. So some of the stages must be left out.

zylo117 on 25 Apr 2020

👍2

Thanks @zylo117 I'm starting to understand but I still have some confusion. From the EfficientNet paper:

EfficientNet-B0 baseline network
Each row describes a stage i with L_i layers, with input resolution and output
channels C_i

| Stage | Operator | Resolution | #Channels | #Layer |
| ------------- | ------------- | ------------- | ------------- | ------------- |
| 1 | Conv3x3 | 224 × 224 | 32 | 1 |
| 2 | MBConv1, k3x3 | 112 × 112 | 16 | 1 |
| 3 | MBConv6, k3x3 | 112 × 112 | 24 | 2 |
| 4 | MBConv6, k5x5 | 56 × 56 | 40 | 2 |
| 5 | MBConv6, k3x3 | 28 × 28 | 80 | 3 |
| 6 | MBConv6, k5x5 | 14 × 14 | 112 | 3 |
| 7 | MBConv6, k5x5 | 14 × 14 | 192 | 4 |
| 8 | MBConv6, k3x3 | 7 × 7 | 320 | 1 |
| 9 | Conv1x1 & Pooling & FC | 7 × 7 | 1280 | 1 |

First: is P1 the output of stage 1 (input res 224 x 224, output res 112 x 112), P2 the output of stage 2 and so on?

If yes, P4_backbone w/h (28, 28) are 1/2 times P3_backbone (56 × 56), not the same (I'm 100% sure I'm missing something but I don't understand exactly what, sorry).

mnslarcher on 25 Apr 2020

oh, I didn't check it carefully, sorry. I meant there are stages whose features' sizes are not pyramid like.

And stage 2 is P1.

Only the second last layers of stage 4,6,8 are used to build BiFPN.

https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/master/efficientdet/model.py#L415

zylo117 on 25 Apr 2020

👍3

Let see what I understood (or not) so far...

From
https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/b5092321ac6a647d48e81b1f8590feb3dbc3ad87/efficientnet/model.py#L154
and below I see that in self._blocks you have only and all the MBConv layers.

From your link I see that, as one can expect, you take the feature maps just before every downsampling.

From my table above this means that at the end of
https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/b5092321ac6a647d48e81b1f8590feb3dbc3ad87/efficientdet/model.py#L408
you have 5 feature maps, the output of

stage 3 - 2° layer (after this layer there is the first downsampling in the MBConv list)
stage 4 - 2° layer (after this layer there is the second downsampling in the MBConv list)
stage 5 - 3° layer (after this layer there is the third downsampling in the MBConv list)
stage 7 - 4° layer (after this layer there is the forth downsampling in the MBConv list)
and
https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/b5092321ac6a647d48e81b1f8590feb3dbc3ad87/efficientdet/model.py#L416
stage 8 - 1° layer (because is the last MBConv layer so it respect the elif condition)

You then discard the None that you have at the beginning of the feature_maps list for how you have implemented this part of the logic:
https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/b5092321ac6a647d48e81b1f8590feb3dbc3ad87/efficientdet/model.py#L420

What I don't get is that here I see 5 feature maps not 3, what I'm not understanding?

mnslarcher on 25 Apr 2020

it's 3, I extract 5 features from the backbone, but only last 3 of them go to BiFPN.

zylo117 on 25 Apr 2020

👍3

Ok so you keep the last 3 feature maps, that, using the EfficientNet table above and changing the input from 224 x 224 to 512 x 512 as it is in the EfficientDet paper for D0 result in:

stage 5 - 3° layer 64 × 64 <- P3 in the paper
stage 7 - 4° layer 32 × 32 <- P4 in the paper
stage 8 - 1° layer 16 × 16 <- P5 in the paper

you then generate:

P6 = Downsampling(P5) 8 x 8
P7 = Downsampling(P6) 4 x 4

Thanks a lot for the explanation @zylo117, I think (I hope) I get it

mnslarcher on 25 Apr 2020

👍1

Hi, your reply to me does not explain why there are 2 residual convs instead of just 1 that have both the job of making the original inputs into the right sizes. You generate two seperate p4_in and p5_in using p4 and p5. The first time is shown in my question 1 and second time in question 2.

p4_in = self.p4_down_channel(p4)
p5_in = self.p5_down_channel(p5)

and

p4_in = self.p4_down_channel_2(p4)
p5_in = self.p5_down_channel_2(p5)

Can you explain why not just have one p4_in/p5_in? Sorry, I could not find this mentioned in ppr or offical implementation.

bluesky314 on 25 Apr 2020

It really is in official implement, you just have to dig deeper. And I explained it on readme, FAQ Q2, the second part, point 8.

zylo117 on 25 Apr 2020

Thanks, I will try to read tf-implementation again.

bluesky314 on 26 Apr 2020

https://github.com/google/automl/blob/3614751749a21ca2fcb299b60238c6651ff51125/efficientdet/efficientdet_arch.py#L493

https://github.com/google/automl/blob/3614751749a21ca2fcb299b60238c6651ff51125/efficientdet/efficientdet_arch.py#L579

https://github.com/google/automl/blob/3614751749a21ca2fcb299b60238c6651ff51125/efficientdet/efficientdet_arch.py#L90

if 0,1, or 2 in 'inputs_offsets', in resample_feature_map() will create new p3,p4 or p5 because the channels of feat[:3] are not equal to config.fpn_num_filters.

Xiao-OMG on 30 Apr 2020

The issue is why is there a p4_down_channel_2, when the first p4_down_channel already created a p4_in of the correct size? In the paper there are two separate streams from p4_in to p4_up and p4_out, so i guess the duplication is to create these two streams. However in the paper there is also stream duplication for P6, so why is this not done in the code?