Caffe: x,y 扭曲问题

[nopattern]

ssd 中随机采样patch .然后缩放到300*300 。这样会导致大量x ,y 方向缩放比例不一致？有两个导致扭曲：
1。长宽比例是随机的
2.如果图像本身的长宽不等。
那么，大部分训练用图像是不真实，是扭曲的？我的理解正确吗？

[lcj1105]

难得看到中文提问... 我也是这么疑惑的...

[John-HW-Cao]

The answer is no problem and also a problem.
And you can find that many other networks also use such methods to resize the input image or proposals. They can still get excellent performance.

On the other hand, the investigation on how this transformation affect the performance should depend on the details of certain specific network. So this is indeed a problem.

To sum up, this problem has not been studied thoroughly and has been overlooked for some time. In my opinion, this problem is not unique to the SSD network. And there is no need to worry about this problem for now.

[nopattern]

Thank you . It's really strange that ssd can learn from deformed patchs. I wonder it will be better if learning from good ones.

[weiliu89]

I have tried to train a SSD300 model on non-distort image, aka keeping the same aspect ratio as was done in Fast(er) RCNN. It is about 1 mAP higher compared to original SSD300 (on warped image). The speed is proportionally slower because input image is slightly bigger. Besides it is not able to do batch processing.

[nopattern]

thank you,weiliu89 . this solved my problem. but it's really strange why it's so effective on warped image.

[weiliu89]

If you warp the same during training and testing, it should work just fine since you fed the same signal to the computer although it looks a bit weird to us (human). Besides, I have constrained how you sample the patches for training (1/2 ~ 2 aspect ratio). So the distortion is not too bad.

Of course, if your image is too wide, i.e. kitti image. Warping naively like what I did for VOC/COCO/ILSVRC would probably not work that good.

[nopattern]

I find most VOC images are not equal in width and height. The typical width and height is about 500 and 375. so the pitchs resized are commonly not natural for human.

[weiliu89]

Yeah. But it is not that bad if I warp the image to 300 x 300. And computer is different from human. It will learn what you feed it with. It doesn't have an idea what natural objects looks like anyway.

But if your image has aspect ratio 3:1 or 4:1 or even higher, then warping the image to 1:1 might be a problem.

[nopattern]

Hi,wei,,I want to train sample pitchs which is croped equally in height and width, Can you give some suggestion to implement it ?

[weiliu89]

You can change the batch_sampler in the ssd_pascal.py to make min_aspect_ratio and max_aspect_ratio to 1. But you have to deal with test time carefully.

[zimenglan-sysu-512]

@weiliu89 can you share the code of keeping the same aspect ratio as was done in Fast(er) RCNN?
i want to try this way, since my dataset has different aspect ratio.
thanks.