Yolact: Tips for training

[sdimantsd]

Hi,
Im try to train the network with smaller input size (200*200) to speed up the network, But the result are not good...
I have 4K imgs (before augmentations) to do it.

I have some questions:
What parameters do i need to change for the training?
Did somone have tips for the training?
I need to train from scratch or do transfer learning?
Enything else that might help?

Thanks!

[abhigoku10]

@sdimantsd did u modify the network since the network is designed for 700, 550 irrespective of input size the resizing happens .

[sdimantsd]

I used this tow configuretions

ws_im200_config = yolact_base_config.copy({
'name': 'ws_im200',

# Dataset stuff
'dataset': ws_datasetV2,
'num_classes': len(ws_datasetV2.class_names) + 1,

# Training params
'lr_steps': (280000, 600000, 700000, 750000, 850000, 1000000, 1100000, 1200000, 1300000, 1350000, 1400000, 1450000),
'max_iter': 80000000,


'masks_to_train': 40,
'max_num_detections': 40,
'max_size': 200,

'backbone': yolact_base_config.backbone.copy({
    'pred_scales': [[int(x[0] / yolact_base_config.max_size * 200)] for x in yolact_base_config.backbone.pred_scales],
}),

})

yolact_plus_ws_im200_config = yolact_base_config.copy({
'name': 'yolact_plus_ws_im200',

# Dataset stuff
'dataset': ws_datasetV2,
'num_classes': len(ws_datasetV2.class_names) + 1,

# Training params
'lr_steps': (280000, 600000, 700000, 750000, 850000, 1000000, 1100000, 1200000, 1300000, 1350000, 1400000, 1450000),
'max_iter': 80000000,


'masks_to_train': 40,
'max_num_detections': 40,
'max_size': 200,

'backbone': yolact_plus_base_config.backbone.copy({
    'pred_scales': [[int(x[0] / yolact_plus_base_config.max_size * 200)] for x in yolact_plus_base_config.backbone.pred_scales],
}),

})

[dbolya]

In our analysis with YOLACT, we found even 400x400 to be really bad, so I would expect 200x200 to do really badly. Honestly, it might just be that instance segmentation as a method requires more pixels in general.

A couple things I would suggest instead is

1. Use a less beefy backbone at 550x550 (here you're using base which is resnet101, try the resnet50 model or even perhaps resnet18 if you're willing to implement that yourself).
2. Use fewer anchors: YOLACT++ has 9 anchors (over YOLACT's 3), but you could reduce that down to even 1 if you wanted that.

[sdimantsd]

Thx. I try this.

[sdimantsd]

@dbolya Sorry for the interruption again. I looked over the all code and can't find where to change the number of anchors. can you help me please?

[jasonkena]

If I'm not mistaken, the number of anchors is called num_features under fpn_base in data/config.py. It's the number of anchors per FPN layer I think.

[sdimantsd]

Thx

[sdimantsd]

@jasonkena, Thx but i don't think that what @dbolya mean.
He say:

Use fewer anchors: YOLACT++ has 9 anchors (over YOLACT's 3), but you could reduce that down to even 1 if you wanted that.

But num_features is 256...

[jasonkena]

Sorry, you're right. It's called pred_aspect_ratios and pred_scales in config.py, together they make up 9 anchors per feature.

[Auth0rM0rgan]

Hey @jasonkena, Could you explain more about pred_aspect_ratios and pred_scales? How they are together make up the anchors? For example, if I want to have more or fewer than 9 anchors what should I do?

Thanks

[jasonkena]

Sure thing. pred_aspect_ratios is the set of possible anchor aspect ratios (proportion between the height and the width of the rectangular anchor), and pred_scales is the size/scale of the anchor itself. So the proposed anchors are a combination of these 2 parameters, eg: pred_aspect_ratios=[1,2,3] and pred_scales=[4,5,6] will produce anchors with aspect ratios and scales (1,4), (1,5), (1,6), (2,5), ... respectively.

I'm guessing the reason the code looks like

"pred_aspect_ratios": [[[1, 1 / 2, 2]]] * 5,
"pred_scales": [ [i * 2 ** (j / 3.0) for j in range(3)] for i in [24, 48, 96, 192, 384] ]

is because each FPN layer (5 layers in this case), has its own set of initial aspect ratios and scales. In this case, although every FPN layer has the same set of aspect ratios, each layer has different sets of scales.

So to be able to change the number of anchors generated, I think you can just change [1, 1 / 2, 2] and [24, 48, 96, 192, 384] into whatever you want, and the total number of anchors will be the product of the lengths of these 2 lists.

Hope it helps.

*the aspect ratios and scales aren't actually fixed, these are the initial predictions, which are later refined by bbox_layer in yolact.py

[dbolya]

@jasonkena Yup, that's exactly correct.

[sdimantsd]

@jasonkena @dbolya thx

[sdimantsd]

In our analysis with YOLACT, we found even 400x400 to be really bad, so I would expect 200x200 to do really badly. Honestly, it might just be that instance segmentation as a method requires more pixels in general.

@dbolya Im traning using 300x300 px input. the segmentations is pretty good, but the classification is bad...
Im try to detect cars, trucks and peoples. on the shame object i get 3 masks. one for each class.
Do you have any ides?

thx

[dbolya]

@sdimantsd You can either use a higher threshold or use cross-class nms: #188

Just add --cross_class_nms=True while evaluating.

[sdimantsd]

@dbolya yesssssssss!!! work like a charm! thx!

[abhigoku10]

@dbolya just a thought does our code support multi system gpu training / distrubted training ??

[dbolya]

@abhigoku10 I don't think it does, but I'm pretty sure you could just replace DataParallel with DistributedDataParallel if you wanted that.