Detectron: anyone have the model Contain key point and mask

[oujieww]

I want to train a model with keypoint and mask,but I don not find it.So i write it by myself ,I don not know is it right?
MODEL:
TYPE: generalized_rcnn
CONV_BODY: FPN.add_fpn_ResNet50_conv5_body
NUM_CLASSES: 2
FASTER_RCNN: True
KEYPOINTS_ON: True
MASK_ON: True

NUM_GPUS: 8

NUM_GPUS: 1
SOLVER:
#WEIGHT_DECAY: 0.0001
#LR_POLICY: steps_with_decay
#BASE_LR: 0.02
#GAMMA: 0.1
#MAX_ITER: 130000
#STEPS: [0, 100000, 120000]
WEIGHT_DECAY: 0.0001
LR_POLICY: steps_with_decay
BASE_LR: 0.005
GAMMA: 0.1
#MAX_ITER: 520000
#STEPS: [0, 400000, 480000]
MAX_ITER: 5000
STEPS: [0, 4000, 4800]
FPN:
FPN_ON: True
MULTILEVEL_ROIS: True
MULTILEVEL_RPN: True
FAST_RCNN:
ROI_BOX_HEAD: head_builder.add_roi_2mlp_head
ROI_XFORM_METHOD: RoIAlign
ROI_XFORM_RESOLUTION: 7
ROI_XFORM_SAMPLING_RATIO: 2
KRCNN:
ROI_KEYPOINTS_HEAD: keypoint_rcnn_heads.add_roi_pose_head_v1convX
NUM_STACKED_CONVS: 8
NUM_KEYPOINTS: 17
USE_DECONV_OUTPUT: True
CONV_INIT: MSRAFill
CONV_HEAD_DIM: 512
UP_SCALE: 2
HEATMAP_SIZE: 56 # ROI_XFORM_RESOLUTION (14) * UP_SCALE (2) * USE_DECONV_OUTPUT (2)
ROI_XFORM_METHOD: RoIAlign
ROI_XFORM_RESOLUTION: 14
ROI_XFORM_SAMPLING_RATIO: 2
KEYPOINT_CONFIDENCE: bbox
MRCNN:
ROI_MASK_HEAD: mask_rcnn_heads.mask_rcnn_fcn_head_v1up4convs
RESOLUTION: 28 # (output mask resolution) default 14
ROI_XFORM_METHOD: RoIAlign
ROI_XFORM_RESOLUTION: 14 # default 7
ROI_XFORM_SAMPLING_RATIO: 2 # default 0
DILATION: 1 # default 2
CONV_INIT: MSRAFill # default GaussianFill
TRAIN:
WEIGHTS: //*/R-50.pkl
DATASETS: ('keypoints_coco_2014_train', 'keypoints_coco_2014_valminusminival','coco_2014_train', 'coco_2014_valminusminival')
SCALES: (640, 672, 704, 736, 768, 800)
MAX_SIZE: 1333
#BATCH_SIZE_PER_IM: 512
BATCH_SIZE_PER_IM: 64
RPN_PRE_NMS_TOP_N: 2000 # Per FPN level
TEST:
DATASETS: ('keypoints_coco_2014_minival','coco_2014_minival',)
SCALES: (800,)
MAX_SIZE: 1333
NMS: 0.5
RPN_PRE_NMS_TOP_N: 1000 # Per FPN level
RPN_POST_NMS_TOP_N: 1000
OUTPUT_DIR: .

is there anyone also want train it?

[ir413]

Hi @oujieww, we plan to add mask+keypoints configs and models in the future.

[Beitadoge]

Hi @oujieww , Did you train the model Containing key point and mask successfully？ i also want to try to do this work.thanks^_^

[terrychenism]

@Beitadoge plz check it here: https://github.com/terrychenism/Detectron/blob/master/configs/12_2017_baselines/e2e_mask_keypoint_rcnn_R-50-FPN_1x.yaml

[Beitadoge]

@terrychenism Oh!!Thanks very much,i will to try it.O(∩_∩)O

[Beitadoge]

hello,@terrychenism ,I use your e2e_mask_keypoint_rcnn_R-50-FPN_1x.ymal ,but i don't train successfully, Did you succeed？if you work ,i have some questions:
1.what model do you choose? 2.do you change other thing? and Could you give me some trick?
thanks for your help !^_^

[terrychenism]

1. resnet-50 as backbone 2. no other modifications
   the result as follows:
   box: 0.5401,0.8285,0.5879,0.3667,0.6136,0.7091
   mask: 0.4534,0.7899,0.4703,0.2413,0.5138,0.6542
   kp: 0.6413,0.8625,0.6919,0.5865,0.7303

[Beitadoge]

Thank you for your kind reply. @terrychenism ，finally i train successfully!O(∩_∩)O

[minhpvo]

@terrychenism Can you please share the pkl model as well?

[oujieww]

@Beitadoge sorry for reply late, the model is ok,it worked

[minhpvo]

@oujieww Can you please share your pkl file as well?

[oujieww]

@minhpvo i have not trained it to end to get the final model pkl，but u can also train as my yaml or auther given

[jpdz]

Hi, I used the ymal file and successfully started training the model. However, the training process will crush due to the limit of gpu memory. Initially, the training process is OK. After maybe 200 iterations, the gpu memory will overflow. I am a little confused about it. Since for each iteration, the same size of data ( batch size ) is trained, I think the gpu usage will remain. Did anyone encounter this problem? Thanks a lot.