Yolact: Changing Feature backbone !

[abhigoku10]

@fanyix @dbolya @chongzhou96 Thanks for sharing the source code , i have few queries

1. If i modify the network from resnet 50 to resnet 18 should train the model from scratch . I wanted to train Ms coco+ custom class
2. If i change the feature backbone to inception , mobilenet should i perform the training again from scratch

[dbolya]

For both, you should train from scratch (and by scratch I mean using imagenet pretrained weights for the backbone). The backbone features of those networks are not likely to look similar at all to the backbone features of resnet50 or 101, so it wouldn't make sense to fine-tune if you swapped out the backbone.

Though, for the less beefy networks you might not have to train as many iterations because those should converge faster.

[abhigoku10]

@dbolya thanks for the response , i could not understand the later part of the sentence where you mentioned " look similar at all to the backbone features of resnet50 or 101, so it wouldn't make sense to fine-tune " can you please elaborate on this .

I trained the network with resenet18 and i am seeing the drop in accuracy and fps compared to resnet101/50 is this the expected behaviour i.e drop in fps since theoretically accuracy drops but fps should increase right ?? any thoughts on this

[dbolya]

I meant that there are no layers in common between inception and resnet50 for instance, so fine tuning with the resnet50 weights wouldn't do anything because all your layers in the backbone are new.

You should get a stark drop in mAP, but it should increase in speed. How did you implement the resnet18 backbone? You should be able to just take my existing resnet backbone and change the init argument (which specified the number of blocks in each layer).

[abhigoku10]

@dbolya i took your existing backbone and changed to resnet 18 i was getting errors when i started training then i modified the stride and padding values and was able to train it , but the fps is not increased when i compare it with resnet50 and resnet18

[dbolya]

You have to be careful with stride because you may be making the output layers bigger. Can you run yolact.py with this command:
python yolact.py your_resnet18_config
That'll print out some useful debug info.

[abhigoku10]

@dbolya thanks for the pointers , i did the same now i am able to see 5% increase in the fps compared to resnet 50,101. i am still debugging to increase the fps

[luyunxi]

@ abhigoku10 Do you complete the work to use resnet18 as the backbone？ I am going to do the same thing.

[abhigoku10]

@luyunxi yes i did it and was successful in training it

[luyunxi]

@abhigoku10 Can you share the code about that, thx.

[dbolya]

Since this has been open so long, I'm going to close it. Feel free to reopen if you have any updates.

[abhigoku10]

@dbolya just curious is there any HRnet feature backbone implementation which we can use for yolact

[dbolya]

@abhigoku10 Not that I know of, but if there's a pytorch implementation, it should be easy to port.

[Auth0rM0rgan]

@abhigoku10, You are almost leaving comments and ask questions in all issues in this repo. However, when someone asks you a question or help, you just simply ignore and not answer them, which I think it's not right at all!!!

[abhigoku10]

@Auth0rM0rgan thanks for pointing it out so generous of you follow my me in issues in all the repo , i work in a firm which has legalities so generic ones i answer others i cannot and i have informed this to the author only and he is fine with it . pls donot raise ur personal things in repo since we communicate willing to or it option always available (apologies for harsh words)

[Auth0rM0rgan]

@abhigoku10,
I'm not following you at all and since I'm reading all the issues of this repo to have a better understanding of Yolact; I can see that you left comments almost in all issues (completely fine) on THIS repo and whenever someone asks you a question or help you simply ignored that. I can understand that you may not able to share your code or don't want it, which is completely fine too! but you can give them some hints or small help to others that figure out their problem.

[abhigoku10]

@Auth0rM0rgan sure i shall try my best to help out as much as possible , i understand totally what you say but i have some legalities with firm . i do respond !!