Tf-pose-estimation: retrain pose model with custom dataset

[benhunt12]

I have a collection of images I would like to use to retrain mobilenet model. I can successfully use train.py to train the model using the COCO dataset, but how can I use my own dataset to improve performance in specific use cases.

Any feedback would be greatly appreciated! Thanks!

[ouceduxzk]

which kind of specific use cases

[benhunt12]

Thanks for the reply, and sorry about my late response.

Let's say I have 100,000 images of people skiing. The 'background' on all of the images should be the same(very similar, white). I want to retrain the model to be better at recognizing people(and therefore, estimate pose).

[muchuanyun]

Hi @benhunt12
How about converting your dataset to Coco format? Though I haven't tried with this case, I do work it out for other algorithms within custom dataset.

[shubjain26]

I tried retraining with custom dataset for specific type of images, first I ran original openpose (caffe), it has coco_json output, then using it to train mobilenet_thin model in tf-pose-estimation.
but it failed terribly, the model is forgetting everything.

original openpose works much better but giving 4 fps, only.

@ouceduxzk @ildoonet can u pls give any suggestions?

[s101010tw]

Hi @ildoonet
I have the same question as @benhunt12
I have no idea how to retrain pose model which means retain the "graph_opt.pb" to specific case.

Hope someone could answer my question,
thanks a lot!!

[ViktoriaO1]

Hi all,
firstly, a big thanks to @ildoonet for this great implementation!

I've also tried to retrain the model for a custom dataset. What I did in order to customise it was this:
_Situation:_ I am not using the same key points and labels as the coco dataset (I'm working with animal images).

1a. I converted my data into the coco dataset format. This blogpost helped a lot to convert my data to the coco format!

1b. Additionally, I had to define a skeleton model like the coco one and also the pairs following the structure that is visualised in this blogpost.

1c. Consequently, I also adapted the Coco, common and CocoMetaData files for my case.

2. I changed the shape of the relevant tensors and layers:
This means that now I don't have 17 features anymore, but in my case less, e.g. 14.

This also means that everywhere a tensor is defined with the shape (..,..,..,19) I had to replace this 19 by the amount of connections/edges I created for my skeleton model (in the Coco case its 17 features connected by 19 edges, which is where the 19 comes from). For my case that was 16.

Additionally, this means that everywhere a tensor is defined with the shape (..,..,..,38), which is double of 19 because of the 2 dimensions of the vector, I also had to replace this with the double amount of my skeleton edges . So for my case this was 32.

3. I then had to retrain the model and allow soft placement (because otherwise there will be a shape mismatch between the CMU backbone network and my new network definition).
_This step is working, but the results are still completely all over the place... It seems like the network is just learning the means of the original heat maps ; Maybe someone has an idea why this is the case?_

4. To evaluate, you must then only freeze the network and run the evaluation script on your trained and frozen network.

Hopefully this can already help you to adapt the code to your case.

[maryam69898895]

Hi @ViktoriaO1,
How did you define your skeleton?
Please, explain 1a,1b,1c.