Tfjs: When can we have PoseNet model based on better datasets rather than COCO?

[aligajani]

A few that seem more sensible are (maybe for 2D/3D single/multi):

http://human-pose.mpi-inf.mpg.de/
http://vision.imar.ro/human3.6m/description.php
http://parnec.nuaa.edu.cn/xtan/data/MPHB.html
https://labicvl.github.io/APE.html

[aligajani]

Do we have any guidance on how we can augment current model using transfer learning?

[aligajani]

A cool paper which solves a lot of problems I highlighted in #1387

https://arxiv.org/abs/1901.01760
http://human-pose.mpi-inf.mpg.de/#results

[dsmilkov]

Marked this issue as "discussion". cc @oveddan, @tylerzhu-github who have more background on this topic.

[tylerzhu-github]

A few that seem more sensible are (maybe for 2D/3D single/multi):

1. http://human-pose.mpi-inf.mpg.de/
2. http://vision.imar.ro/human3.6m/description.php
3. http://parnec.nuaa.edu.cn/xtan/data/MPHB.html
4. https://labicvl.github.io/APE.html

Thanks for initiating the discussion. These are all good questions! We choose COCO because it is a large scale dataset that contains multiple people in different environment (both indoor and outdoor environment). It has more images than MPII (1) and it has more diverse subject and background than H36M (2).

we have any guidance on how we can augment current model using transfer learning?

We have used transfer learning to train the model since the model weights are initialized from ImageNet classification pretrained checkpoint.

A cool paper which solves a lot of problems I highlighted in #1387

https://arxiv.org/abs/1901.01760
http://human-pose.mpi-inf.mpg.de/#results

Thanks for sharing the paper and it's a cool paper.

[aligajani]

Alright, but there are some basic pose detection issues that I have highlighted in other issues.

[kavikode]

How can we use the pre-trained PoseNet model to apply transfer learning to the addition of custom set of images because I want to include people using wheelchairs and people using walkers and crutches as well? @tylerzhu-github @oveddan

[velociraptor111]

Is there a training script where we can run and try to improve the model with our own set of images?

[dsmilkov]

Unfortunately, the training script for PoseNet is not open-sourced. @tylerzhu-github for more details on that front.

[velociraptor111]

Thank you for replying. Are there any plans in the future to perhaps make this part open-sourced? If not then I might try and replicate one myself.

Is Posenet solely based of "PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up,Part-Based, Geometric Embedding Model" or does it also have some implementation from "Towards Accurate Multi-person Pose Estimation in the Wild" ? More specifically, is there a paper detailing how PoseNet was created? (training, etc ..)

[sdll]

Any updates on this?

[rajat-008]

@tylerzhu-github does TensorFlow plan on open-sourcing the training script for PoseNet?
After having tried various implementations for pose estimation PoseNet is by fas the best when it comes to being user friendly.
It'll be of great help to have a process to apply transfer learning to PoseNet.