Tf-pose-estimation: Faster & Improved Model Ideas?

[ildoonet]

As I released faster models, there are many people exciting on using this to make applications.

I think, I need to push a little bit more to make better models. So I need lots of helps.

First of all, I want to make it more accurate. More data, different augmentation, new network architecture, or new training strategies.

Second of all, It should be much faster and run on embedded and mobile. Quantization, new architecture, ...

Please share ideas on anything that can help.

[ildoonet]

Detection

• YOLO, YOLOv2, SSD, DSSD, Improved SSD
• Faster RCNN
• OHEM
• Online Batch Selection for Faster Training of Neural Networks
• Focal Loss for Dense Object Detection
• Online Video Object Detection using Association LSTM

Segmentation

• FusionNet

Small Network

• LCNN
• Mobilenet
• Shufflenet
• Keynote: Small Neural Nets Are Beautiful:Enabling Embedded Systems with Small Deep-Neural-Network Architectures

Pose Estimation

• Stacked Hourglass Networks for Human Pose Estimation
• Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• Dual Path Networks for Multi-Person Human Pose Estimation
• Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation
• PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
• Deep Multitask Architecture for Integrated 2D and 3D Human Sensing
• Compositional Human Pose Regression : https://www.youtube.com/watch?v=c-hgHqVK90M
• Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera

Dataset

• PoseTrack: A Benchmark for Human Pose Estimation and Tracking
• Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
• MPII Human Pose Dataset
• Coco Keypoints Dataset

[ouceduxzk]

@ildoonet I would suggest to use mobilenetv2, which is a improved version of mobilenetv1, however this is no pretrained model available now, so we probably need to train it from scratch on imagenet https://github.com/timctho/mobilenet-v2-tensorflow/blob/master/models.py

[paperclipt]

I'm using OpenPose as a starting point for my problem of estimating pose for occluded people in crowded scenes (some standing directly under the camera). I would appreciate any idea on how to leverage on OpenPose and tailor it to my situation.

[ouceduxzk]

@paperclipt you are in the wrong place, for your problem, you care more about performance instead of speed, you can check googles' cvpr paper 2017 https://arxiv.org/abs/1701.01779, which is more accurate but much slower as it is a top down approach

[ildoonet]

@ouceduxzk Mobilenet v2 seems to be promising. I will try with that.

[ouceduxzk]

@ildoonet I saw there is a pytorch pretrained model, do you use that to transform to tf model or you train a mobilenetv2 from scratch?

[ildoonet]

@ouceduxzk That is possible, but I guess I can try with tensorflow from scratch.
Also, I'm considering decrease number of layers by using different loss modeling.

[ouceduxzk]

@ildoonet or we can use some other pre-trained model , for example, mobilenet-ssd, which is trained to detect objects, include person and the features learned there might be more robust to pose estimation, to speed up, might use less layers as you suggested

[filipetrocadoferreira]

@ildoonet what's your opinion in post-processing? Do you think would be interesting to invest some time to speed up post-processing? Where is currently the bottleneck?

[filipetrocadoferreira]

Definitely, pre/post processing is current bottleneck

nr rois:  1
network took: 0.024294614791870117
inference image: images/image.png in 0.1416 seconds.

[ouceduxzk]

I used a pretrained mobilenet_v2 model, so far slower than worse result

[ildoonet]

@filipetrocadoferreira I will spend some times to refact post processing parts. I have a plan to develop it in c++.

[liljom]

I have an idea for videos captured by fixed position cameras.
Preprocessing would speed up detection by sending smaller images to evaluation containing only frames as big as the people. It's not needed to check a 656x368 frame, processing all the "empty" space (not containing people), only 120x64 frames (optimal resolution has to be found) containing the parts of the picture, where people are.
Also, when there's no movement, where a person is, we can assume, that they keep the same pose.
Hopefully I can include it in my Java project.

[basicvisual]

What do you think of using posenet for pose estimation. It seems to be light weight . (as far as i have read it )