Ml5-library: [teachable machine] Support Pose Model classification

[joeyklee]

Dear ml5 community,

I'm submitting a new issue. Please see the details below.

→ Step 1: Describe the issue 📝

Did you find a bug? Want to suggest an idea for feature?

• feature request

Making a note to add support for the new teachable machine Pose model classifier.

One thing I'm curious to get feedback on is:

1. Should this be more like: ml5.poseNet(teachMachineModelUrl) or ml5.imageClassifier(teachMachineModelUrl)
   or
2. ml5.KNNClassifier(teachMachineModelUrl)

cc/ @irealva @shiffman @yining1023

[shiffman]

This is a really tricky one! Here are some options that might (?) make sense (I'm pretty sure that TM is not using KNN classification under the hood so that is not a good choice.)

This is a somewhat radical idea?

const poseNet = ml5.poseNet(video, modelLoaded);

poseNet.on("pose", gotPoses);
poseNet.on("classify", poseNetModelURL, gotResults);

Or maybe:

const poseNet = ml5.poseNet(video, poseNetModelURL, modelLoaded);
poseNet.on("pose", gotPoses);
poseNet.on("classify", gotLabels);

Also:

const poseNet = ml5.poseNet(video, poseNetModelURL, modelLoaded);
poseNet.on("pose", gotPoses);

function gotPoses(error, poses) {
  poseNet.classify(poses, gotResults);
}

I might actually suggest the following though!

const poseNet = ml5.poseNet(video, modelLoaded);
const brain = ml5.neuralNetwork(poseNetModelURL);
poseNet.on("pose", gotPoses);

function gotPoses(error, poses) {
  brain.classify(poses, gotResults);
}

Using await/async would really simplify the above idea!

We should probably meet and discuss this one!

[joeyklee]

@shiffman - Whew! Thanks for these suggestions. This is definitely something we should chat about in person. Lots of possibilities!

[champierre]

Hi, I am curious about this issue. I look forward to seeing ml5.js support of the pose model classification on Teachable Machine. Will this be implemented in the near future?

[shiffman]

Hi @champierre, we definitely want to do this, but I don't believe anyone is actively working on i so it seems unlikely before next fall. For an interim solution and clues on how we can make it work you can check out this video tutorial which only uses ml5. https://youtu.be/FYgYyq-xqAw

[champierre]

@shiffman Thank you very much for the information about a very helpful tutorial. I will look at it.