Ml5-library: "Classifier with Feature Extractor" might not work with 3+ classes.

[honnet]

The given example works smoothly with 2 classes:
https://ml5js.org/docs/custom-classifier

...but adding a 3rd prediction class doesn't seem to be working yet.
Reproduction sketch:
https://alpha.editor.p5js.org/CedHon/sketches/S1yCjnr-7

Let me know if I can add any detail ;)

[cvalenzuela]

I'll look into this. But changing the hyperparameters might help: https://ml5js.org/docs/FeatureExtractor#parameters

[honnet]

Thanks for the answer!
You suggested it during the workshop and we tried it (see lines 18-19), but it seems that something else is missing...

[cezary69]

For some reason numClasses is always = 2 whatever I put into hyperparameters object
just console.log(classifier) to see it
that's why we can't have more than 2 classes with actual ml5

[gick]

I experienced the same issue, a quick fix that's working for me is to set numClasses explicitely

featureExtractor = ml5.featureExtractor('MobileNet', modelReady);
featureExtractor.numClasses=3

Then by putting a breakpoint in

function classify() {
  classifier.classify(gotResults);
}

And inspecting classifier properties, numClasses is at 3.

I changed a bit the code with a "Add bird image" button, using similar pattern than "Add dog" and "Add cat" and it works correctly, distinguishing 3 classes.

I put the working example on p5.js :
https://alpha.editor.p5js.org/gick/sketches/rJFlNNKGX

[lenny76]

@gick great solutions!
it works for me too.
thanks!

[artismarti]

<3 thank you for this!!!!

[shiffman]

I investigated this a bit more and another way to do this is the following:

featureExtractor = ml5.featureExtractor('MobileNet', { numClasses: 3 }, modelReady);

The number of classes has to be set in advance b/c when examples are added they are "one-hot encoded" according to the total number of classes. This is distinctly different than the KNNClassifier examples which builds up the examples and labels on the fly.

In light of this...

1. Specifying the number of classes should probably be required. I suppose a default of 2 is reasonable for some very basic demos, but it's probably a good idea for the user to think about and specify the number.
2. The number of classes is only relevant when the feature extractor is turned into a classifier. Therefore the number of classes should probably be specified. What about the following:

featureExtractor = ml5.featureExtractor('MobileNet', modelReady);
classifier = featureExtractor.classification(3, video, videoReady);

or in the case of not working with video

featureExtractor = ml5.featureExtractor('MobileNet', modelReady);
classifier = featureExtractor.classification(3);

[cvalenzuela]

Yes, that's the default behavior (see docs) and I believe we fix this in a PR a few months ago. Perhaps adding an example with more classes will help

[shiffman]

Thanks @cvalenzuela. Thinking about this a bit more it feels like the number of classes should be set when calling classification() since it's not relevant to the featureExtractor until that point? We can discuss next week!

classifier = featureExtractor.classification({ numClasses: 3 });

Also, probably worth discussing if we make the terminology more consistent with KNNClassifier and use "labels".

[OmarTahoun]

Thanks for this!! i've been stuck for a minute here trying to figure out why it never classifies the last class added.

@shiffman Don't you think the function classifier.addImage('Cat') should return a warning or an error if the number of labels added to the classifier is already more than the number specified in the classifier object?
This would make it easier to debug.
I would love to work on that!!

[OmarTahoun]

I imagine it to be something like this inside the addImage() function.

if (this.mapStringToIndex.length <= this.numClasses)
// Can add image
else
// Raise Warning("More labels than specified")

[aaaven]

Thanks for this! Tested both methods, worked for me.

featureExtractor = ml5.featureExtractor('MobileNet',{numClasses:4},function(){
    console.log("model is loaded...");
  });

and

  featureExtractor = ml5.featureExtractor('MobileNet',function(){
    console.log("model is loaded...");
  });
  featureExtractor.numClasses = 4;

[joeyklee]

Hi All! This is now resolved with https://github.com/ml5js/ml5-examples/pull/142. Closing up this issue for now. Thanks for this nice discussion! 🙏

[rokkoo]

In last update they change name to numClasses to numLabels

[redet-G]

In last update they change name to numClasses to numLabels
thanks man!

[snkyogi]

For people using 0.4.1 or above, numClasses has been changed to numLabels.

featureExtractor = ml5.featureExtractor('MobileNet', { numLabels: 3 }, modelReady);