Ml5-library: custom neural net

[genekogan]

Is there any way to create your own neural nets, the way you can in Wekinator? right now, it looks like all you can do is use images/video as inputs and use MobileNet, but it would be nice if you could at least define your own input layer, so you could e.g. use facetracker (CLM tracker) landmark points as an input or an FFT or posenet skeleton positions or lots of other things.

for example (very rough) something like:

let layers = [ {"type":"fc", "numNeurons":36 }, {"type":"fc", "numNeurons":20 }, {"type":"classifier", "numClasses":5} ]
const nn = ml5.neuralNet(layers)
var output = nn.predict(input)   // input is a list of 36 input values, output = 0-4

maybe this is more low-level than the rest of the library, so you may be able to hide some of it the way Wekinator does (like using formulas to figure out the hidden layers and just letting the user specify num inputs and num outputs and whether they are continuous or categorical).

[cvalenzuela]

Yes, this is something we have been considering for a while. One of the first iterations of the library had a NN class.

We where tracking this here https://github.com/ml5js/ml5-library/issues/112 but then we moved it here https://github.com/ml5js/ml5-library/issues/121

@shiffman also made a example we want to port: https://github.com/shiffman/Tensorflow-JS-Examples

[jeffin07]

Hey anyone working on this. I like to work on this issue?

[joeyklee]

@jeffin07 - Hi! Thanks for your interest! I know @shiffman has made a version - https://github.com/CodingTrain/Toy-Neural-Network-JS - and has made/is refining a version using tensorflow to evenutally make it into ml5 here: https://github.com/shiffman/Tensorflow-JS-Examples/tree/master/04_neuro_evolution_flappy.

Maybe there's an opportunity to discuss? If I remember correctly, the biggest question for all of us is how the API would look like to expose a generic neural net to the ml5 audience. Still TBD. Any thoughts are welcome! Thanks!

[joeyklee]

Hi All,
With the upcoming release of the NeuralNetwork class. I think we can close this issue for now. Thanks all!