Brain.js: Halt and Resume training

[joelnet]

I am interested loading a pre-trained JSON from a file and resuming training.

I have attempted to do just this, though it appears the training is restarted from scratch (based on the training errors for each iteration).

Is there an easy way to do this?

[Nickproger]

+1 I need the same...

[soofft91]

+1

[robertleeplummerjr]

There are two operational halves mentioned here: halt and resume. The latter can be handled via:

const net = new brain.NeuralNetwork();
net.train(data, { keepNetworkIntact: true });

Also, since we aren't version 1 yet, I _really_ like reducing the property name keepNetworkIntact to resume.

I'm open to suggestions on the first operation (and anything else).

[Macil]

Maybe "resumable" instead of resume? Resume sounds like an option you'd use on the subsequent training only.

[Nickproger]

@robertleeplummerjr, thank you for your attention to this problem.

What i noted is that if we use keepNetworkIntact: true like i mentioned here: https://github.com/harthur-org/brain.js/issues/83
then it doesn't work like it's expected,
but actually the weights of neurons are remaining
and if we join 2 training data sets, old with new one and then retrain with keepNetworkIntact: true then our Neural Network will be retrained much much faster than as if we retrain it from scratch.

var net = new brain.NeuralNetwork();

// pre-training
net.train([
    {input: [0, 0], output: [0]},
    {input: [1, 1], output: [0]}
 ]);

// resume training with new data set
net.train([
        {input: [0, 0], output: [0]},  // old training data set
        {input: [1, 1], output: [0]}
    ].concat([
        {input: [0, 1], output: [1]},  // joining new training data set
        {input: [1, 0], output: [1]},
    ],
    {keepNetworkIntact:true}
);

I'm not an expert but just want to put a suggestion:

What if Brain.js will store all previous training data sets by it's own somewhere in the Object Instance if we use some flag and will join each new training data set each time when train() is called or similar?

Also those historic training data sets could be saved/restored into/from JSON if we want:
something like this: net.toJSON(json, {resumableTraining: true});

Also if training data set is quite huge we can try to filter it, removing duplicates and getting just representative data ...but it's other issue...

@robertleeplummerjr , what do you think? can it be done like that?

[robertleeplummerjr]

That is doable, but taking a step back, can we shorten both keepNetworkIntact and resumableTraining?

[Nickproger]

@robertleeplummerjr, what about reinforce or reinforcement ?

net.train([/*...*/], {reinforce: true});

This word is related to theory and doesn't describe implementation like it was with keepNetworkIntact.

That is doable

I just wonder, is there some other more proper way to achieve this without saving all previous input data sets?

[robertleeplummerjr]

I love the name suggestions!

That is a good point on exporting all previous training data. It actually strikes me as an anti-pattern, because it implies I no longer have control of my data, and cannot feed it back into the net.

[robertleeplummerjr]

I think the more proper way would be to concat your old data to new data and be on your way.

[robertleeplummerjr]

Also, thinking about halting training would have to be an async operation, otherwise, we are stuck in a loop until we reach either a low enough error, or we run out of iterations.

[robertleeplummerjr]

For the time being, lets at least address continuing training. I think "reinforce" is the perfect name. If anyone else has a better suggestion, perhaps I can add that later, but for the time being, I'll go ahead and add it.

[robertleeplummerjr]

As for the other side of this: https://github.com/harthur-org/brain.js/issues/88

[robertleeplummerjr]

I looked into this for a few hours today, and didn't reach any conclusion that would impress anyone. I did however have a moment of insight. If you put iterations to 1, and have a parent function call the training, you by indirect definition are halting and resuming training.

So simply do this:

net.train(data, { iterations: 1 });

Or maybe a larger number.

As for resuming training on more new data, simply concatenate with previous data, and Robert is your mother's brother (Bob's your uncle).

Devs, this may be a solution!

P.S. But I'm still changing from "keepNetworkIntact" to "reinforce", it is simply pure genius

[robertleeplummerjr]

Merged!: https://github.com/harthur-org/brain.js/pull/90

[Nickproger]

Hmm.. interesting...
Just wonder.. is there a way to get know how long training can take?
I mean how many iterations or how many percent is done after particular amount of iterations?
Can we get a difference of errors between start and end of training?

[robertleeplummerjr]

I suppose if we had a callback for after each iteration, and the callback returned a means of stopping the iteration, you could do whatever you like in there. How does that sound?

[robertleeplummerjr]

bump

[robertleeplummerjr]

Proposal:

Stops training:

net.train({
  callback: (error, iterations, time) => {
    return false;
  }
});

Note: I added time, thinking I'd like to stop after an hour, for example, which doesn't really translate to iterations very well. I think too we could add it in the options for training if it is liked.

[freddyC]

Sorry I am a few months old on this but right now we have an error threshold for the network, perhaps we could also have a timeout for the network so that if we have reached the error threshold or the timeout it stops training, and include the state in either a state object or as a result's object that can be handled.

So training would look like this:

net.train (trainingData, {
  errorThresh: 0.000119,
  timeout: 360000 // In c# I would use a time span but not sure what is best in js ... millis?
});

console.log (net.getStatus ()) 
/* {
 *   errorThreshold: 0.0005431245,
 *   finished: false,
 *   timeTrained: 360000
 *    // insert other data about the training that has taken place on the network
 * }
 */

[robertleeplummerjr]

I do like the timeout or timetrained idea. It wouldn't take a lot to add this, and this issue really does need handled. I suppose the best question to ask would be, taking a step back "What is the best way to stop a resource intensive process so that it may continue elsewhere later?"

[freddyC]

The way I look at it is like this. A network in a way works similar to a class. It has it's own state, as you train you update the state (i.e. iterations, proficiency, so on). When we serialize the network we also serialize the state, so that in the future when we hydrate the network it it can pick up where it left off on the old state. I may be oversimplifying it but it seems like that should work.

[perkyguy]

So it looks like this could be done if you leverage the reinforce and
keepNetworkIntact options for NN and RNN respectively. Then it just comes
down to how do you break the training process?

[robertleeplummerjr]

@perkyguy you are exactly right. Yesterday I had some time, and threw this together: https://github.com/BrainJS/brain.threadsafe.js
I may put it in the brain.js repo if there is enough desire. In short, it uses thaw.js, which in turn uses setTimeout and some stability around it so that it doesn't lock up the existing thread. So if there was, say a certain keystroke that could be listened for, we could easily stop training, out the data, and return at a later time using reinforce.

Anybody want to throw a small script together that does that? I may if I have some time.

[freddyC]

I will do some playing with it and if I get anything decent put together I will put up a pull request, but I probably won't have time till later tomorrow night.

[freddyC]

So I created the Pull Request for retraining and utilizing the threadsafe.js mentioned above. One of the things I did is I removed the reinforce options and made it default to reinforce. Here is the difference in functionalities:

Old Behavior

Create Network
-> Train Network
  -> If reinforce === false
    -> Create new layers for network with given ohiddenLayers options (or defaults if no size given)
  -> If reinforce === true
    -> Continue training Network assuming layers are established

The biggest problem I can see (and I actually did without realizing it on my first project) is calling train a second time without setting reinforce to true would wipe the old network and replace it. You wouldn't pick up where you left off instead you would recreate and start from scratch.

A secondary problem is for if by some chance a user tries to train a network the first time and sets the reinforce param to true, it will crash the program (I am pretty sure, I didn't actually test it).

New Behavior

Create Network
-> Train Network
  -> If network layers are not created
    -> Create new layers for network with given hiddenLayers options (or defaults if no size given)
  -> if network layers are created
    -> Continue training Network assuming layers are established

The user will create the network as they originally would, and then when they train they can send in the network size they want. When training it will establish the network size if it is not established but then any time after that it will just reinforce by default.

The pull request will also introduce a trainAsync method that utilizes thaw.js

[robertleeplummerjr]

elegant and simple, I like it. Any other feedback?

[robertleeplummerjr]

So if we want a full featured assistant here, we could use https://www.npmjs.com/package/keypress and bind it to key events. Starting would then be simply running the app, pausing would be via key combination... OR, we could simply bind to before exit: https://stackoverflow.com/questions/14031763/doing-a-cleanup-action-just-before-node-js-exits

[josiahbryan]

I see #90 says it was merged, but I don't show any mention of reinforce in https://github.com/BrainJS/brain.js/blob/develop/src/neural-network.js (checked master and develop branch) ...? Did I miss something somewhere...?

[robertleeplummerjr]

We opted to just reinforce when the layers are already defined (look at the sizes property): https://github.com/BrainJS/brain.js/blob/develop/src/neural-network.js#L294

[josiahbryan]

Ahh, I see that now. Sorry about that, thanks!

[robertleeplummerjr]

No problem! Ty for the interest and courtesy.

[dfarkas4]

I tried to continue learning using the XOR example but it doesn't work for me. Am I doing something wrong?

I want the system to continue learning without completely retraining the network.

net.train([
    {input: [0, 1], output: [1]},
    {input: [1, 0], output: [1]},
    {input: [1, 1], output: [0]}]);

net.train([{input: [0, 0], output: [0]}]);

[mubaidr]

Should be possible as of: https://github.com/BrainJS/brain.js/pull/118

If you still think there is something missing, please open a new issue.

[ken0x0a]

Hello @mubaidr
I couldn't understand how to resume training from link you specified.
Could you let me know how to achieve it?

Thank you in advance!

[mubaidr]

Example: https://jsfiddle.net/robertleeplummerjr/jcvu0tx6/2/

[ken0x0a]

@mubaidr Thank you for your reply!

But I think it will not allow me to resume training.

Is this possible to do following?

net.train(data, trainOps);
var output = net.run('doe');  // ', a deer, a female deer'
var json = net.toJSON();
net.fromJSON(json);
/**
 * I want to do something like next line.
 * Is this possible?
 * As I tried before, this will train from 0 and not resume training.
 */
net.train(data, trainOps); 
var output = net.run('doe');  // ', a deer, a female deer' <= I expect to get different result here.

[opiepj]

@mubaidr this does not work for me when using new training data. Opened an issue #419

[c4ud3x]

Example: https://jsfiddle.net/robertleeplummerjr/jcvu0tx6/2/

This is not about training new data but reusing an already trained net. I think you misunderstood the question.