brain.js too slow at training?

[elpeleq42]

I'm trying to train a chatbot using brain.js as the brain, giving it some basic sentences to learn and when It doesn't answer anything, the bot asks the user how it should answer(so I can train it again with previous information + new information).

Problem is: To make a chatbot which is able to learn, it's necessary for me to train him multiple times, to include new information. And training is REALLY slow (took like a minute to train a few sentences, with 250 iterations). Is there a way to speed up it? I saw a video of someone using it and training didn't take more than 2 seconds

I'm using the browser version (since I've problems downloading npm's on nodejs). May that be the reason?

[robertleeplummerjr]

Can you put a jsfiddle of your example up so we can diagnose?

[elpeleq42]

I've never made a jsfiddle before, so I hope this works
https://jsfiddle.net/tfwc6osh/7/
(I had to fuse 2 Css files into one and 2 javascript files too)

I also am really having a hard time finding more information about brain.js(that not in this github main page)

[robertleeplummerjr]

Which video was it that you saw?

[kcamcam]

I would try getting node to work, I have had better results with it. Here’s a quick walk through (once you have node working):

$ cd /into/directory/of/project
$ npm install brain.js

from there create an app.js file (lets keep ur node app separate for now)
Paste in ur brain script. It should look something like this:

const brain = require("brain.js");
const trainingData = [....];
const config = {
  iterations: 5000,
  log: true,
  logPeriod: 5,
  learningRate: 0.3,
};
net.train(trainingData, config);
    ...

then simply run:
$ node app

I have found that changing iterations and learning rate speeds up/slows down training time.
Have you tried exporting your net.toJSON() or tried net.toString()? This would keep the training data alive past the current session. There's also a parameter called keepNetworkIntact:true, I'm not sure how it works exactly, but it might be worth looking into.

[robertleeplummerjr]

keepNetworkIntact was removed in favor of just doing net.fromJSON(json) and then net.train(data).

Like "If I've gone through the trouble of loading data into the net, it is highly likely I want to further train it."

[kcamcam]

Thanks @robertleeplummerjr for clarifying 🙂.
Also, I found GRU() slightly faster than LSTM() when it comes to training strings.

[elpeleq42]

The problem is that I have a few(many) problems downloading and installing npms with node, so I nearly always have to install it manually on my project. That's why i'm trying to use the browser version.

and the video was that: https://www.youtube.com/watch?v=9Hz3P1VgLz4

[robertleeplummerjr]

@kcamcam nice find! Wait till we get the new architecture running in v2!

[robertleeplummerjr]

@elpeleq42 that explains this in part: the video you are watching is for a feedforward neural network (brain.NeuralNetwork), which they are much much faster because of how simple they are. This, however, doesn't mean we cannot speed things up with the recurrent (rnn, lstm, or gru) networks. In-fact, we are building out in v2 a completely gpu driven system that we are aiming at being much much faster.

[elpeleq42]

I just hope it doesn't get too much GPU dependant. There must be a way to improve the trainning process for things such as strings.

EDIT: and GRU makes the training error a lot bigger, for a tiny performance improvement

[elpeleq42]

Brainjs uses matrices of characters in the training(for strings) doesn't it? Maybe if each element was a word instead of a character, it could speed up things AND make the results more accurate

[robertleeplummerjr]

I just hope it doesn't get too much GPU dependant.

It's all math. The way the recurrent neural networks work with strings is by activating a single neuron that represents a value. So when upgrades comes, we'll make sure they are compatible.

[robertleeplummerjr]

@elpeleq42 it could, but how many words are in the english dictionary?

[elpeleq42]

Actually it could be bad, since then the bot wouldn't understand variations of some words/expressions such as "What is", "What's" and "whats" (which now it can, if the trainning goes well)
I take back my words

[Macil]

If you used a pre-built word embedding, then similar words will have similar values so that the neural net can easily understand the similarity of words.