Brain.js: network error rate is unexpected NaN

[c4tz]

What is wrong?

I get the error: network error rate is unexpected NaN when training a RNN on the data below.

How do we replicate the issue?

Training data

trainingData = //paste array from link here

net = new brain.recurrent.RNN();
net.train(trainingData);

console.log(net.run(trainingData[0].input))

Expected behavior (i.e. solution)

The network should be trained successfully and the output should be ~0.48213740458015264

Other Comments

The datasets' values are percentages from -100 to 100, normalized to a scale from 0 to 1. I get the same error when I leave out the normalization.

[Connum]

Why isn't anything happening here? Did you ever get it to work?

I'm currently trying to get the gist of brain.js, but no matter which example I try, I'm always getting NaN. I have some training data quite similiar to yours, all normalized to values between 0 and 1, and all i get is NaN. (I'm using NeuralNetwork instead of recurrent.RNN however).

I also tried different examples available on the web, cloned their repos, didn't change a single value, ran them, and they failed with NaN. It seems to me like brain.js is currently not working at all, but only few people seem to complain, and those who do don't get any answer at all.

[robertleeplummerjr]

Are all the inputs the same size?

[perkyguy]

So I made this work by changing things from using an RNN to a NeuralNetwork. Is there a particular reason you wanted to use the RNN?

So essentially just change to this

net = new brain.NeuralNetwork();

You can see what I did here https://jsfiddle.net/garrettascott/v3fjbLd9/

I get something like 0.4947563707828522 (an example run)

[Connum]

After I figured out that NaN was a result of a malformed input format on my side, I think that my problem is distinct from the OP's and I should post my own issue in order not to hijack this one. Great to see that my chiming in resulted in two hopefully helpful answers for the OP though! :-)

[robertleeplummerjr]

@perkyguy what do you think of possibly putting a warning if input sizes differ when using NeuralNetwork?

[Connum]

@perkyguy ,

using the second training data input via

document.write(net.run(trainingData[1].input))

- shouldn't the result be more towards 0.57...? It seems to me that no matter which input you use, the result is always around 0.49... This is similiar to my test data, where I always get values around 0.99... no matter what. Is it just a case of not enough training data?

[perkyguy]

@robertleeplummerjr, we could provide an error, and then options to handle those situations. For example,

• remove the training data points which have incomplete values
• fill those values with a set number (e.g. any missing values are set to 0, or 0.5)

I think it's a good idea, but maybe we should start up a new issue for that though, so that we can focus on @c4tz 's issue here

[robertleeplummerjr]

maybe we should start up a new issue

@perkyguy, good point

The network should be trained successfully and the output should be ~0.48213740458015264

@c4tz, you are using the wrong net for training, I think. You should take a look at https://github.com/BrainJS/brain.js#for-training-with-rnntimestep-lstmtimestep-and-grutimestep. The reason it likely isn't working: The "standard" recurrent net is actually much more than just a recurrent net (this is changing in v2 to be much more modular, more later). The recurrent net works by identifying, by default, all the individual characters you give it. So if you feed it a big long number, it sees the characters in the input, and not the number itself. The timestep recurrent net is the much more simple version which looks at raw inputs and is the net you are looking for.

A couple examples:

• https://github.com/BrainJS/brain.js/blob/1d3da730157e1ceee36b304f76896ca2ff925396/test/recurrent/rnn-time-step.js#L97
• https://github.com/BrainJS/brain.js/blob/1d3da730157e1ceee36b304f76896ca2ff925396/test/recurrent/rnn-time-step.js#L116

You will probably ask yourself "What if I want to input characters and output math?" or "What if I want to input math and output characters?". What I mentioned for v2 is modularity via "layer composition", in short all these features are just different types of layers. So be looking forward to (or even help output with!) v2.

[perkyguy]

@c4tz and @Connum with this set is that the training set's outputs don't really express too large of a swing. They range from ~0.344 to ~0.655, with the majority of results being in the 0.4-0.5 range. Therefore, we could either re-normalize the output values to be between 0-1, or get more training data to exercise the full range of output.

I have a little visualization of this net with a renormalized output (inputs are left untouched). You can see that it was much better swing. https://jsfiddle.net/mubaidr/dw0cL6hj/6/ . You can move the input bars and the dot represents the output value with those new inputs. This is the normalized output value, so you would have to do a little translation to get it back to normal (trivial though). If you think of this as between those two values though, you can see how there won't be much variance, and how it will be hard to get it out of that ~0.5 +/- 0.05 range.

As to why it didn't shift when using the second element, this might have been just an outlier for that set.

[mubaidr]

I am closing this issue, in case you need further help, please don't hesitate to re-open this issue or create a anew one.