Brain.js: Strings?

[arjunmenon]

Hey
Would like to know if the inputs objects support text strings.

I did something like this

var brain = require('brain.js')
var net = new brain.NeuralNetwork();

net.train([
           {input: "my unit-tests failed.", output: "software"},
           {input: "tried the program, but it was buggy.", output: "software"},
           {input: "i need a new power supply.", output: "hardware"},
           {input: "the drive has a 2TB capacity.", output: "hardware"}
         ]);

It outputs { error: NaN, iterations: 1 }

One of the comments says that rnn supports them, so is it only available for it or other models can also make use of strings?

[riatzukiza]

Usually one has to one hot encode a string to use it in a neural network.

On Thu, Mar 16, 2017 at 1:48 PM, Arjun Menon [e-mail]
wrote:

Hey
Would like to know if the inputs objects support text strings.

I did something like this

var brain = require('brain.js')
var net = new brain.NeuralNetwork();

net.train([
{input: "my unit-tests failed.", output: "software"},
{input: "tried the program, but it was buggy.", output: "software"},
{input: "i need a new power supply.", output: "hardware"},
{input: "the drive has a 2TB capacity.", output: "hardware"}
]);

It outputs { error: NaN, iterations: 1 }

One of the comments says
https://github.com/harthur-org/brain.js/issues/47#issuecomment-269367451
that rnn supports them, so is it only available for it or other models
can also make use of strings?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/harthur-org/brain.js/issues/65, or mute the thread
https://github.com/notifications/unsubscribe-auth/AKLqvXgXlW91UbNRcG5CiofKzcOnSukKks5rmYP9gaJpZM4MfugO
.

[robertleeplummerjr]

A recurrent neural net in some form is probably the way to go here, the reason being is that their whole design is based on a varying length input, where as a traditional neural net generally has a defined sized input. There are ways around this, of course, but are essentially hacky, like: a maximum length input, normalize all characters, and the rest where doesn't take up the rest of the input, set to 0.... But don't do that, too icky.

Try this:

var brain = require('brain.js')
var net = new brain.recurrent.LSTM();
net.train([
  {input: "my unit-tests failed.", output: "software"},
  {input: "tried the program, but it was buggy.", output: "software"},
  {input: "i need a new power supply.", output: "hardware"},
  {input: "the drive has a 2TB capacity.", output: "hardware"},
  //added for less overfitting
  {input: "unit-tests", output: "software"},
  {input: "program", output: "software"},
  {input: "power supply", output: "hardware"},
  {input: "drive", output: "hardware"},
]);

console.log(net.run("drive"));

Try this with a bunch of data, to avoid overfitting, and see what happens.

It, working (careful, it takes time to train, we are working to make that _way way way_ faster): https://jsfiddle.net/g1poj29x/

[robertleeplummerjr]

May I get some feedback to see if this solved your issue?

[arjunmenon]

Hey
Sorry could not respond. Was juggling between projects.
Would try this approach and expand it for large datasets, by end of day.
By way you have put up the example, I believe the model is too sensitive with stopwords.

[arjunmenon]

First tried encoding the data with a simple BOW vector and then later tried with normal strings.
Label 0 is software and 1 is hardware
All tests run in CLI.

net.train([{input: [ 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], output: [0]},
           {input: [ 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], output: [0]},
           {input: [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0 ], output: [1]},
           {input: [ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1 ], output: [1]},
           //added for less overfitting
           {input: [ 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], output: [0]},
           {input: [ 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], output: [0]},
           {input: [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0 ], output: [1]},
           {input: [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0 ], output: [1]}
], {
  errorThresh: 0.005,  
  iterations: 20000,  
  log: true,
  logPeriod: 10, 
  learningRate: 0.3
});

with

var net = new brain.NeuralNetwork();

training

{ error: 0.004978560090833529, iterations: 156 }

and the result was

output = net.run([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0 ]); // "drive"

Float64Array { '0': 0.8837038196102716 } // "should have been 1 - hardware"

==========

with
var net = new brain.recurrent.LSTM();

training resulted a higher error than neural network
{ error: 1.1078238334154429, iterations: 20000 }

> output = net.run([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0 ]);
'1' // correct!

AND BY GOING WITH THE PREVIOUS EXAMPLE WITH STRINGS with LSTM

{ error: 1.1040929337492722, iterations: 20000 }
> output = net.run("drive")
'hardware' // correct!

Observations

1. Neural network resulted in incorrect output
2. Neural network training time was marvelously short.

3. Neural network reported very low margin of error

4. LSTM resulted in correct output

5. LSTM training time was way way way too long. And particularly longer if using strings.
6. LSTM reported higher error as compared to neural network.

Conclusion, waiting when recurrent network would be _way way way faster_!

_PS - I would post result from more tests particularly from the reuters dataset._

[robertleeplummerjr]

This is an outstanding bug, the error rate for recurrent neural net does not equate to that of the standard recurrent neural net. How long did it take to train? One of the things we are working towards is a refactor of the recurrent neural net that will run both much faster on the cpu, and eventually the gpu.

[robertleeplummerjr]

Also, fantastic work!

[robertleeplummerjr]

Since we were able to put strings in, I'll go ahead and close this issue for continuity, let me know if that isn't satisfactory.

[MarkusPint]

I'm trying the example in @robertleeplummerjr 's Mar 17 post, and the training takes forever. It's been already 4-5 minutes at least. And these strings are pretty short, what would happen with long texts? 500 words? 1000 words?

Any clues to how to overcome this issue? I need to classify texts similarly to @arjunmenon

[robertleeplummerjr]

Can you share your code?

[hellonearthis]

With the text input data, you used full sentences and then key words from the sentences, does that enhance the learning of those patterns and is that how phrases should be learned?

[robertleeplummerjr]

@MarkusPint, would you like to open an issue? I'd like to try and solve for performance if you are up for it.

[binary-person]

The performance does really suffer when it comes to strings in lstm. There's a better alternative of using brain.js though: https://github.com/lordpoint/neural-network-author-classifier but I really _really_ look forward to see lstm perform at least half the speed of the neural network.

In the browser on a chromebook, its even worse, a 8 letter phrase combined with another 6 letter phrase takes about 10 minutes...

[vs-123]

So has anything changed yet?

[binary-person]

I would just use the tensorflow library. This library is really good for beginners but not for complicated applications

[robertleeplummerjr]

I've started to address this in the GPU version, I will try and have something out this week. Faster will come soon!

[timendez]

Also coming across slow speeds with strings in LSTM ->
training data (roughly 2k entries like this):

  { input: 'Test', output: 18.77 },
  { input: 'Fchjk', output: 15.5 },
  { input: 'Guest', output: 7.25 },
  { input: 'Guest', output: 13.2 },
  { input: 'Test First', output: 28.25 },
  { input: 'steve', output: 10.25 },
  { input: 'Auto', output: 0.53 },
  { input: 'Test First', output: 9.74 },
  { input: 'Guest', output: 9.5 },
  { input: 'Justin', output: 10.5 },
  { input: 'Guest', output: 9.85 },
  { input: 'Auto', output: 15.54 },

it's about 5 minutes for every 10 iterations

Didn't notice anything about LSTMGPU mode in the docs, but please let me know if I should attempt a faster training method for string inputs.

Thanks!

[robertleeplummerjr]

You need to encode the numbers as strings using the RNN.

[timendez]

Thanks @robertleeplummerjr! It's much faster now. I think it may unfortunately still be too slow for my uses.

I was able to refocus my goal, however, and use a {string: bool} map with a low number of keys, so that way both my inputs and outputs would be more uniform. From there I used a standard neural net.

I'll definitely check LSTM back out if there ends up being a GPU version or if LSTMs can even take advantage of multiple cores. Thanks again!