Brain.js: LSTM for Hate Speech

[equan4647]

I am trying to train a classifier which can classify hate speech mainly detect toxic comments from any data request. The final work of the code is that once i integrate it the system (a blog) i will use it to classify all the text that is toxic and deny posts....Seems fair enough :D

The Statement : i am still leaning into the system and as far as i know LSTM is best used for training text data (_suggestions welcome_) . I am using Toxic Training data set train the network. The CSV file contains data as text and tags as, Toxic,Insult,Severe......,
After cleaning the data and getting it into JSON format for API's the final training data looks like:

Removed the hard language in the toxic example

The Processed Training set is almost 150,000

 {
        "input": "You, sir, are my hero. Any chance you remember what page that's on?",
        "output": "safe"
    },
    {
        "input": "Congratulations from me as well, use the tools well "",
        "output": "safe"
    },
    {
        "input": "DONT PISS AROUND ON MY WORK",
        "output": "toxic"
    },
    {
        "input": "Your vandalism to the Matt Shirvington article has been reverted.  Please don't do it again, or you will be banned.",
        "output": "safe"
    }

CODE:

//Choosing  Net
const net = new brain.recurrent.LSTM();

const readdata = readJson('dataset-01.json');
function readJson(datafile) {
    var obj = JSON.parse(fs.readFileSync(datafile));
    return obj;
}

//Train
net.train(readdata, {
    iterations: 1000,  
    errorThresh: 0.005,  
    log: true,          
    logPeriod: 10,      
    learningRate: 0.3,    
    momentum: 0.1,        
    callback: null,       
    callbackPeriod: 10,   
    timeout: Infinity 
});
net.run('I Hate you');

Problem

Its taking a huge time for learning , not that it would matter to me, as save to JSON/function are there but the main issue is that despite of giving output in two forms 'Safe'/'Toxic' it return garbage values For example " /H. Kaks". Any Help?
I used another data form in which output was given in array and each index indicate a data point but again it wont get the desired output

[
    {
        "input": "D'aww! He matches this background colour I'm seemingly stuck with. Thanks.  (talk) 21:51, January 11, 2016 (UTC)",
        "output": [
            0,
            0,
            0,
            0,
            0
        ]
    },
    {
        "input": "Hey man, I'm really not trying to edit war. It's just that this guy is constantly removing relevant information and talking to me through edits instead of my talk page. He seems to care more about the formatting than the actual info.",
        "output": [
            0,
            0,
            0,
            0,
            0
        ]
    },
    {
        "input": "Dude, I hate your face",
        "output": [
            0,
            0,
            1,
            0,
            0
        ]
    }

[Starman0620]

Yeah, I've been having the same issue with classifying things but usually after time it will clear up. I'll run your code when I get home and see what I can do to at least get it to that point.

[Starman0620]

Been getting the same issue, but I just noticed your iteration amount is low (compared to what I've done) so I am going to gradually increase it and see if it gets better.

[Starman0620]

Hmm, still after a lot of training I am getting random stuff. But I think I know your issue now that I read over the code again, it is interpreting it as if you want to replicate that text and thus it is doing just that. I have to go now so I cannot directly help but just look at this github page, its where I learned how to do what you're doing https://github.com/bradtraversy/brainjs_examples/blob/master/02_hardware-software.js

If you're curious, here's 25000 iterations, libe MY WORKs aellatinnone MY WORKs aellatinnone MY WORKs aellatinnone MY WORKs aellatinnone

[equan4647]

Exactly, Sorry for the late response i was busy working on the same thing but using TensorFlow.JS
I did increased the iteration amount the time was ridiculous huge to train.
On the other hand Tensor is giving a much better response with a word vectorizer and Convolution Neural Net.
Thanks for the Help

[yaneony]

@equan4647 do you have an example in using same with TensorFlow.JS?

[Ashish117]

@equan4647 I have the same problem can you share more information and code you have done using TensorFlow?

[SpeedyCraftah]

@equan4647 dont be fat share it with us

[vorticalbox]

Take a look at this article that uses a neural network and converting text into an array of number to predict if a tweet is trump or Kim Kardashian, you can use the same method but to detect the sentiment and should train a lot faster that LSTM.

Edit: you can also train on the GPU making it even faster.

https://itnext.io/you-can-build-a-neural-network-in-javascript-even-if-you-dont-really-understand-neural-networks-e63e12713a3

[SpeedyCraftah]

Take a look at this article that uses a neural network and converting text into an array of number to predict if a tweet is trump or Kim Kardashian, you can use the same method but to detect the sentiment and should train a lot faster that LSTM.

Edit: you can also train on the GPU making it even faster.

https://itnext.io/you-can-build-a-neural-network-in-javascript-even-if-you-dont-really-understand-neural-networks-e63e12713a3

I tried to implement it into NodeJS, but failed, any support for NodeJS?

[vorticalbox]

Take a look at this article that uses a neural network and converting text into an array of number to predict if a tweet is trump or Kim Kardashian, you can use the same method but to detect the sentiment and should train a lot faster that LSTM.
Edit: you can also train on the GPU making it even faster.
https://itnext.io/you-can-build-a-neural-network-in-javascript-even-if-you-dont-really-understand-neural-networks-e63e12713a3

I tried to implement it into NodeJS, but failed, any support for NodeJS?

Yes it works in nodejs, its what I wrote all my code in.

Be sure that all your inputs are of the same length the length of your longest Input and fill all the shorts ones with 0 to match the length.

[SpeedyCraftah]

Yeah i get NaN on both the results when i tried it, (installed latest brain.js and imported it with require)

[Ashutosh-3601]

Any update on this issue?

[SpeedyCraftah]

Any update on this issue?

Nope, but a found a solution using tensorflow js and neural networks. I'll send you test code later. It does struggle with some longer text though and gets confused. It uses alot of memory if you don't dispose it correctly.

[Ashutosh-3601]

I found an interesting thing to fix NaN.

If you are following https://itnext.io/you-can-build-a-neural-network-in-javascript-even-if-you-dont-really-understand-neural-networks-e63e12713a3

Suppose your training data looks like this-

{ input: 'I will write a book.', output: {future:1} },
  { input: 'He will be writing a book.', output: {future:1} },
  { input: 'They will have written the book.', output: {future:1} },
  { input: 'I will not cry.', output: {future:1} },
  { input: 'You will find happiness', output: {future:1} },
  { input: 'I am about to leave.', output: {future:1} },
  { input: 'Will you go out?', output: {future:1} },
  { input: 'I shall leave.', output: {future:1} },
  { input: 'We will rock you.', output: {future:1} },
  { input: 'I shall bring the laptop.', output: {future:1} },

  { input: 'Will Smith was late.', output: {past:1} },
  { input: 'I had been to that place.', output: {past:1} },
  { input: 'I was selfish.', output: {past:1} },
  { input: 'We had money.', output: {past:1} },
  { input: 'You were so young!', output: {past:1} },
  { input: 'It was the best day.', output: {past:1} },
  { input: 'What were you saying?', output: {past:1} },
  { input: 'I had been to London.', output: {past:1} },
  { input: 'I should not have left.', output: {past:1} },
  { input: 'What was I thinking?', output: {past:1} },

  { input: 'I am here.', output: {present:1} },
  { input: 'Are you eating regularly?', output: {present:1} },
  { input: 'Let me in.', output: {present:1} },
  { input: 'I am running.', output: {present:1} },
  { input: 'Please stop screaming.', output: {present:1} },
  { input: 'I cannot help you', output: {present:1} },
  { input: 'Obey the rules.', output: {present:1} },
  { input: 'Call me a lawyer.', output: {present:1} },
  { input: 'Am I wrong?', output: {present:1} },
  { input: 'Right this way.', output: {present:1} }

As you can see, first one is for future, then after few objects data for past starts and after that for present.
All you have to do is keep all different training data for different results together once, what I mean is ->

{ input: 'I am here.', output: {present:1} },
  { input: 'Will Smith was late.', output: {past:1} },
{ input: 'I will write a book.', output: {future:1} },


  { input: 'He will be writing a book.', output: {future:1} },
  { input: 'They will have written the book.', output: {future:1} },
  { input: 'I will not cry.', output: {future:1} },
  { input: 'You will find happiness', output: {future:1} },
  { input: 'I am about to leave.', output: {future:1} },
  { input: 'Will you go out?', output: {future:1} },
  { input: 'I shall leave.', output: {future:1} },
  { input: 'We will rock you.', output: {future:1} },
  { input: 'I shall bring the laptop.', output: {future:1} },

  { input: 'Will Smith was late.', output: {past:1} },
  { input: 'I had been to that place.', output: {past:1} },
  { input: 'I was selfish.', output: {past:1} },
  { input: 'We had money.', output: {past:1} },
  { input: 'You were so young!', output: {past:1} },
  { input: 'It was the best day.', output: {past:1} },
  { input: 'What were you saying?', output: {past:1} },
  { input: 'I had been to London.', output: {past:1} },
  { input: 'I should not have left.', output: {past:1} },
  { input: 'What was I thinking?', output: {past:1} },

  { input: 'I am here.', output: {present:1} },
  { input: 'Are you eating regularly?', output: {present:1} },
  { input: 'Let me in.', output: {present:1} },
  { input: 'I am running.', output: {present:1} },
  { input: 'Please stop screaming.', output: {present:1} },
  { input: 'I cannot help you', output: {present:1} },
  { input: 'Obey the rules.', output: {present:1} },
  { input: 'Call me a lawyer.', output: {present:1} },
  { input: 'Am I wrong?', output: {present:1} },
  { input: 'Right this way.', output: {present:1} }

*Notice that first three objects have output for present,past and future.

[Ashutosh-3601]

https://ashutosh.why-am-i-he.re/4TzAlB.png Here is proof

[mubaidr]

This is very interesting, the order of input should not matter for training output. You sure you did not change any other thing?

[Ashutosh-3601]

Nothing sir.. Only order