Keras: Add "Character-Aware Neural Language Models" as an example

[futurely]

The major benefit of the new model is fewer parameters and morphology robustness.

Yoon Kim, Yacine Jernite, David Sontag, Alexander M. Rush. Character-Aware Neural Language Models. AAAI 2016. https://github.com/yoonkim/lstm-char-cnn

"We describe a simple neural language model that relies only on character-level inputs. Predictions are still made at the word-level. Our model employs a convolutional neural network (CNN) and a highway network over characters, whose output is given to a long short-term memory (LSTM) recurrent neural network language model (RNN-LM). On the English Penn Treebank the model is on par with the existing state-of-the-art despite having 60% fewer parameters. On languages with rich morphology (Czech, German, French, Spanish, Russian), the model consistently outperforms a Kneser-Ney baseline and word-level/morpheme-level LSTM baselines, again with far fewer parameters. Our results suggest that on many languages, character inputs are sufficient for language modeling."

[ylqfp]

Have this already an example?

[futurely]

The paper author's implementation: https://github.com/yoonkim/lstm-char-cnn
He also integrated the algorithm into a larger project: https://github.com/harvardnlp/seq2seq-attn
A TensorFlow master's version: https://github.com/carpedm20/lstm-char-cnn-tensorflow

[anujgupta82]

@ylqfp @futurely
Is there a keras implementation of this ?
Wondering if https://github.com/fchollet/keras/blob/master/examples/imdb_cnn_lstm.py related to this paper

[futurely]

https://github.com/search?utf8=%E2%9C%93&q=Character-Aware+Neural+Language+Models&type=

[mmxgn]

I might be late to the party, but I added a step-by-step implementation here:

https://github.com/mmxgn/tutorials/blob/master/CharacterAwareNeuralModels/Implementing%20Character%20Aware%20Neural%20Models.ipynb

I fail to reproduce the results of the paper. If someone could have a look at it and provide me with feedback (mainly I am looking what I am doing wrong) I would be grateful.