Models: Request: pre-trained text summarization model

[loretoparisi]

As in many cases in ML/DL when doing text summarization, sequence to sequence for NMT, to train LSTM it is needed the Annotated English Gigaword dataset in the LDC corpora.
Due to licensing issues (the Dataset license costs 6,000$), tensorflow cannot provide the dataset itself:

We used the Gigaword dataset described in Rush et al. A Neural Attention Model for Sentence Summarization.

We cannot provide the dataset due to the license. See ExampleGen in data.py about the data format. data/data contains a toy example. Also see data/vocab for example vocabulary format. In How To Run below, users can use toy data and vocab provided in the data/ directory to run the training by replacing the data directory flag.

But what about to provide pre-trained models to run the graph? Facebook Research fairseq provides several pre-trained models for seq2seq tasks, etc. So is it possibile to have basic pre-trained models (not necessary state-of-the-art, thought that would be the best case)? If not, that is due to licensing issues that involves data transform from one domain (text) to another (float tensors)?

Thank you

[tiberiuichim]

I'm also interested in having this model available.

[transitive-bullshit]

Unfortunately, until this is included, the textsum model will not be very useful to many people given the high cost of training.

Does anyone know of a paid, hosted API that offers this type of text summarization?

All of the text summarization APIs I could find just do simple key sentence extraction, which is relatively very easy. I know I'm not alone in saying that I'd pay to play around with a hosted API exposing a basic, pre-trained textsum model.

[loretoparisi]

@fisch0920 I understand the point, by the way, these are some hints that could help to improve tensorflow about text summarization, since (this is my opinion, but I'm using several ML libraries) so far the only text sum model I was able to train / test and inference - and ready for production -
was the one provided by OpenNMT:

@tiberiuichim See here:

Text Summarization on Gigaword and ROUGE Scoring

Pre-trained models of text summarization on GigaWord are in Models and Recipes

Also since OpenNMT provides C++ standalone binaries for the inference, it's possibile to use CTranslate compiled (multi-platform) and the pre trained models to run in a very easy way.

I think that tensorflow will provide pre-trained models and simplify this process soon.

[transitive-bullshit]

@loretoparisi thank you very much -- your hints were extremely useful in pointing me in the right direction. OpenMNT would be perfect if it only included a better solution for handling OOV/unk words.

Still hoping textsum releases some basic pre-trained models soon :)

[dreamibor]

Still hoping textsum releases in 2018!

[ThatAIGeek]

Any news regarding pre-trained models?

[ishandutta2007]

Any update ?

[ishandutta2007]

Found another repo which has a pretrained model.
https://github.com/harvardnlp/sent-summary
and the corresponding model :
https://s3.amazonaws.com/opennmt-models/textsum_epoch7_14.69_release.t7

The above repo's readme page is pretty empty, here I found another repo which has trained model as well(on same Gigaword dataset) and has more details
https://github.com/thunlp/TensorFlow-Summarization

[hemanthadawale]

@ishandutta2007 @loretoparisi can you suggest a pre-trained Summarization model in 2019...

[CTres]

how about in 2020 ? :)

[saberkun]

https://github.com/tensorflow/models/tree/master/official/nlp/nhnet is a text summarization model but we cannot release pretrained checkpoints due to policies.
The summarization model essentially can be an encoder-decoder transformer.

[loretoparisi]

@saberkun thank you, it could be worth to pre-trained the same model on a different textsum dataset that could let you to openly release the pre-trained!