Flair: How to use Flair for segmenting text

[emrul]

Hi, I'd like to use Flair to experiment with putting paragraphs into text similar to http://aclweb.org/anthology/N18-2075 (I have some other segmentation problems to try but trying to begin with one where I can generate a lot of data with.)

However I'm unsure if the training format Flair needs supports this. I don't have to use the format used in the paper - I could for example produce a data file with one paragraph per line if necessary.

Any pointers would be appreciated.

[bedapudi6788]

You can train Flair's NER model for segmenting text and it works excellently well. Doing this will get you results similar to https://github.com/bedapudi6788/deepsegment/

[emrul]

Thanks @bedapudi6788 this looks perfect. I didn't realise I could do it with a NER model but I will go try it this week!

[alanakbik]

@emrul great, we'd be interested to hear how well it works!

[bedapudi6788]

@alanakbik , I have a trained Flair model for english that does segmentation for punctuated and unpunctuated text excellently. If anyone is interested, I can provide a link for the checkpoints.

[alanakbik]

Oh that is great - we'd appreciate if you could share the models!

[krzynio]

@alanakbik

I can provide polish model developed with the same method if there is a need for that.

[alanakbik]

@krzynio we'd appreciated it if you shared it!

[emrul]

I’d appreciate a link to these also - starting on my model work this week

[krzynio]

@krzynio we'd appreciated it if you shared it!

https://drive.google.com/file/d/1CyCzM4lo--0_v6Ajzr-gndta1kUtjTcf/view?usp=sharing

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.