Simpletransformers: How Should I format the output generated my LanguageGenerationModel Class

[Krish-Nerkar]

I recently finetuned a gpt2 model on a small lyrics dataset using this library, and while generating results i want the output to mbe formatted with a "\n" so that it is in seperate lines. How should i do this.

This is my colab notebook -
https://colab.research.google.com/drive/1BTG99wG1IsVPWYPyKGTDNNZ4mhosdUoT?usp=sharing

This is my output -

I want to format it at the places where clear concatenation of strings of different lines is seen

[ThilinaRajapakse]

I think this might be happening due to the training data that was used. The model should generate punctuations and new lines.

For example:

INFO:simpletransformers.language_generation.language_generation_model:Today is a really nice day for me to tell my friends and family.

I'll miss the family at home, as it gives me the strength to work in my work to continue to contribute to my profession.

A great life is a journey and it takes time and I hope that people get out there and learn from my contributions.

Today is a really nice day for me to tell my friends and family.

I'd like to thank everyone who has supported me this whole journey and I am proud of them, their stories and the generosity of the entire community for their kind words.

Donations of any amount or level are appreciated and encouraged and it would be amazing to share this story of how you made this amazing journey.

If you would like to share this story on Facebook, please use the hashtag #myhype

I would love to hear from you in the future.

Thank you for your support.

Shawn

This article has been rewritten to include the information from a previous version that referred you to me. It has been updated to include additional information about myself.

I want to thank all those who have followed me on this journey and have supported me.

Please respect these people. It has been so hard and I want to take this burden with us to make our lives easier for everyone, particularly the people that have followed me on this journey and help spread the word about me.<|endoftext|>

[Krish-Nerkar]

@ThilinaRajapakse

My training file is already separated by /n charcaters, and I even tried by separating them with two /n characters like this, and fine tuned but still the out put is like that!
IMG_20200801_232742.jpg

[Krish-Nerkar]

I feel it is clearly generating outputs in different lines but while displaying it it's not showing properly and is concatenating them as you can see the following pattern shown in the image is throughout the result

[abedkhooli]

The training text is probably missing periods (.) either between sentences or at end of paragraph or both (if the last print produces a single period (.)).

[Krish-Nerkar]

@abedkhooli My training text is in the following format throughout -

What do you mean?
Oh, oh, oh
When you sometimes say yes
But you sometimes say no
What do you mean?
Hey, yeah
When you don't want me to move
But you tell me to go
What do you mean?
Oh
What do you mean?
Said you're running out of time
What do you mean?
Oh, oh, oh
What do you mean?
Better make up your mind
What do you mean? You're so indecisive, what I'm saying
Trying to catch the beat, make up your heart
Don't know if you're happy or complaining
Don't want for us to end
Where do I start?
First you wanna go to the left and you want to turn right (ooh)
Wanna argue all day, hangin' out all night (oh)
First you're up, then you're down and in between
Oh, I really want to know What do you mean?

And the same dataset when trained using nshepperd's fork, its giving proper results with nicely formatted text. like a normal lyrics, but that model is taking 5-8 minutes to generate results and this one takes seconds hence i use this one. What should i do?

[abedkhooli]

@abedkhooli My training text is in the following format throughout -

What do you mean?
Oh, oh, oh
When you sometimes say yes
But you sometimes say no
What do you mean?
Hey, yeah
When you don't want me to move
But you tell me to go
What do you mean?
Oh
What do you mean?
Said you're running out of time
What do you mean?
Oh, oh, oh
What do you mean?
Better make up your mind
What do you mean? You're so indecisive, what I'm saying
Trying to catch the beat, make up your heart
Don't know if you're happy or complaining
Don't want for us to end
Where do I start?
First you wanna go to the left and you want to turn right (ooh)
Wanna argue all day, hangin' out all night (oh)
First you're up, then you're down and in between
Oh, I really want to know What do you mean?

And the same dataset when trained using nshepperd's fork, its giving proper results with nicely formatted text. like a normal lyrics, but that model is taking 5-8 minutes to generate results and this one takes seconds hence i use this one. What should i do?
I am working on a poetry dataset and forced a period after each line (plus a dash between the two parts of the line - specific to Arabic poetry) and in output, just reformat. It works fine in my case.

for line in generated[0].split('.')[:-1]:
        print(line)

[Krish-Nerkar]

@abedkhooli Thanks So Much!! Its working for me :))))