Parlai: Blender - fine-tuning your own models, dataset format?

[GraphGrailAi]

In section Fine-tuning your own models - what is the dataset format example?
In example command to train not sure where is dataset specified and what is format. Should it follow a simple guideline from https://parl.ai/docs/tutorial_task.html#getting-a-new-dataset-into-parlai-the-simplest-way ? (consider we have several tasks - blended_skill_talk,wizard_of_wikipedia,convai2)

python examples/train_model.py -t blended_skill_talk,wizard_of_wikipedia,convai2:normalized -m transformer/generator --multitask-weights 1,3,3,3 --init-model zoo:tutorial_transformer_generator/model --dict-file zoo:tutorial_transformer_generator/model.dict --embedding-size 512 --n-layers 8 --ffn-size 2048 --dropout 0.1 --n-heads 16 --learn-positional-embeddings True --n-positions 512 --variant xlm --activation gelu --skip-generation True --fp16 True --text-truncate 512 --label-truncate 128 --dict-tokenizer bpe --dict-lower True -lr 1e-06 --optimizer adamax --lr-scheduler reduceonplateau --gradient-clip 0.1 -veps 0.25 --betas 0.9,0.999 --update-freq 1 --attention-dropout 0.0 --relu-dropout 0.0 --skip-generation True -vp 15 -stim 60 -vme 20000 -bs 16 -vmt ppl -vmm min --save-after-valid True --model-file /tmp/test_train_90M

Should we also create a dict fille in addition to dataset to launch fine-tuning?
https://parl.ai/docs/cli_usage.html#module-parlai.scripts.build_dict

[stephenroller]

The dataset comes from the -t argument. You can replace it and the multitask weights argument for putting in your own dataset.

[stephenroller]

The dict file needs to stay the same though.

[GraphGrailAi]

The dataset comes from the -t argument. You can replace it and the multitask weights argument for putting in your own dataset.

Seems you do expect i understand docs more than i really do (smile here).
I am still need full example, consider i have a dataset described here https://parl.ai/docs/tutorial_task.html#getting-a-new-dataset-into-parlai-the-simplest-way
in format with tabs and every dialog on new line:

text:hello how are you today?   labels:i'm great thanks! what are you doing?
text:i've just been biking.     labels:oh nice, i haven't got on a bike in years!       episode_done:True

Where should i put this file to get it loaded smoothly ? (read some about teachers, still get stuck in code, i assume i need some kind different format for every task blended_skill_talk,wizard_of_wikipedia,convai2)

[aadil-srivastava01]

The dataset comes from the -t argument. You can replace it and the multitask weights argument for putting in your own dataset.

Seems you do expect i understand docs more than i really do (smile here).
I am still need full example, consider i have a dataset described here https://parl.ai/docs/tutorial_task.html#getting-a-new-dataset-into-parlai-the-simplest-way
in format with tabs and every dialog on new line:

text:hello how are you today?   labels:i'm great thanks! what are you doing?
text:i've just been biking.     labels:oh nice, i haven't got on a bike in years!       episode_done:True

Where should i put this file to get it loaded smoothly ? (read some about teachers, still get stuck in code, i assume i need some kind different format for every task blended_skill_talk,wizard_of_wikipedia,convai2)

Create a dataset in the format specified in the link you shared and then you can simply run the training job. For example:
python examples/train_model.py -t fromfile:parlaiformat --fromfile_datapath ./path/to/data.txt -m transformer/generator --multitask-weights 1,3,3,3 --init-model zoo:tutorial_transformer_generator/model --dict-file zoo:tutorial_transformer_generator/model.dict --embedding-size 512 --n-layers 8 --ffn-size 2048 --dropout 0.1 --n-heads 16 --learn-positional-embeddings True --n-positions 512 --variant xlm --activation gelu --skip-generation True --fp16 True --text-truncate 512 --label-truncate 128 --dict-tokenizer bpe --dict-lower True -lr 1e-06 --optimizer adamax --lr-scheduler reduceonplateau --gradient-clip 0.1 -veps 0.25 --betas 0.9,0.999 --update-freq 1 --attention-dropout 0.0 --relu-dropout 0.0 --skip-generation True -vp 15 -stim 60 -vme 20000 -bs 16 -vmt ppl -vmm min --save-after-valid True --model-file /tmp/test_train_90M -bs 4

[pedramsalimi]

I created my own dataset in the right ParlAi format, I followed the first and the easiest way in the tutorial which is the following:
text:[question] labels:[answer]
I tried to display data, it was correct then I started training. after the training phase on Blender 90M, it started to validating...

the problem arises here because the validation phase never ends!
I couldn't figure out why. I also tried smaller models like seq2seq but the problem still exists.

if you know the solution please give me some help. Thanks.

[stephenroller]

Can you post a log, @pedramsalimi?

[pedramsalimi]

I am using the following command:

!python -m parlai.scripts.train_model \
-t fromfile:parlaiformat --fromfile_datapath /content/parlai_fallout_dataset.txt \
-mf zoo:blender/blender_90M/model \
-eps 18.28

_I just copy the log and paste it bellow. sorry about that!
Also, I've specified that weird number of epochs since it will start from 18.25_

[ building dictionary first... ]
[ dictionary already built .]
[ Using CUDA ]
/root/ParlAI/parlai/utils/fp16.py:144: UserWarning: You set --fp16 true with --fp16-impl apex, but fp16 with apex is unavailable. To use apex fp16, please install APEX from https://github.com/NVIDIA/apex.
'You set --fp16 true with --fp16-impl apex, but fp16 '
Dictionary: loading dictionary from /root/ParlAI/data/models/blender/blender_90M/model.dict
[ num words = 54944 ]
Total parameters: 87,508,992 (87,508,992 trainable)
[ Loading existing model params from /root/ParlAI/data/models/blender/blender_90M/model ]
[creating task(s): fromfile:parlaiformat]
[loading parlAI text data:/content/parlai_fallout_dataset.txt]
/root/ParlAI/parlai/core/teachers.py:1418: UserWarning: The data in {path} looks like one very long episode. If this is intentional, you may ignore this, but you MAY have a bug in your data.
'The data in {path} looks like one very long episode. If this '
[ training... ]
[ time:67686.0s total_exs:365601 epochs:18.28 time_left:0s ]
clip \
1
exs \
1
gnorm \
30.21
gpu_mem \
.3895
loss \
2.707
lr \
7.5e-06
ppl \
14.99
token_acc \
.5789
total_train_updates \
339574
tpb \
19
updates
1

[ num_epochs completed:18.28 time elapsed:67686.59965538979s ]
[ Using CUDA ]
Dictionary: loading dictionary from /root/ParlAI/data/models/blender/blender_90M/model.dict
[ num words = 54944 ]
Total parameters: 87,508,992 (87,508,992 trainable)
[ Loading existing model params from /root/ParlAI/data/models/blender/blender_90M/model ]
[creating task(s): fromfile:parlaiformat]
[loading parlAI text data:/content/parlai_fallout_dataset.txt]
[ running eval: valid ]
/pytorch/aten/src/ATen/native/BinaryOps.cpp:81: UserWarning: Integer division of tensors using div or / is deprecated, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.

_and from here it goes forever.._

I also tried another task like Ubuntu but the problem still exists.
It should be noted that it won't let me specify this command: -mf '/tmp/model'. If I do this it raise this error:
RuntimeError: Need to set model argument to use create_agent.

_Sorry for a lengthy post and thanks for your consideration_.

[stephenroller]

This is because you're "continuing" to train the blenderbot model, rather than starting a fresh model from scratch. See our projects page for fine tuning instructions. Namely, you need to provide this rather large command:

python examples/train_model.py -t $YOURDATASET -m transformer/generator --multitask-weights 1,3,3,3 --init-model zoo:tutorial_transformer_generator/model --dict-file zoo:tutorial_transformer_generator/model.dict --embedding-size 512 --n-layers 8 --ffn-size 2048 --dropout 0.1 --n-heads 16 --learn-positional-embeddings True --n-positions 512 --variant xlm --activation gelu --skip-generation True --fp16 True --text-truncate 512 --label-truncate 128 --dict-tokenizer bpe --dict-lower True -lr 1e-06 --optimizer adamax --lr-scheduler reduceonplateau --gradient-clip 0.1 -veps 0.25 --betas 0.9,0.999 --update-freq 1 --attention-dropout 0.0 --relu-dropout 0.0 --skip-generation True -vp 15 -stim 60 -vme 20000 -bs 16 -vmt ppl -vmm min --save-after-valid True --model-file /tmp/test_train_90M

[stephenroller]

Btw, it looks to me like it's validating, it's just a large dataset and taking a long time.

[stephenroller]

Oh also, your batch size is 1 right now, which is why it's super slow.

[pedramsalimi]

Thank you so much! it worked but when I try to choose other metrics instead of ppl like f1 I am getting KeyError. Actually when I am using -vmt f1 or -vmt f1,ppl or -vmt default I'll face the same error. I also tried to use -mcs f1, accuracy, hits@1 but there is no sign of them. I've don't know where I'm doing wrong!

I have another question: I didn't split my dataset into train, valid and test like other tasks. It's just one text dataset in parlai format but it performed validation and test as well as training, I would like to know does parlai handle splitting dataset by itself?
Sorry for asking too many lengthy questions, I'm a novice parlai user.

[GraphGrailAi]

I also have another question regarding Blender not sure should i create separate issue:

The structure of bot answers is good, i mean it answer question, express opinion and the ask follow up question like humans do. BUT in your paper you are confused with stats of what is the most common word phrases bot usually repeats - it some times becomes annoying - asking "what is your hobby" and "what are you doing for living".
Can i change THE STRUCTURE of these particular questions - change them to my specific dataset's questions, but do not touch other abilities ie emphaty and transformer-based text understanding ?
What should i do with dataset for fine-tuning ?

[stephenroller]

Thank you so much! it worked but when I try to choose other metrics instead of ppl like f1 I am getting KeyError. Actually when I am using -vmt f1 or -vmt f1,ppl or -vmt default I'll face the same error. I also tried to use -mcs f1, accuracy, hits@1 but there is no sign of them. I've don't know where I'm doing wrong!

If you turn --skip-generation false then you'll be able to use the word based metrics of f1 and accuracy. It will be much slower

I have another question: I didn't split my dataset into train, valid and test like other tasks. It's just one text dataset in parlai format but it performed validation and test as well as training, I would like to know does parlai handle splitting dataset by itself?

Ah by default it does not separate them out, unfortunately. The easiest way is to create folder/train.txt, folder/valid.txt, and folder/test.txt as separate files, and then use --task fromfile --fromfile-datapath folder --fromfile-datatype-extension true.

[stephenroller]

I also have another question regarding Blender not sure should i create separate issue:

The structure of bot answers is good, i mean it answer question, express opinion and the ask follow up question like humans do. BUT in your paper you are confused with stats of what is the most common word phrases bot usually repeats - it some times becomes annoying - asking "what is your hobby" and "what are you doing for living".
Can i change THE STRUCTURE of these particular questions - change them to my specific dataset's questions, but do not touch other abilities ie emphaty and transformer-based text understanding ?
What should i do with dataset for fine-tuning ?

We chose these datasets because we think they are the best choices for training the bot. But you can certainly modify these datasets, or choose different data sets to fine tune on. I don't have specific recommendations thought.

[pedramsalimi]

You helped me a lot, thank you for your time and attention!

[GraphGrailAi]

@stephenroller could you answer why after fine-tuning with

python examples/train_model.py -t fromfile:parlaiformat --fromfile_datapath ./test_dataset.txt -m transformer/generator --multitask-weights 1,3,3,3 --init-model zoo:tutorial_transformer_generator/model --dict-file zoo:tutorial_transformer_generator/model.dict --embedding-size 512 --n-layers 8 --ffn-size 2048 --dropout 0.1 --n-heads 16 --learn-positional-embeddings True --n-positions 512 --variant xlm --activation gelu --skip-generation True --fp16 True --text-truncate 512 --label-truncate 128 --dict-tokenizer bpe --dict-lower True -lr 1e-06 --optimizer adamax --lr-scheduler reduceonplateau --gradient-clip 0.1 -veps 0.25 --betas 0.9,0.999 --update-freq 1 --attention-dropout 0.0 --relu-dropout 0.0 --skip-generation True -vp 15 -stim 60 -vme 20000 -bs 16 -vmt ppl -vmm min --save-after-valid True --model-file /test_train_90M -bs 4

i have got a model file 1GB size ? It works, but your original model is 168MB only ?

[stephenroller]

The original model file looks to be ~1gb to me:

$ du -hcs *
1.1G    model
584K    model.dict
340K    model.dict.codecs
8.0K    model.dict.opt
4.0K    model.opt
1.2G    total

[GraphGrailAi]

strange things, located in ParlAI/data/models/blender/blender_90M/

[stephenroller]

That version has the optimizer state stripped from the model file to make it smaller. When you re-fine-tune, the optimizer state is still stored in order to enable resuming training.

[GraphGrailAi]

After training was finished how do i check my persona changed ?
Should i use --include-personas False to get overall fune-tuned model of i can specify some persona that are similar to my dataset like here: https://github.com/facebookresearch/ParlAI/issues/2644 ?
Rephrasing: how do i fune-tune to my persona ?

[pedramsalimi]

Hi, I have a little problem with Episodes . my dataset is one big episode that when I start to train it returns a warning. So I would like to know at first is this condition affect my model performance or its learning process or how long does it take to learn?
And then I want to know how to make my dataset in many episodes consider that I made my dataset in the easiest way which is in your tutorial that I mentioned earlier ( I mean in the format of text:__ labels:__ ) . Is there any straight forward way or command for that to specify number of episodes?

[stephenroller]

An episode is just a conversation. Add \tepisode_done:True to the lines in the file that are the last utterances in a conversation.

[pedramsalimi]

Thank you so much! I added an episode after each conversation and what happened is it trains MUCH FASTER!!

[stephenroller]

This thread is becoming a bit of a catch all. I'm going to close it. If any of you have further issues, please open a new issue so we can try to keep discussion focused. Thanks!