Parlai: Troubles with fine-tuning on specialized dialogue task ...

[mvh57]

Hello all,
I've been working with the Parlai framework (full disclosure: I'm fairly new to chatbots) and trying to fine tune the 90 M parameter model using my own task (fairly simple 2 person dialogue task I created called 'therapy').

python examples/train_model.py -t therapy,blended_skill_talk,wizard_of_wikipedia,convai2:normalized -m transformer/generator --multitask-weights 1,3,3,3 --init-model zoo:blender/blender_90M/model --dict-file zoo:blender/blender_90M/model.dict --embedding-size 512 --n-layers 8 --ffn-size 2048 --dropout 0.1 --n-heads 16 --learn-positional-embeddings True --n-positions 512 --variant xlm --activation gelu --skip-generation True --fp16 True --text-truncate 512 --label-truncate 128 --dict-tokenizer bpe --dict-lower True -lr 1e-06 --optimizer adamax --lr-scheduler reduceonplateau --gradient-clip 0.1 -veps 0.25 --betas 0.9,0.999 --update-freq 1 --attention-dropout 0.0 --relu-dropout 0.0 --skip-generation True -vp 15 -stim 60 -vme 20000 -bs 16 -vmt ppl -vmm min --save-after-valid True --model-file /tmp/saved_model_file

I find that after fine-tuning on my GPU for a couple days, the model gets much worse (than the original 90M parameter model presented in the paper), evidenced by my testing with the chatbot in interactive mode. Most problematic is that the responses become very repetitive and terse, and less engaging.

Thinking that this was a problem with my new task (therapy), I initialed the original 90 M model and further trained it on the same tasks it was originally trained on, without my new task, and I found that the model developed the same undesirable features (more repetition, less engagement, shorter responses).

I'm not entirely sure what the issue is, and why the model seems to get worse when it is fine-tuned, regardless of the task. Any help / input would be greatly appreciated. Thank you!!
-Michael

[stephenroller]

Yeah try adding these arguments when you chat with the bot (like in interactive): --beam-min-length 20 --beam-block-ngram 3 --beam-context-block-ngram 3 --beam-size 10 --inference beam.

We bundled these args with the models we released, but when you train your own model, you'll have to add them yourself.

[mvh57]

Thank you for the quick reply :) I’ll give this a try!

[mvh57]

It worked great and solved my issue! Thanks!