Tacotron2: Pre-trained model to resume/fine-tune train from?

[duvtedudug]

I notice that the pre-trained model does not include optimiser details. @rafaelvalle mentioned in another thread it is not published as a checkpoint to resume from.

It would be great to have a model that can be resumed, for fine-tuning on smaller datasets.

Can anybody share a LJSpeech or other heavily trained model checkpoint?

Thanks in advance, Duvte.

[rohan6366]

Would be handy to have a pre-trained Tacotron 2 to continue training.

[tugstugi]

See https://github.com/NVIDIA/tacotron2#inference-demo

[rohan6366]

Correct me if I'm wrong. But can you fine-tune this?

[tugstugi]

sure. I have successfully used it to fune tune Mongolian TTS. For a foreign language, you have to filter out the embedding weights.

[hatiff]

Sorry, but could you explain more precisely what needs to be done for this?

[hatiff]

If i use pretreined model from https://github.com/NVIDIA/tacotron2#inference-demo exactly for eng language. How can i make it?

[duvtedudug]

@tugstugi I couldn't fine-tune with the published model. (Meaning I couldn't resume training LJSpeech and for the initial loss to be low.) So @rohan6366 I agree with you.

Maybe there's something I'm missing?

Would love to fine-tune from a well trained model.

[tugstugi]

I have fine tuned the provided pre-trained model for a Mongolian 5h dataset. You have to use --checkpoint_path and --warm_start. Because I am using a foreign language, I have to manipulate https://github.com/NVIDIA/tacotron2/blob/master/train.py#L92 so that the embedding weights are not loaded. After that, I got already alignment after 3000 steps. Here is a sample after 10K steps: https://github.com/NVIDIA/waveglow/issues/84#issuecomment-467570655 Because the pre-trained model works for a foreign male voice, I am totally sure that it will also work for an English dataset :) See also https://github.com/NVIDIA/tacotron2/issues/135

[rafaelvalle]

We'll soon make changes to the repo to make pre-training easier.

[duvtedudug]

You have to use --checkpoint_path and --warm_start.

Thanks @tugstugi !!!

I used --warm_start and trained from the published LJ model. I continued with LJ and it definitely transfers better (loss 0.3 after a 100 or so steps).

However, it definitely loses quality from the original model.

Perhaps the default learning rate should be reduced from 1e-3? @rafaelvalle what did you anneal LR to?

Any other optimisation parameters I should look at?

Thanks in advance!

[rafaelvalle]

Yes! it's very important to anneal the learning rate.

[rafaelvalle]

We have updated the README and code to make it easier to train a new model starting from our pre-trained model.This should be the best option when training a new model.

[primejava]

我已经为蒙古5h数据集精心调整了提供的预训练模型。你必须使用--checkpoint_path和--warm_start。因为我使用的是外语，所以我必须操纵https://github.com/NVIDIA/tacotron2/blob/master/train.py#L92，以便不加载嵌入权重。在那之后，我在3000步之后已经对齐了。以下是10K步骤之后的示例：NVIDIA / waveglow＃84（评论）因为预训练模型适用于外国男性声音，我完全相信它也适用于英语数据集:)参见＃135

can you show the modified code ？I'm new to this.thank you very much!

[Jeevesh8]

Yes! it's very important to anneal the learning rate.

@rafaelvalle I don't think annealing is done in the repo. Can you confirm this ? I will make a pull request, if it hasn't been done.

[titocaco]

I have fine tuned the provided pre-trained model for a Mongolian 5h dataset. You have to use --checkpoint_path and --warm_start. Because I am using a foreign language, I have to manipulate https://github.com/NVIDIA/tacotron2/blob/master/train.py#L92 so that the embedding weights are not loaded. After that, I got already alignment after 3000 steps. Here is a sample after 10K steps: NVIDIA/waveglow#84 (comment) Because the pre-trained model works for a foreign male voice, I am totally sure that it will also work for an English dataset :) See also #135

@tugstugi Please, I would like to know what exactly have you done with the train.py file. I am trying to continue training for another foreign language too, and the symbols are a bit different from the English ones. If the embedding weights are loaded, the pre-trained model is not able to be used, because of a shape mismatch, probably caused by the symbols difference. Could you, please, share your train.py file?