Tensorflowtts: Add support for German TTS

[monatis]

Hi all,
I searched for "German," and it didn't give any results, so I decided to open an issue to prevent duplicated work in this. Recently, I've just met an open source dataset for TTS in German by @thorstenMueller, and it sounds pretty cool to train a TensorFlowTTS in German by using this dataset. Anyone interested?

p.s.:I'm also working on a dataset in Turkish to release as open source, but it will take some more time.

[machineko]

There is a lot of free German datasets why using this I would probably prefer to first train multi-speaker model on M-AILABS

[domcross]

Your mileage may vary, but to my experience the german M-AILABS datasets are not feasible for TTS training, at least without heavy pre-processing. A lot of audio files are not exactly trimmed to the word ends, e.g. cutting of the last syllable.

[dathudeptrai]

Hi all,
I searched for "German," and it didn't give any results, so I decided to open an issue to prevent duplicated work in this. Recently, I've just met an open source dataset for TTS in German by @thorstenMueller, and it sounds pretty cool to train a TensorFlowTTS in German by using this dataset. Anyone interested?

p.s.:I'm also working on a dataset in Turkish to release as open source, but it will take some more time.

you want to support German to this framework or you are requesting support it ? :)).

[monatis]

Hi @dathudeptrai, I'll give it a try to train German TTS with this dataset, and the issue is all about preventing us from duplicated work, i.e., I just wanted anyone with this idea to see someone is working on it. You can properly mark/label it.

[dathudeptrai]

@monatis thanks :D. looking forward german TTS :D. do you want me add other label for this issue, eg: help wanted :)).

[manmay-nakhashi]

@monatis working on all European languages.

[thorstenMueller]

Just in case it's interesting for you:
With support of @domcross i've released a version 02 of "thorsten" dataset which is normalized to -24dB. Both download links (dataset v01 and v02) are available on my github page (https://github.com/thorstenMueller/deep-learning-german-tts/)

[thorstenMueller]

We’ve made a first taco2 model draft based on dataset v02.
Taco2 training for 460k steps and PwGAN vocoder with 925k training steps.

Some audio sample „teasers“ are available here:
https://community.mycroft.ai/t/free-german-tts-voice-for-mycroft-sneak-preview/9458?u=thorsten

[thorstenMueller]

Hi @monatis and @dathudeptrai .
Just being curious if there's any updates.

Personally i'm playing around with WaveGrad training on my public "thorsten" dataset.

[dathudeptrai]

@monatis :D

[monatis]

hi @dathudeptrai and @thorstenMueller, I was busy with some commercial projects for some time, but I'll be able to play a little bit with Thorsten dataset later this week, thinking that I can use some TFRC credits for the compute power required to train a German model with it. Updates will come as I make progress.

[monatis]

I added support for German with Thorsten dataset in my fork, and I'm training multiband_melgan now. After it's finished as well as Tacotron 2, I'll make a PR.

[monatis]

I'm sharing a quick preview with multiband_melgan at 820k steps and tacotron2 at 81.5k steps. I will continue to train both of them till they converge around 950k and 100k steps respectively, but it already sounds like a decent model to my non-native ear. Here is the Colab with inference code. @thorstenMueller, any comments?

[thorstenMueller]

Hi @monatis .
First of all a huge thanks to you for your effort 🥇
For comparison reasons i put together a github page with samples from different vocoders. I’ve generated a few samples with the notebook you provided and put them there. I hope that’s okay for you, if not i’ll remove them.

https://thorstenmueller.github.io/deep-learning-german-tts/audio_compare#tensorflowtts

For your training steps it sounds quite useable.

• generation speed is fast
• Sounds a little metalic (like all GAN vocoders)
• Some german umlauts (not all) are making trouble (maybe more training steps required)

And again - many thanks 👍 .

[monatis]

Hi @thorstenMueller. Thanks a lot for your efforts to prepare audio samples and share them publically --it's totally fine for me.

I will let you know if I can improve the quality with more training steps. Now it's time to make PR to the main repo :D

[monatis]

I exported models to saved_model and TF Lite formats, so it became independent from the TensorFlowTTS package during inference. See my repo for exported model artifacts and inference examples.