Tensorflowtts: Double Decoder Consistency for Tacotron 2

[ZDisket]

Hi, I found this interesting post while regularly checking up on erogol's blog. Any chances of having it here? Having a Tacotron 2 that is practically immune to alignment problems sounds very state of the art.
For reference, Mozilla/TTS has it.

[dathudeptrai]

@ZDisket training tacotron-2 take too much time for me but i will consider. There are many paper solve alignment problem and training very fast such as glow-tts, flow-tts, aligntts. I don't think we really need tacotron-2 because fastspeech2 can produce comparable results and alignTTS can learn very good alginment with serveral hours. So the question is, if we can find a model can be inference real-time, training time < 1 day and learn very good alignment with serveral hours, do we need tacotron-2 anymore ?. I'm focus about flow-tts and alignTTS now.

[ZDisket]

@dathudeptrai I know that too. If you get any of these working well (or FastSpeech2 without teacher model) then just ignore my suggestion.

[dathudeptrai]

Yeah :))) fastspeech2 without any external aligner is comming :))) be patient

[dathudeptrai]

BTW, to: @erogol is "Double Decoder" training slower than original Tacotron-2 ? cc: @ZDisket

[erogol]

@dathudeptrai it makes training only for 5% slower per iteration but faster to converge. And it definitely is worth it. It solves to attention problems once for all at least for my runs.

The problem with FastSpeech it requires initial alignments (from an aligner or a teacher model). Therefore to make the whole process easier and more accessible to everyone, I still believe targeting Tacotron like attention models is a worthy effort.

Even you like to train FastSpeech you can use DDC to generate even better attention alignments labels for your FS training.

[erogol]

Yeah :))) fastspeech2 without any external aligner is comming :))) be patient

you mean align-TTS?

[dathudeptrai]

@erogol flow-tts :D

[dathudeptrai]

@ZDisket i will read erogol's blog then i will consider to implement here but maybe you need training by urself :(. Tell me if it's ok ;v

[erogol]

I can try a PR but I need at least one day being 48 hours :)

[dathudeptrai]

@erogol what do you mean :)), one day cann't being 48 hours :))).

[erogol]

not in Mars :)

[dathudeptrai]

@erogol decoder (fine) is still 2 LSTM layers as normal and decoder (coarse) is 1 LSTM layers, am i right?

[erogol]

no the architecture is the same bur reduction rate is larger. But its a good idea to use even less layers.

[ZDisket]

@dathudeptrai I can train it from scratch myself although I estimate it'll take a week to get to a usable state unless DDC speeds it up a ton.