Tacotron-2: How to implement ground truth-aligned predictions??

[Yeongtae]

To train wave generator such as wavenet, waveRNN, and waveglow,
We need to generate GTA mel-spectrograms from a dataset.

We can see the detailed explanation in tacotron2 paper,

But There are Insufficient information to implement.
What is the the corresponding previous frame?
is it just linear projection value? to feed 2 layer pre-net? or not?

[Rayhane-mamah]

It is already implemented and can be turned on by using
"train_with_GTA=True" in hparams.
It will requirw that you do GTA synthesis with Tacotron before training
wavenet

On Thu, 22 Nov 2018, 08:59 Yeongtae <[email protected] wrote:

To train wave generator such as wavenet, waveRNN, and waveglow.
We need to extract GTA mel-spectrograms from a dataset.

We can see the detailed explanation in tacotron2 paper,
[image: image]
https://user-images.githubusercontent.com/13597229/48889042-e973cf00-ee77-11e8-897a-f84ffd27507c.png

But There are Insufficient information to implement.
What is the the corresponding previous frame?
is it just linear projection value? to feed 2 layer pre-net? or not?
[image: image]
https://user-images.githubusercontent.com/13597229/48889056-f395cd80-ee77-11e8-9a37-d0522bec9242.png

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Rayhane-mamah/Tacotron-2/issues/275, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AhFSwFeUsB--WG3P6UV_DspUNR56NjvOks5uxllrgaJpZM4Yu1Iq
.

[Yeongtae]

I know, but I would like to understand the detail.

Though I'm looking at your code, it's hard to understand how to feed and replace corresponding previous frame from ground truth.

[Rayhane-mamah]

For GTA you don't use previous model outputs, you feed back the True Mel
targets just like training (teacher forcing) thus the name Ground Truth
Aligned.

So we use mel targets to ensure tht the attention mechanism will follow the
groud truth case.
We do that so that the model does not produce a sentence smaller than
expected and result in shape mismatch in wavenet. I.e: wavenet expects
mel_frames = wav_samples / hop_size.

Long story short, GTA test case is exactly training without dropout/zoneout
and back-propagation.

Extra:
When r>1, we ensure that mel targets are padded to fit the reduction
factor, and we trim the outputs after that to ensure once again n_frames =
wav_samples / hop_size for wavenet.

On Fri, 23 Nov 2018, 02:01 Yeongtae <[email protected] wrote:

I know, but I would like to understand the detail.

Though I'm looking at your code, it's hard to understand how to feed and
replace corresponding previous frame from ground truth.

[image: image]
https://user-images.githubusercontent.com/13597229/48925416-add41600-ef06-11e8-8117-79ce149d9ef3.png

[image: image]
https://user-images.githubusercontent.com/13597229/48925419-b62c5100-ef06-11e8-9c4c-14e1d84db82b.png

[image: image]
https://user-images.githubusercontent.com/13597229/48925427-be848c00-ef06-11e8-8df7-bd6d49332fb0.png

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Rayhane-mamah/Tacotron-2/issues/275#issuecomment-441139339,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhFSwJZDFeQFOvwseuoRk-JWpV-QMCgAks5ux0j0gaJpZM4Yu1Iq
.