Tacotron2: Loading spectrogram from disk and attention isn't being learned

[xDuck]

I am using the load_mel_from_disk option to load spectrograms I generated manually (n_mels=20, values range from about +10 to about -10) but I am not seeing the alignment get better as training goes on even though the predicted spectrogram looks nearly identical to the target (Loss < 0.15 after 72k iterations). I appended silence to the start and end of my spectrogram to see if that helps but it doesn't seem to have much effect.

My alignment graph is pretty much entirely dark (~0.00) except for a small bit in the bottom right corner (encoder = ~0 and decoder = ~800). During training it develops kind of horizontally before disappearing.

Also, I tried to use the model for inference but got essentially garbage even though during training and validation the Tensorboard spectrograms looked really good.

Is there anything I can try to get it to learn alignments? Also, what value do you guys use to pad the spectrograms with silence?

EDIT: After changing the value I use for silence in the spectrograms to -11.5 and changing the hop_length to match exactly what my spectrogram is using, I am now getting a faint horizontal line for my attention. Here is my attention plot after 1 iteration. I forgot to save the plot at 72k iterations but I'll post that when I get there again.

[xDuck]

FYI - Regenerated spectrograms with modified silence, adjusted learning rate, ran it a bit longer and its now aligning properly.

[rafaelvalle]

Great to hear!

[kannadaraj]

@xDuck . thanks.. I also encounter the same.. Can you say what adjustment you made for learning rate? Did you decrease it and by how much? For silence modification did you add silence at the end?

[xDuck]

@xDuck . thanks.. I also encounter the same.. Can you say what adjustment you made for learning rate? Did you decrease it and by how much? For silence modification did you add silence at the end?

I don't remember exactly what kind of learning rate I used but I increased it significantly. My fork is also significantly modified so I wouldn't recommend using the same parameters I used. I added 150ms and 300ms of silence to the front and back respectively as well as ensured that all my sentences had end of sentence tokens (period).