Tacotron2: You can use the pretrained LJSpeech waveglow for any language. It will even work for a male voice.

[khushbu-mulani]

You can use the pretrained LJSpeech waveglow for any language. It will even work for a male voice.

_Originally posted by @tugstugi in https://github.com/NVIDIA/tacotron2/issues/158#issuecomment-469281376_

@tugstugi How do I configure male voice for pretrained Tacotron2 and WaveGlow models?
Or do I need to train another model using a pre-trained model with different dataset(for male) altogether?

Got it working for the default female voice using the pretrained models. I am pretty new to this and really need to understand what needs to be done next. Just looking for male voice for English language.

Thanks in advance.

[EuphoriaCelestial]

I dont think you can do that, you must train a new model with male voice dataset.

[rafaelvalle]

WaveGlow performs rather well on unseen speakers and languages.
Try using this WaveGlow and check if the results are good enough for you. Otherwise fine-tune it on your data.

[ksaidin]

WaveGlow performs rather well on unseen speakers and languages.
Try using this WaveGlow and check if the results are good enough for you. Otherwise fine-tune it on your data.

@rafaelvalle , and we end up with : AttributeError: 'WN' object has no attribute 'cond_layers'

I tried : python convert_model.py waveglow_256channels_v4.pt waveglow_256channels_ljs_v4.pt as it is suggested in https://github.com/NVIDIA/waveglow/issues/154#issuecomment-531566024

That didn't work, still getting AttributeError.

[rafaelvalle]

https://github.com/NVIDIA/waveglow/blob/master/convert_model.py

On Tue, Apr 14, 2020 at 7:11 PM KSaidin [e-mail] wrote:

WaveGlow performs rather well on unseen speakers and languages.
Try using this WaveGlow
https://drive.google.com/open?id=1Rm5rV5XaWWiUbIpg5385l5sh68z2bVOE and
check if the results are good enough for you. Otherwise fine-tune it on
your data.

@rafaelvalle https://github.com/rafaelvalle , and we end up with :
AttributeError: 'WN' object has no attribute 'cond_layers'

I tried : python convert_model.py waveglow_256channels_v4.pt
waveglow_256channels_ljs_v4.pt as it is suggested in NVIDIA/waveglow#154
(comment)
https://github.com/NVIDIA/waveglow/issues/154#issuecomment-531566024

That didn't work, still getting AttributeError.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NVIDIA/tacotron2/issues/333#issuecomment-613774631,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AARSFD3JVUTKDVPWOJKM3EDRMUJU7ANCNFSM4MG6MNPA
.

[ksaidin]

v2 is working fine.

This already converted WaveGlow v3 is not working: AttributeError.

WaveGlow v4 is not working after converted by @rafaelvalle suggested convert model: yet again AttributeError.

[CookiePPP]

Tacotron2 is using a slightly older glow.py by default.
If you want to use WaveGlow v4 (and v3(?) ) then update the Tacotron2/waveglow with the latest WaveGlow

---

https://gthub.com/NVIDIA/waveglow/blob/4b1001fa3336a1184b8293745bb89b177457f09b/glow.py#L120

        self.cond_layers = torch.nn.ModuleList()
...
        for i in range(n_layers):
...
            cond_layer = torch.nn.Conv1d(n_mel_channels, 2*n_channels, 1)
            cond_layer = torch.nn.utils.weight_norm(cond_layer, name='weight')
            self.cond_layers.append(cond_layer)

---

verses the updated version.
https://github.com/NVIDIA/waveglow/blob/master/glow.py#L133

        cond_layer = torch.nn.Conv1d(n_mel_channels, 2*n_channels*n_layers, 1)
        self.cond_layer = torch.nn.utils.weight_norm(cond_layer, name='weight')

---

[ksaidin]

Thank you @CookiePPP , it works for me now.

So basically it is OK to use version 4 of WaveGlow after updating from https://github.com/NVIDIA/waveglow and convert_model.

[EuphoriaCelestial]

https://github.com/NVIDIA/waveglow/blob/master/convert_model.py

this not work for me, I trained a new waveglow model but cant load the checkpoint into inference file, even after run convert_model.py

[ksaidin]

@EuphoriaCelestial , and error code is?

[EuphoriaCelestial]

@ksaidin AttributeError: 'WN' object has no attribute 'cond_layers'

[khushbu-mulani]

I trained tacotron2 model till 6000 iterations as of now on m-ailab male voice dataset, later using the pretrained waveglow model as suggested in the above comments for speech synthesis. But this gives totally unacceptable results. The voice seems to be broken and echoing after synthesis.

How do I know if I am moving in the right direction? @rafaelvalle

[bolattleubayev]

Hi there,

I have issues training Tacotron 2 for Russian (20h dataset, https://www.caito.de/2019/01/the-m-ailabs-speech-dataset/) and Kazakh (a little less than 10h dataset, with +9 characters to Russian alphabet) languages.

I have tried Tacotron 1 for Kazakh dataset, and it worked fine, I could generate clear audio with clearly distinguishable words.

I have tried to rely on answers from issues section of this repo and tried all possible variations I could come up with.

1) I have tried to tune the filter_length, hop_length, win_length to conform to 16,000 Hz (743, 186, 743 respectively) and run from warm start for 170,000 iterations with batch_size 4 (setting it higher crashes the script as I run out of memory).

2) I also tried to upsample the dataset to 22,050 Hz and train for 420,000 iterations with batch_size 4 (Somewhere it was told to stop training at 30k, I considered that 30k at batch 64, so stopped at 420k for batch 4).

3) I tried to modify the attention and decoder dropout from 0.1 to 0.4, trained for around 100k at batch size 4-6 at different times. I have also modified the separately and together at once.

4) Also experimented with fixed and variable (don't remember formula now, but found it in one of the issues on this repo) learning rate.

5) Tried both to train from scratch and from warm start, the result is always pretty much the same

6) In the beginning, I tried to come up with my own cleaners, but finally decided to use only basic_cleaners.

For every dataset I used different characters set and used basic_cleaners on all runs and modifications stated above. I also converted the given waveglow model to a new model with script. The loss usually converges to something around 0.02

On almost all outputs I only got a long beep sound. I definitely doing something wrong, please help me to figure it out

Usually I had output as shown below:

[ksaidin]

@bolattleubayev , I can confirm that you can count on answers from issues section of this repo.

You don't need to tune parameters in the beginning, if your dataset is correct, you will get somewhat acceptable results.

Check this one, it is more than enough to start with.

[bolattleubayev]

@ksaidin okay, I will try again, thanks!

[rafaelvalle]

Closing due to inactivity.