Tacotron2: I have trained the tacotron2 model, does waveglow need to be retrained?
dataset ： 48khz 16bit pcm chinese
hparams：
################################
# Experiment Parameters #
################################
epochs=500,
iters_per_checkpoint=1000,
seed=1234,
dynamic_loss_scaling=True,
fp16_run=False,
distributed_run=False,
dist_backend="nccl",
dist_url="tcp://localhost:54321",
cudnn_enabled=True,
cudnn_benchmark=False,
ignore_layers=['embedding.weight'],
################################
# Data Parameters #
################################
load_mel_from_disk=False,
#training_files='filelists/ljs_audio_text_train_filelist.txt',
#validation_files='filelists/ljs_audio_text_val_filelist.txt',
training_files='filelists/bznsyp_audio_text_train_filelist.txt',
validation_files='filelists/bznsyp_audio_text_valid_filelist.txt',
text_cleaners=['chinese_cleaners'],
################################
# Audio Parameters #
################################
max_wav_value=32768.0,
sampling_rate=48000,#22050
filter_length=1024,
hop_length=256,
win_length=1024,
n_mel_channels=80,
mel_fmin=0.0,
mel_fmax=8000.0,
################################
# Model Parameters #
################################
n_symbols=35, #len(symbols),
symbols_embedding_dim=512,
# Encoder parameters
encoder_kernel_size=5,
encoder_n_convolutions=3,
encoder_embedding_dim=512,
# Decoder parameters
n_frames_per_step=1, # currently only 1 is supported
decoder_rnn_dim=1024,
prenet_dim=256,
max_decoder_steps=1000,
gate_threshold=0.5,
p_attention_dropout=0.1,
p_decoder_dropout=0.1,
# Attention parameters
attention_rnn_dim=1024,
attention_dim=128,
# Location Layer parameters
attention_location_n_filters=32,
attention_location_kernel_size=31,
# Mel-post processing network parameters
postnet_embedding_dim=512,
postnet_kernel_size=5,
postnet_n_convolutions=5,
################################
# Optimization Hyperparameters #
################################
use_saved_learning_rate=False,
learning_rate=1e-3,
weight_decay=1e-6,
grad_clip_thresh=1.0,
batch_size=8,
mask_padding=True # set model's padded outputs to padded values
hugangyongxiangsha
All 10 comments
This repo provides pretrained waveglow model which works fine with female speaker, but its sample size is 22khz. You either downsample your dataset and retrain tacotron or train waveglow from scratch with 48k.
ksaidin
on 27 May 2020
how to re-train wave-glow model using pre-trained wave-glow model?
shwetagargade216
on 18 Jun 2020
@shwetagargade216 , like this.
ksaidin
on 18 Jun 2020
dataset ： 48khz 16bit pcm chinese
hparams：#
Experiment Parameters
#
epochs=500,
iters_per_checkpoint=1000,
seed=1234,
dynamic_loss_scaling=True,
fp16_run=False,
distributed_run=False,
dist_backend="nccl",
dist_url="tcp://localhost:54321",
cudnn_enabled=True,
cudnn_benchmark=False,
ignore_layers=['embedding.weight'],################################ # Data Parameters # ################################ load_mel_from_disk=False, #training_files='filelists/ljs_audio_text_train_filelist.txt', #validation_files='filelists/ljs_audio_text_val_filelist.txt', training_files='filelists/bznsyp_audio_text_train_filelist.txt', validation_files='filelists/bznsyp_audio_text_valid_filelist.txt', text_cleaners=['chinese_cleaners'], ################################ # Audio Parameters # ################################ max_wav_value=32768.0, sampling_rate=48000,#22050 filter_length=1024, hop_length=256, win_length=1024, n_mel_channels=80, mel_fmin=0.0, mel_fmax=8000.0, ################################ # Model Parameters # ################################ n_symbols=35, #len(symbols), symbols_embedding_dim=512, # Encoder parameters encoder_kernel_size=5, encoder_n_convolutions=3, encoder_embedding_dim=512, # Decoder parameters n_frames_per_step=1, # currently only 1 is supported decoder_rnn_dim=1024, prenet_dim=256, max_decoder_steps=1000, gate_threshold=0.5, p_attention_dropout=0.1, p_decoder_dropout=0.1, # Attention parameters attention_rnn_dim=1024, attention_dim=128, # Location Layer parameters attention_location_n_filters=32, attention_location_kernel_size=31, # Mel-post processing network parameters postnet_embedding_dim=512, postnet_kernel_size=5, postnet_n_convolutions=5, ################################ # Optimization Hyperparameters # ################################ use_saved_learning_rate=False, learning_rate=1e-3, weight_decay=1e-6, grad_clip_thresh=1.0, batch_size=8, mask_padding=True # set model's padded outputs to padded values
Did you have any luck doing this? I am trying to train with the same dataset as well, and am figuring out the right parameters to use for sampling rate of 48kHz. I assume it should be possible to use the pretrained Taco2 model and fine-tune it for 48kHz params, but waveglow might have to be trained from scratch if the channels related config have to be changed.
ricktjwong
on 22 Jun 2020
If the sampling rate is not 22050 Waveglow, or any vocoder available today, will need to be re-trained.
rafaelvalle
on 25 Jun 2020
If the sampling rate is not 22050 Waveglow, or any vocoder availabel today, will need to be re-trained.
@rafaelvalle Thanks! Do you know what these params have to be for 48Khz sample rate? I think win_length should be 2400 (for a 50ms window size) and hop_length should be 600 (win_length/4). What about the rest of the params?
filter_length=1024,
hop_length=256,
win_length=1024,
n_mel_channels=80,
mel_fmin=0.0,
mel_fmax=8000.0
I assume all audio params should be a function of sample rate (except max_wav_value which is a function of bit depth).
ricktjwong
on 25 Jun 2020
Take a look at this issue.
rafaelvalle
on 25 Jun 2020
@ksaidin How to downsample audios from 22050 to 8000?
Is there any good approach to this?
shwetagargade216
on 29 Jun 2020
Try using sox or ffmpeg.
rafaelvalle
on 29 Jun 2020
@hugangyongxiangsha Hi, i'm using tacotron2 to train mandarin too, but i can not get a good model file, what's your label? pinyin? and what your symbols? can you help me? thx a lot
HandsLing
on 7 Jul 2020
Related issues
hadaev8
·
6Comments
deadskull7
·
5Comments
lsuperman
·
4Comments
xDuck
·
4Comments
Mr-DDDAlKilanny
·
3Comments
Most helpful comment
This repo provides pretrained waveglow model which works fine with female speaker, but its sample size is 22khz. You either downsample your dataset and retrain tacotron or train waveglow from scratch with 48k.