Tts: RuntimeError: Error(s) in loading state_dict for Tacotron

Created on 30 May 2019  路  18Comments  路  Source: mozilla/TTS

I'm trying to generate speech from the latest checkpoint. I did

  • Checkout this project
  • Download "Tacotron2-iter-260K" checkpoint "checkpoint_261000.pth.tar" and "config.json" from table https://github.com/mozilla/TTS#checkpoints-and-audio-samples
  • Checkout git commit "824c091" from same table
  • Install with python setup.py develop
  • Modify "server/conf.json" to point on the two files from step 1
  • Start the server with python server/server.py -c server/conf.json

I get the following error

AttributeError: 'AttrDict' object has no attribute 'embedding_size'

Which is easy to fix with embedding_size: "256", then I get:

python server/server.py -c server/conf.json
 > Loading model ...
 | > model config:  /media/alex/data/project/dreambank/models/Tacotron2-iter-260K/ljspeech-April-08-2019_07+32PM-8a47b46/config.json
 | > model file:  /media/alex/data/project/dreambank/models/Tacotron2-iter-260K/ljspeech-April-08-2019_07+32PM-8a47b46/checkpoint_261000.pth.tar
 > Setting up Audio Processor...
 | > fft size: 2048, hop length: 275, win length: 1102
 | > Audio Processor attributes.
   | > bits:None
   | > sample_rate:22050
   | > num_mels:80
   | > min_level_db:-100
   | > frame_shift_ms:12.5
   | > frame_length_ms:50
   | > ref_level_db:20
   | > num_freq:1025
   | > power:1.5
   | > preemphasis:0.98
   | > griffin_lim_iters:60
   | > signal_norm:True
   | > symmetric_norm:False
   | > mel_fmin:0.0
   | > mel_fmax:8000.0
   | > max_norm:1.0
   | > clip_norm:True
   | > do_trim_silence:True
   | > n_fft:2048
   | > hop_length:275
   | > win_length:1102
 | > Number of characters : 61
Traceback (most recent call last):
  File "server/server.py", line 16, in <module>
    config.model_config, config.use_cuda)
  File "/home/alex/Project/TTS/server/synthesizer.py", line 34, in load_model
    self.model.load_state_dict(cp['model'])
  File "/home/alex/miniconda3/lib/python3.7/site-packages/torch-1.1.0-py3.7-linux-x86_64.egg/torch/nn/modules/module.py", line 777, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Tacotron:
    Missing key(s) in state_dict: "encoder.prenet.layers.0.weight", "encoder.prenet.layers.0.bias", "encoder.prenet.layers.1.weight", "encoder.prenet.layers.1.bias", "encoder.cbhg.cbhg.conv1d_banks.0.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.0.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.0.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.0.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.0.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.1.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.1.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.1.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.1.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.1.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.2.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.2.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.2.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.2.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.2.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.3.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.3.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.3.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.3.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.3.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.4.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.4.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.4.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.4.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.4.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.5.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.5.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.5.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.5.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.5.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.6.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.6.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.6.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.6.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.6.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.7.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.7.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.7.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.7.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.7.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.8.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.8.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.8.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.8.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.8.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.9.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.9.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.9.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.9.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.9.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.10.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.10.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.10.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.10.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.10.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.11.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.11.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.11.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.11.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.11.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.12.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.12.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.12.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.12.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.12.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.13.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.13.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.13.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.13.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.13.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.14.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.14.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.14.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.14.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.14.bn.running_var", "encoder.cbhg.cbhg.conv1d_banks.15.conv1d.weight", "encoder.cbhg.cbhg.conv1d_banks.15.bn.weight", "encoder.cbhg.cbhg.conv1d_banks.15.bn.bias", "encoder.cbhg.cbhg.conv1d_banks.15.bn.running_mean", "encoder.cbhg.cbhg.conv1d_banks.15.bn.running_var", "encoder.cbhg.cbhg.conv1d_projections.0.conv1d.weight", "encoder.cbhg.cbhg.conv1d_projections.0.bn.weight", "encoder.cbhg.cbhg.conv1d_projections.0.bn.bias", "encoder.cbhg.cbhg.conv1d_projections.0.bn.running_mean", "encoder.cbhg.cbhg.conv1d_projections.0.bn.running_var", "encoder.cbhg.cbhg.conv1d_projections.1.conv1d.weight", "encoder.cbhg.cbhg.conv1d_projections.1.bn.weight", "encoder.cbhg.cbhg.conv1d_projections.1.bn.bias", "encoder.cbhg.cbhg.conv1d_projections.1.bn.running_mean", "encoder.cbhg.cbhg.conv1d_projections.1.bn.running_var", "encoder.cbhg.cbhg.highways.0.H.weight", "encoder.cbhg.cbhg.highways.0.H.bias", "encoder.cbhg.cbhg.highways.0.T.weight", "encoder.cbhg.cbhg.highways.0.T.bias", "encoder.cbhg.cbhg.highways.1.H.weight", "encoder.cbhg.cbhg.highways.1.H.bias", "encoder.cbhg.cbhg.highways.1.T.weight", "encoder.cbhg.cbhg.highways.1.T.bias", "encoder.cbhg.cbhg.highways.2.H.weight", "encoder.cbhg.cbhg.highways.2.H.bias", "encoder.cbhg.cbhg.highways.2.T.weight", "encoder.cbhg.cbhg.highways.2.T.bias", "encoder.cbhg.cbhg.highways.3.H.weight", "encoder.cbhg.cbhg.highways.3.H.bias", "encoder.cbhg.cbhg.highways.3.T.weight", "encoder.cbhg.cbhg.highways.3.T.bias", "encoder.cbhg.cbhg.gru.weight_ih_l0", "encoder.cbhg.cbhg.gru.weight_hh_l0", "encoder.cbhg.cbhg.gru.bias_ih_l0", "encoder.cbhg.cbhg.gru.bias_hh_l0", "encoder.cbhg.cbhg.gru.weight_ih_l0_reverse", "encoder.cbhg.cbhg.gru.weight_hh_l0_reverse", "encoder.cbhg.cbhg.gru.bias_ih_l0_reverse", "encoder.cbhg.cbhg.gru.bias_hh_l0_reverse", "decoder.prenet.layers.0.weight", "decoder.prenet.layers.0.bias", "decoder.prenet.layers.1.weight", "decoder.prenet.layers.1.bias", "decoder.attention_rnn.rnn_cell.weight_ih", "decoder.attention_rnn.rnn_cell.weight_hh", "decoder.attention_rnn.rnn_cell.bias_ih", "decoder.attention_rnn.rnn_cell.bias_hh", "decoder.attention_rnn.alignment_model.loc_conv.1.weight", "decoder.attention_rnn.alignment_model.loc_linear.weight", "decoder.attention_rnn.alignment_model.loc_linear.bias", "decoder.attention_rnn.alignment_model.query_layer.weight", "decoder.attention_rnn.alignment_model.query_layer.bias", "decoder.attention_rnn.alignment_model.annot_layer.weight", "decoder.attention_rnn.alignment_model.annot_layer.bias", "decoder.attention_rnn.alignment_model.v.weight", "decoder.project_to_decoder_in.weight", "decoder.project_to_decoder_in.bias", "decoder.decoder_rnns.0.weight_ih", "decoder.decoder_rnns.0.weight_hh", "decoder.decoder_rnns.0.bias_ih", "decoder.decoder_rnns.0.bias_hh", "decoder.decoder_rnns.1.weight_ih", "decoder.decoder_rnns.1.weight_hh", "decoder.decoder_rnns.1.bias_ih", "decoder.decoder_rnns.1.bias_hh", "decoder.proj_to_mel.weight", "decoder.proj_to_mel.bias", "decoder.memory_init.weight", "decoder.stopnet.linear.weight", "decoder.stopnet.linear.bias", "postnet.cbhg.conv1d_banks.0.conv1d.weight", "postnet.cbhg.conv1d_banks.0.bn.weight", "postnet.cbhg.conv1d_banks.0.bn.bias", "postnet.cbhg.conv1d_banks.0.bn.running_mean", "postnet.cbhg.conv1d_banks.0.bn.running_var", "postnet.cbhg.conv1d_banks.1.conv1d.weight", "postnet.cbhg.conv1d_banks.1.bn.weight", "postnet.cbhg.conv1d_banks.1.bn.bias", "postnet.cbhg.conv1d_banks.1.bn.running_mean", "postnet.cbhg.conv1d_banks.1.bn.running_var", "postnet.cbhg.conv1d_banks.2.conv1d.weight", "postnet.cbhg.conv1d_banks.2.bn.weight", "postnet.cbhg.conv1d_banks.2.bn.bias", "postnet.cbhg.conv1d_banks.2.bn.running_mean", "postnet.cbhg.conv1d_banks.2.bn.running_var", "postnet.cbhg.conv1d_banks.3.conv1d.weight", "postnet.cbhg.conv1d_banks.3.bn.weight", "postnet.cbhg.conv1d_banks.3.bn.bias", "postnet.cbhg.conv1d_banks.3.bn.running_mean", "postnet.cbhg.conv1d_banks.3.bn.running_var", "postnet.cbhg.conv1d_banks.4.conv1d.weight", "postnet.cbhg.conv1d_banks.4.bn.weight", "postnet.cbhg.conv1d_banks.4.bn.bias", "postnet.cbhg.conv1d_banks.4.bn.running_mean", "postnet.cbhg.conv1d_banks.4.bn.running_var", "postnet.cbhg.conv1d_banks.5.conv1d.weight", "postnet.cbhg.conv1d_banks.5.bn.weight", "postnet.cbhg.conv1d_banks.5.bn.bias", "postnet.cbhg.conv1d_banks.5.bn.running_mean", "postnet.cbhg.conv1d_banks.5.bn.running_var", "postnet.cbhg.conv1d_banks.6.conv1d.weight", "postnet.cbhg.conv1d_banks.6.bn.weight", "postnet.cbhg.conv1d_banks.6.bn.bias", "postnet.cbhg.conv1d_banks.6.bn.running_mean", "postnet.cbhg.conv1d_banks.6.bn.running_var", "postnet.cbhg.conv1d_banks.7.conv1d.weight", "postnet.cbhg.conv1d_banks.7.bn.weight", "postnet.cbhg.conv1d_banks.7.bn.bias", "postnet.cbhg.conv1d_banks.7.bn.running_mean", "postnet.cbhg.conv1d_banks.7.bn.running_var", "postnet.cbhg.conv1d_projections.0.conv1d.weight", "postnet.cbhg.conv1d_projections.0.bn.weight", "postnet.cbhg.conv1d_projections.0.bn.bias", "postnet.cbhg.conv1d_projections.0.bn.running_mean", "postnet.cbhg.conv1d_projections.0.bn.running_var", "postnet.cbhg.conv1d_projections.1.conv1d.weight", "postnet.cbhg.conv1d_projections.1.bn.weight", "postnet.cbhg.conv1d_projections.1.bn.bias", "postnet.cbhg.conv1d_projections.1.bn.running_mean", "postnet.cbhg.conv1d_projections.1.bn.running_var", "postnet.cbhg.pre_highway.weight", "postnet.cbhg.highways.0.H.weight", "postnet.cbhg.highways.0.H.bias", "postnet.cbhg.highways.0.T.weight", "postnet.cbhg.highways.0.T.bias", "postnet.cbhg.highways.1.H.weight", "postnet.cbhg.highways.1.H.bias", "postnet.cbhg.highways.1.T.weight", "postnet.cbhg.highways.1.T.bias", "postnet.cbhg.highways.2.H.weight", "postnet.cbhg.highways.2.H.bias", "postnet.cbhg.highways.2.T.weight", "postnet.cbhg.highways.2.T.bias", "postnet.cbhg.highways.3.H.weight", "postnet.cbhg.highways.3.H.bias", "postnet.cbhg.highways.3.T.weight", "postnet.cbhg.highways.3.T.bias", "postnet.cbhg.gru.weight_ih_l0", "postnet.cbhg.gru.weight_hh_l0", "postnet.cbhg.gru.bias_ih_l0", "postnet.cbhg.gru.bias_hh_l0", "postnet.cbhg.gru.weight_ih_l0_reverse", "postnet.cbhg.gru.weight_hh_l0_reverse", "postnet.cbhg.gru.bias_ih_l0_reverse", "postnet.cbhg.gru.bias_hh_l0_reverse", "last_linear.0.weight", "last_linear.0.bias". 
    Unexpected key(s) in state_dict: "encoder.convolutions.0.net.0.weight", "encoder.convolutions.0.net.0.bias", "encoder.convolutions.0.net.1.weight", "encoder.convolutions.0.net.1.bias", "encoder.convolutions.0.net.1.running_mean", "encoder.convolutions.0.net.1.running_var", "encoder.convolutions.0.net.1.num_batches_tracked", "encoder.convolutions.1.net.0.weight", "encoder.convolutions.1.net.0.bias", "encoder.convolutions.1.net.1.weight", "encoder.convolutions.1.net.1.bias", "encoder.convolutions.1.net.1.running_mean", "encoder.convolutions.1.net.1.running_var", "encoder.convolutions.1.net.1.num_batches_tracked", "encoder.convolutions.2.net.0.weight", "encoder.convolutions.2.net.0.bias", "encoder.convolutions.2.net.1.weight", "encoder.convolutions.2.net.1.bias", "encoder.convolutions.2.net.1.running_mean", "encoder.convolutions.2.net.1.running_var", "encoder.convolutions.2.net.1.num_batches_tracked", "encoder.lstm.weight_ih_l0", "encoder.lstm.weight_hh_l0", "encoder.lstm.bias_ih_l0", "encoder.lstm.bias_hh_l0", "encoder.lstm.weight_ih_l0_reverse", "encoder.lstm.weight_hh_l0_reverse", "encoder.lstm.bias_ih_l0_reverse", "encoder.lstm.bias_hh_l0_reverse", "decoder.attention_layer.query_layer.linear_layer.weight", "decoder.attention_layer.inputs_layer.linear_layer.weight", "decoder.attention_layer.v.linear_layer.weight", "decoder.attention_layer.v.linear_layer.bias", "decoder.attention_layer.location_layer.location_conv.weight", "decoder.attention_layer.location_layer.location_dense.linear_layer.weight", "decoder.decoder_rnn.weight_ih", "decoder.decoder_rnn.weight_hh", "decoder.decoder_rnn.bias_ih", "decoder.decoder_rnn.bias_hh", "decoder.linear_projection.linear_layer.weight", "decoder.linear_projection.linear_layer.bias", "decoder.go_frame_init.weight", "decoder.prenet.layers.0.linear_layer.weight", "decoder.prenet.layers.0.bn.weight", "decoder.prenet.layers.0.bn.bias", "decoder.prenet.layers.0.bn.running_mean", "decoder.prenet.layers.0.bn.running_var", "decoder.prenet.layers.0.bn.num_batches_tracked", "decoder.prenet.layers.1.linear_layer.weight", "decoder.prenet.layers.1.bn.weight", "decoder.prenet.layers.1.bn.bias", "decoder.prenet.layers.1.bn.running_mean", "decoder.prenet.layers.1.bn.running_var", "decoder.prenet.layers.1.bn.num_batches_tracked", "decoder.attention_rnn.weight_ih", "decoder.attention_rnn.weight_hh", "decoder.attention_rnn.bias_ih", "decoder.attention_rnn.bias_hh", "decoder.stopnet.1.linear_layer.weight", "decoder.stopnet.1.linear_layer.bias", "postnet.convolutions.0.net.0.weight", "postnet.convolutions.0.net.0.bias", "postnet.convolutions.0.net.1.weight", "postnet.convolutions.0.net.1.bias", "postnet.convolutions.0.net.1.running_mean", "postnet.convolutions.0.net.1.running_var", "postnet.convolutions.0.net.1.num_batches_tracked", "postnet.convolutions.1.net.0.weight", "postnet.convolutions.1.net.0.bias", "postnet.convolutions.1.net.1.weight", "postnet.convolutions.1.net.1.bias", "postnet.convolutions.1.net.1.running_mean", "postnet.convolutions.1.net.1.running_var", "postnet.convolutions.1.net.1.num_batches_tracked", "postnet.convolutions.2.net.0.weight", "postnet.convolutions.2.net.0.bias", "postnet.convolutions.2.net.1.weight", "postnet.convolutions.2.net.1.bias", "postnet.convolutions.2.net.1.running_mean", "postnet.convolutions.2.net.1.running_var", "postnet.convolutions.2.net.1.num_batches_tracked", "postnet.convolutions.3.net.0.weight", "postnet.convolutions.3.net.0.bias", "postnet.convolutions.3.net.1.weight", "postnet.convolutions.3.net.1.bias", "postnet.convolutions.3.net.1.running_mean", "postnet.convolutions.3.net.1.running_var", "postnet.convolutions.3.net.1.num_batches_tracked", "postnet.convolutions.4.net.0.weight", "postnet.convolutions.4.net.0.bias", "postnet.convolutions.4.net.1.weight", "postnet.convolutions.4.net.1.bias", "postnet.convolutions.4.net.1.running_mean", "postnet.convolutions.4.net.1.running_var", "postnet.convolutions.4.net.1.num_batches_tracked". 
    size mismatch for embedding.weight: copying a param with shape torch.Size([62, 512]) from checkpoint, the shape in current model is torch.Size([61, 256]).
    size mismatch for decoder.attention_rnn_init.weight: copying a param with shape torch.Size([1, 1024]) from checkpoint, the shape in current model is torch.Size([1, 256]).
    size mismatch for decoder.decoder_rnn_inits.weight: copying a param with shape torch.Size([1, 1024]) from checkpoint, the shape in current model is torch.Size([2, 256]).

So I tried using commit f08be38 (from branch remotes/origin/dev-tacotron2) and I get another error

AttributeError: 'AttrDict' object has no attribute 'tts_path'
picture dubreuia
🎉1 👍1

Most helpful comment

Now Tacotron2-iter-260K-824c091 is also fixed!

picture erogol on 15 Aug 2019
👍3

All 18 comments

Same with commit db7f3d3, similar problem:

 > Loading model ...
 | > model config:  /media/alex/data/project/dreambank/models/Tacotron-iter-185K/queue-February-16-2019_03+16AM-90f0cd6/config.json
 | > model file:  /media/alex/data/project/dreambank/models/Tacotron-iter-185K/queue-February-16-2019_03+16AM-90f0cd6/best_model.pth.tar
 > Setting up Audio Processor...
 | > fft size: 2048, hop length: 275, win length: 1102
 | > Audio Processor attributes.
   | > bits:None
   | > sample_rate:22050
   | > num_mels:80
   | > min_level_db:-100
   | > frame_shift_ms:12.5
   | > frame_length_ms:50
   | > ref_level_db:20
   | > num_freq:1025
   | > power:1.5
   | > preemphasis:0.98
   | > griffin_lim_iters:60
   | > signal_norm:True
   | > symmetric_norm:False
   | > mel_fmin:0
   | > mel_fmax:None
   | > max_norm:1.0
   | > clip_norm:True
   | > do_trim_silence:True
   | > n_fft:2048
   | > hop_length:275
   | > win_length:1102
 | > Number of characters : 256
Traceback (most recent call last):
  File "server/server.py", line 16, in <module>
    config.model_config, config.use_cuda)
  File "/home/alex/Project/TTS/server/synthesizer.py", line 34, in load_model
    self.model.load_state_dict(cp['model'])
  File "/home/alex/miniconda3/lib/python3.7/site-packages/torch-1.1.0-py3.7-linux-x86_64.egg/torch/nn/modules/module.py", line 777, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Tacotron:
    size mismatch for embedding.weight: copying a param with shape torch.Size([61, 256]) from checkpoint, the shape in current model is torch.Size([256, 1025]).
    size mismatch for encoder.prenet.layers.0.weight: copying a param with shape torch.Size([256, 256]) from checkpoint, the shape in current model is torch.Size([256, 1025]).
    size mismatch for decoder.prenet.layers.0.weight: copying a param with shape torch.Size([256, 400]) from checkpoint, the shape in current model is torch.Size([256, 10]).
    size mismatch for decoder.proj_to_mel.weight: copying a param with shape torch.Size([160, 256]) from checkpoint, the shape in current model is torch.Size([10, 256]).
    size mismatch for decoder.proj_to_mel.bias: copying a param with shape torch.Size([160]) from checkpoint, the shape in current model is torch.Size([10]).
    size mismatch for decoder.memory_init.weight: copying a param with shape torch.Size([1, 400]) from checkpoint, the shape in current model is torch.Size([1, 10]).
    size mismatch for decoder.stopnet.linear.weight: copying a param with shape torch.Size([1, 416]) from checkpoint, the shape in current model is torch.Size([1, 266]).
    size mismatch for postnet.cbhg.conv1d_banks.0.conv1d.weight: copying a param with shape torch.Size([128, 80, 1]) from checkpoint, the shape in current model is torch.Size([128, 2, 1]).
    size mismatch for postnet.cbhg.conv1d_banks.1.conv1d.weight: copying a param with shape torch.Size([128, 80, 2]) from checkpoint, the shape in current model is torch.Size([128, 2, 2]).
    size mismatch for postnet.cbhg.conv1d_banks.2.conv1d.weight: copying a param with shape torch.Size([128, 80, 3]) from checkpoint, the shape in current model is torch.Size([128, 2, 3]).
    size mismatch for postnet.cbhg.conv1d_banks.3.conv1d.weight: copying a param with shape torch.Size([128, 80, 4]) from checkpoint, the shape in current model is torch.Size([128, 2, 4]).
    size mismatch for postnet.cbhg.conv1d_banks.4.conv1d.weight: copying a param with shape torch.Size([128, 80, 5]) from checkpoint, the shape in current model is torch.Size([128, 2, 5]).
    size mismatch for postnet.cbhg.conv1d_banks.5.conv1d.weight: copying a param with shape torch.Size([128, 80, 6]) from checkpoint, the shape in current model is torch.Size([128, 2, 6]).
    size mismatch for postnet.cbhg.conv1d_banks.6.conv1d.weight: copying a param with shape torch.Size([128, 80, 7]) from checkpoint, the shape in current model is torch.Size([128, 2, 7]).
    size mismatch for postnet.cbhg.conv1d_banks.7.conv1d.weight: copying a param with shape torch.Size([128, 80, 8]) from checkpoint, the shape in current model is torch.Size([128, 2, 8]).
    size mismatch for postnet.cbhg.conv1d_projections.1.conv1d.weight: copying a param with shape torch.Size([80, 256, 3]) from checkpoint, the shape in current model is torch.Size([2, 256, 3]).
    size mismatch for postnet.cbhg.conv1d_projections.1.bn.weight: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([2]).
    size mismatch for postnet.cbhg.conv1d_projections.1.bn.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([2]).
    size mismatch for postnet.cbhg.conv1d_projections.1.bn.running_mean: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([2]).
    size mismatch for postnet.cbhg.conv1d_projections.1.bn.running_var: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([2]).
    size mismatch for postnet.cbhg.pre_highway.weight: copying a param with shape torch.Size([128, 80]) from checkpoint, the shape in current model is torch.Size([128, 2]).
    size mismatch for last_linear.0.weight: copying a param with shape torch.Size([1025, 256]) from checkpoint, the shape in current model is torch.Size([80, 256]).
    size mismatch for last_linear.0.bias: copying a param with shape torch.Size([1025]) from checkpoint, the shape in current model is torch.Size([80]).

Should be written as 2b970c5 in the table, do you want me to do a PR?

dubreuia picture dubreuia on 30 May 2019

Not sure on the first point, but I think your second comment is addressed here: https://github.com/mozilla/TTS/issues/190#issuecomment-491881439

nmstoker picture nmstoker on 30 May 2019

@dubreuia sir have you resolved that first one. i am facing same issue.

MuruganR96 picture MuruganR96 on 7 Jun 2019

@nmstoker Yes I was able to make that work thanks.

@MuruganR96 No I'm unable to make it work, but I haven't tried anything else since. I'm pretty sure the problem is that commit 824c091 isn't the proper one, but probably another commit (maybe in another branch) could work. I'll try other commits when I have time.

dubreuia picture dubreuia on 13 Jun 2019
👍1

Does anybody make this work?
I'm getting the same errors.

DoroshAlex picture DoroshAlex on 14 Aug 2019

@a-dorosh It's half working. From table https://github.com/mozilla/TTS/wiki/Released-Models#english, the model Tacotron2-iter-260K with commit 824c091 definitely do not work, I've tried various commits in branches to no avail. The Tacotron-iter-185K model works but not with the commit db7f3d3 from the table, you need to use 2b970c5.

@erogol What's your take on this? I can update the table with the good commit for one of the model but I can't make it work for the other.

dubreuia picture dubreuia on 14 Aug 2019

I fixed server.py for Tacotron-185K benchmark notebook was already working. Checkout the new branch on the model table.

erogol picture erogol on 15 Aug 2019
👍2

Now Tacotron2-iter-260K-824c091 is also fixed!

erogol picture erogol on 15 Aug 2019
👍3

Great thank you @erogol

dubreuia picture dubreuia on 19 Aug 2019

Hello, unfortunately I'm unable to make it work:

git checkout -b Tacotron2-iter-260K-824c091 origin/Tacotron2-iter-260K-824c091
python server/server.py -c server/conf.json
#  > Loading model ...
#  | > model config:  C:\Users\Alex\Projects\TTS-run\config.json
#  | > model file:  C:\Users\Alex\Projects\TTS-run\Tacotron2-iter-260K.pth.tar
#  > Setting up Audio Processor...
#  | > bits:None
#  | > sample_rate:22050
#  | > num_mels:80
#  | > min_level_db:-100
#  | > frame_shift_ms:12.5
#  | > frame_length_ms:50
#  | > ref_level_db:20
#  | > num_freq:1025
#  | > power:1.5
#  | > preemphasis:0.98
#  | > griffin_lim_iters:60
#  | > signal_norm:True
#  | > symmetric_norm:False
#  | > mel_fmin:0.0
#  | > mel_fmax:8000.0
#  | > max_norm:1.0
#  | > clip_norm:True
#  | > do_trim_silence:True
#  | > n_fft:2048
#  | > hop_length:275
#  | > win_length:1102
#  > Using model: Tacotron2
# Traceback (most recent call last):
#   File "server/server.py", line 16, in <module>
#     config.model_config, config.use_cuda)
#   File "C:\Users\Alex\Projects\TTS-run\server\synthesizer.py", line 36, in load_model
#     self.model.load_state_dict(cp['model'])
#   File "C:\Users\Alex\Miniconda3\envs\dreambank2\lib\site-packages\torch\nn\modules\module.py", line 845, in load_state_dict
#     self.__class__.__name__, "\n\t".join(error_msgs)))
# RuntimeError: Error(s) in loading state_dict for Tacotron2:
#         Missing key(s) in state_dict: "decoder.attention_layer.ta.weight", "decoder.attention_layer.ta.bias".
dubreuia picture dubreuia on 21 Aug 2019

Also, I think you made a mistake in 8921554632242082babaa321a3ba2ae2b6edd1f5 lines server/synthesizer.py:58 and server/synthesizer.py:61, the variable sen should be used instead of text or else the whole text will get synth at each iteration.

dubreuia picture dubreuia on 21 Aug 2019

Also, I think you made a mistake in 8921554 lines server/synthesizer.py:58 and server/synthesizer.py:61, the variable sen should be used instead of text or else the whole text will get synth at each iteration.

@dubreuia hi there!

have you found the solution for your problem regarding:

Missing key(s) in state_dict: "decoder.attention_layer.ta.weight" ?

Because i faced the same problem too even i use the fix one.

haqkiemdaim picture haqkiemdaim on 10 Oct 2019

Hello @haqkiemdaim, no unfortunately it doesn't work, you'll have to use Tacotron-185K.

dubreuia picture dubreuia on 10 Oct 2019

@dubreuia do you have any idea on testing the trained model instead of using the server.py to generate audio from input text ?

haqkiemdaim picture haqkiemdaim on 10 Oct 2019

@haqkiemdaim You can use the Synthesizer class in synthesizer.py, in a Python shell, you should be able to do from synthesizer import Synthesizer, then instantiate and load the model with load_model (your trained model), and finally call the tts method with your text producing the wav file.

dubreuia picture dubreuia on 10 Oct 2019

Hello @haqkiemdaim, no unfortunately it doesn't work, you'll have to use Tacotron-185K.

Which means that i still not able to run my newly trained model of Tacotron2 for testing?

@haqkiemdaim You can use the Synthesizer class in synthesizer.py, in a Python shell, you should be able to do from synthesizer import Synthesizer, then instantiate and load the model with load_model (your trained model), and finally call the tts method with your text producing the wav file.

Yeap but it still stuck at:

syntesizer = Syntesizer(config)

@dubreuia

haqkiemdaim picture haqkiemdaim on 11 Oct 2019

Which means that i still not able to run my newly trained model of Tacotron2 for testing?

I meant the pre-trained Tacotron2-iter-260K doesn't work (the code of commit 824c091 doesn't fit the model). Concerning your model, if you can train it, you can run it.

Yeap but it still stuck at: synthesizer = Synthesizer(config)

You should be able to instantiate the config somehow. Personally I've rewritten the Synthesizer class for easier use, here's my code:

class Synthesizer(object):

  def __init__(self,
               model_path=os.path.curdir,
               model_name=os.path.join("bundles",
                                       "Tacotron-iter-185K.pth.tar"),
               model_config="config.json",
               use_cuda=False):
    """The use_cuda param cannot be True if tensorflow uses the GPU, bacause it
    allocates all the memory. Check tf per_process_gpu_memory_fraction."""
    model_config = os.path.join(model_path, model_config)
    self.model_file = os.path.join(model_path, model_name)
    self.config = load_config(model_config)
    self.use_cuda = use_cuda
    self.ap = AudioProcessor(**self.config.audio)
    self.model = Tacotron(61, self.config.embedding_size, self.ap.num_freq,
                          self.ap.num_mels, self.config.r)
    # load model state
    if use_cuda:
      cp = torch.load(self.model_file)
    else:
      cp = torch.load(
        self.model_file, map_location=lambda storage, loc: storage)
    # load the model
    self.model.load_state_dict(cp['model'])
    if use_cuda:
      self.model.cuda()
    self.model.eval()

  def save_wav(self, wav, path):
    # wav *= 32767 / max(1e-8, np.max(np.abs(wav)))
    wav = np.array(wav)
    self.ap.save_wav(wav, path)

  def tts(self, text):
    text_cleaner = [self.config.text_cleaner]
    wavs = []
    for sentence in text.split('.'):
      if len(sentence) < 3:
        continue
      sentence = sentence.strip()
      sentence += '.'
      sentence = sentence.strip()
      seq = np.array(
        phoneme_to_sequence(sentence, text_cleaner,
                            self.config.phoneme_language))
      chars_var = torch.from_numpy(seq).unsqueeze(0).long()
      if self.use_cuda:
        chars_var = chars_var.cuda()
      mel_out, linear_out, alignments, stop_tokens = self.model.forward(
        chars_var)
      linear_out = linear_out[0].data.cpu().numpy()
      wav = self.ap.inv_spectrogram(linear_out.T)
      wavs += list(wav)
      wavs += [0] * 10000

    out = io.BytesIO()
    self.save_wav(wavs, out)

    return out


if __name__ == "__main__":
  synthesizer = Synthesizer("C:/Users/dubreuia/Projects/theproject/sound/system",
                            "bundles/Tacotron-iter-185K.pth.tar",
                            "config.json",
                            True)
  wav_binary = synthesizer.tts(
    """If you're looking for an Old English Translator, then click that link. 
    Old English is very different to Shakespearean/Elizabethan English, which 
    falls under the category of Early Modern English. Thus, this translator 
    converts Modern English into (exaggerated) Early Modern English.""")
  with open(os.path.join("output", "speech", "out.wav"), "wb") as file:
    file.write(wav_binary.read())
dubreuia picture dubreuia on 11 Oct 2019

You should be able to instantiate the config somehow. Personally I've rewritten the Synthesizer class for easier use, here's my code:

@dubreuia woww good effort on witing the code. Thank you. I could use it later

I meant the pre-trained Tacotron2-iter-260K doesn't work (the code of commit 824c091 doesn't fit the model). Concerning your model, if you can train it, you can run it.

Unfortunately, i'm not able to run it using server.py for my self-trained Tacotron2 - my own dataset (Malaysian Language) it shows the error below :

Using model: Tacotron2
Traceback (most recent call last):
File "server.py", line 14, in
synthesizer = Synthesizer(config)
File "/var/www/html/TTS/synthesizer.py", line 30, in __init__
self.load_tts(self.config.tts_path, self.config.tts_file, self.config.tts_config, config.use_cuda)
File "/var/www/html/TTS/synthesizer.py", line 57, in load_tts
self.tts_model.load_state_dict(cp['model'])
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 845, in load_state_dict
self.__class__.__name__, "nt".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Tacotron2:
Missing key(s) in state_dict: "decoder.attention_rnn_init.weight", "decoder.go_frame_init.weight", "decoder.decoder_rnn_inits.weight".

i attach together my model just in case you would like to give it a try:

best_modeltaco2.pth.tar
configtaco2.json

haqkiemdaim picture haqkiemdaim on 11 Oct 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

domcross picture domcross  路  6Comments
PetrochukM picture PetrochukM  路  9Comments
faaip picture faaip  路  5Comments
nowingnet picture nowingnet  路  5Comments
floatingatoll picture floatingatoll  路  3Comments