Tacotron-2: Where is the wavenet input "tacotron_output/gta/map.txt"?

[begeekmyfriend]

I have trained out a Tacotron model and the directory structure is as follows:
tacotron_output
├──eval
│ ├──map.txt
│ └──*.npy
└──logs-eval
├──plots
└──wavs
And then I wanted to train the WaveNet model but it confused me that I need the input tacotron_output/gta/map.txt. Where can I find it? Is it tacotron_output/eval/map.txt? But the matadata fetched in the feeder seemed not the proper one.

[Rayhane-mamah]

You need to make a GTA synthesis using the Tacotron model. If you trained
using "Tacotron-2" or "Both" as model name, did it not launch automatically?

On Fri, 25 May 2018, 03:47 Leo Ma, [e-mail] wrote:

I have trained out a Tacotron model and the directory structure is as
follows:
tacotron_output
├──eval
│ ├──map.txt
│ └──*.npy
└──logs-eval
├──plots
└──wavs
And then I wanted to train the WaveNet model but it confused me that I
need the input tacotron_output/gta/map.txt. Where can I find it? Is it
tacotron_output/eval/map.txt? But the matadata fetched in the feeder
seemed not the proper one.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Rayhane-mamah/Tacotron-2/issues/52, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AhFSwMnXv0azJi7PxJ5toKP6el1qCTO1ks5t13DGgaJpZM4UNSM0
.

[Rayhane-mamah]

My bad, just noticed I disabled it by mistake. In our project, since it is a Tacotron-2 project, we only train Wavenet once we have the Spectrogram Prediction Network and made GTA synthesis with it (even when not training wavenet with GTA).

[begeekmyfriend]

I have tried synthesis and it did not generate tacotron_output/gta.

python train.py # GTA=True model='Tacotron-2'
python synthesize.py # GTA=True model='Tacotron-2'

[begeekmyfriend]

@Rayhane-mamah It still throws exception in feeder.py after I merged your latest commit. It seems the args.input_path kept the old one.

Traceback (most recent call last):
File "train.py", line 83, in
main()
File "train.py", line 75, in main
wavenet_train(args, log_dir, hparams, args.wavenet_input)
File "/home/leoma/Tacotron-2/wavenet_vocoder/train.py", line 243, in wavenet_train
train(log_dir, args, hparams, input_path)
File "/home/leoma/Tacotron-2/wavenet_vocoder/train.py", line 163, in train
feeder = Feeder(coord, input_path, args.base_dir, hparams)
File "/home/leoma/Tacotron-2/wavenet_vocoder/feeder.py", line 36, in __init__
with open(metadata_filename, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'tacotron_output/gta/map.txt'

[Rayhane-mamah]

I believe the correct way to run the synthesis is:

python synthesize.py --mode=synthesis --GTA=True --model='Tacotron'

Since (ec0cd50d24d7f5561f696a78d0cd47d5a2a14a43) I have maid the "eval" mode default behavior for synthesis assuming "Tacotron-2" will be default behavior for synthesis in the future, so to generate using GTA, one must explicitly set mode=synthesis

It will not work if you use "Tacotron-2" For synthesis on all dataset because it supposes you want to generate using Both Tacotron and Wavenet, using model=Tacotron will do the task you wanna achieve. I will retest the entire pipeline later to check if this is still a problem..

[MXGray]

@Rayhane-mamah
Ran training using --mode='Tacotron-2', and it started to train the feature prediction network.
I set feature prediction training to stop at 50K steps, and so it started to train wavenet vocoder after saving this 50KS featuer prediction checkpoint. However:

I Encountered the same problem as mentioned in this issue. So, my question is:

• How do I begin to train wavenet vocoder from the 50KS checkpoint that was saved when I trained the feature prediction network?

Please advise. Otherwise, it'll be an awful waste of time and resources for me to start over and train feature prediction network using --model='Tacotron' this time instead of --model='Tacotron-2', just to start training wavenet vocoder.

[Rayhane-mamah]

@MXGray please verify that this line is not commented in your train.py:
https://github.com/Rayhane-mamah/Tacotron-2/blob/fb5564b7584ae0dc62ffecaa89d463ff24a3c251/train.py#L34

Run the training again with Tacotron-2, it will skip Feature Prediction Network training whem it finds the checkpoint. It should run GTA synthesis then wavenet training. Tell me how it goes

[MXGray]

@Rayhane-mamah
Thanks! Seems to be working now - Used --model='Tacotron-2', and training skipped immediately to wavenet training after detecting the latest feature prediction checkpoint (50KS).
It's now at 3% synthesis; and
Here's what I left de-commented in train.py:

input_path = tacotron_synthesize(args, hparams, checkpoint)
input_path = 'tacotron_output/gta/map.txt'

Cheers! :)

[Rayhane-mamah]

Yeah these lines were wrong in previous version of the repo but are fixed now. Feel free to share your results with Wavenet once it's done

[MXGray]

@Rayhane-mamah
Been encountering errors while trying to synthesize text using the saved checkpoints for both the encoder network and the wavenet vocoder; and
My CLI command is:

python synthesize.py --model='Tacotron-2' --mode='eval'

And, I get this:

synthesize(args, hparams, taco_checkpoint, wave_checkpoint, sentences)
synthesize.py, line 33, in synthesize
wavenet_synthesize(args, hparams, wave_checkpoint)
/wavenet_vocoder/synthesize.py", line 78, in wavenet_synthesize
run_synthesis(args, checkpoint_path, output_dir, hparams)
/wavenet_vocoder/synthesize.py", line 19, in run_synthesis synth.load(checkpoint_path, hparams)
/wavenet_vocoder/synthesizer.py", line 27, in load sh_saver = create_shadow_saver(self.model)
/wavenet_vocoder/train.py", line 53, in create_shadow_saver return tf.train.Saver(shadow_dict, max_to_keep=5)
/tensorflow/python/training/saver.py", line 1293, in __init__ self.build()
/tensorflow/python/training/saver.py", line 1302, in build self._build(self._filename, build_save=True, build_restore=True)
/tensorflow/python/training/saver.py", line 1339, in _build build_save=build_save, build_restore=build_restore)
/tensorflow/python/training/saver.py", line 796, in _build_internal restore_sequentially, reshape)
/tensorflow/python/training/saver.py", line 449, in _AddRestoreOps restore_sequentially)
/tensorflow/python/training/saver.py", line 847, in bulk_restore return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
/tensorflow/python/ops/gen_io_ops.py", line 1030, in restore_v2
shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def)
/tensorflow/python/framework/ops.py", line 3271, in create_op op_def=op_def)
/tensorflow/python/framework/ops.py", line 1650, in __init__ self._traceback = self._graph._extract_stack() # pylint: disable=protected-accessNotFoundError (see above for traceback): Key model_1/model/ResidualConv1dGLU_0/residual_block_cin_conv/bias_residual_block_cin_conv/ExponentialMovingAverage not found in checkpoint
[[Node: model_1/save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_model_1/save/Const_0_0, model_1/save/RestoreV2/tensor_names,model_1/save/RestoreV2/shape_and_slices)]]

What could be the problem?

[Rayhane-mamah]

Hello again @MXGray, yes I have encountered same problems too, didn't find time to upload corrections today, I will upload it in a couple of minutes

[MXGray]

Hi @Rayhane-mamah,
Just wanted to kindly ask if you've already uploaded the corrected files?
Please advise. Thanks! :)

[Rayhane-mamah]

Yes I did, the bug is mainly due to Exponential Moving Average requesting checkpoints from a wrong scope at synthesis time. I don't quite remember which scopes were wrong, but this issue is supposed to be fixed 2 commits ago, naturally if you're using a model trained before the scopes bug was fixed, checkpoint files will still have wrong scopes in them..

I am hoping you didn't train the Wavenet for too long..

[cuicp]

Can you load the Wavenet model successfully now? @MXGray
I trained the model with the latest code(e2f9780b18648b8d0b671e1c85dd95d20d6f8553), but when i executed 'python synthesize.py', the WaveNet model still load failed，and i move the folder "logs-Both/wave_pretrained/" to "logs-WaveNet/wave_pretrained/", then i execute "pthon synthesize.py --model='WaveNet'", the Wavenet model can be loaded succesful，but the synthesized audio has bad quality while the audio synthesized in training process has good quality， the details #82，

[v-yunbin]

@MXGray do you have successfully run "python synthesize.py --model='Tacotron-2' --mode='eval'"?

[Rayhane-mamah]

I believe this is fixed with latest commit, if the problem persists please reopen the issue!

[shartoo]

Can you tell me how to fix the bug?I'm using someone else's code basing on older version of this,i can't simply replace it with current code.Thank you!