Hi
Currently wav2letter support spectrogram augmentation (SAUG) and dither noise. What if I want to apply more augmentation? What is the proposed way of doing some augmentation say adding modeled noise (pink, babble, etc.), adding tempo and gain perturbations?
I also read in SpecAugment.h that currently time warping is not supported. Is it a WIP or there is no plan to implement this source of noise?
Thank you in advance
hajix
All 4 comments
Since my question is about the same topic, I will add it here. Looking at the recipes, the architecture file for the streaming convnets includes SpecAugment as suggested by the paper. However, the config-file does not have SAUG flags (--use_saug and --saug_start_update). Is either/both of these flags unnecessary if the architecture file already includes SAUG?
AVSuni
on 13 Apr 2020
@AVSuni
In case of transformer AM model we need to have warmup stage during which we don't use augmentation. To support this for now we solved it by added flags use_saug and saug_start_update. So in case you want to train model with warmup (and no augmentation during warmup) you just operate with these flags and spec augment flags and don't include SAUG layer in the arch file. If you don't do warmup you can simply specify SAUG layer in the arch.
Also now one can use more complicated combinations, like use augmentation at some specific period of time during training.
tlikhomanenko
on 13 Apr 2020
@tlikhomanenko
Is time warping applied in SAUG or not? I'm a little bit confused with the documentation.
hajix
on 14 Apr 2020
@hajix, sorry for delay in answering.
About SpecAugment:
- In SAUG there is no time warping
- in the original paper they didn't see huge gain with time warping. For now it is not top priority for us to add time warping, but any pull request is welcome.
About other augmentations:
- If you need more augmentation you always can add them by yourself following the interface of SpecAugment.
- We have plans to add more augmentations, but this won't happen in upcoming weeks.
And always pull requests are welcome and appreciated!
tlikhomanenko
on 15 Apr 2020
Related issues
megharangaswamy
路
5Comments
nihiluis
路
5Comments
Terry1504
路
4Comments
tarang-jain
路
3Comments
gauenk
路
3Comments
Most helpful comment
@AVSuni
In case of transformer AM model we need to have warmup stage during which we don't use augmentation. To support this for now we solved it by added flags
use_saugandsaug_start_update. So in case you want to train model with warmup (and no augmentation during warmup) you just operate with these flags and spec augment flags and don't include SAUG layer in the arch file. If you don't do warmup you can simply specify SAUG layer in the arch.Also now one can use more complicated combinations, like use augmentation at some specific period of time during training.