Opennmt-py: Fixed seeds do not work

[Maggione]

I set the "opt.seed" as a positive int value, however, the output results are different each time without changing anything. How can I reproduce the result of models?

[JianyuZhan]

Are you training using GPU? If yes, you probably need to use this torch.cuda.manual_seed, see http://pytorch.org/docs/master/cuda.html?highlight=torch%20manual_seed#torch.cuda.manual_seed

If that works, I will change the code. Thanks.

[Maggione]

Yes, it has been used in train.py as follow.

if opt.gpuid:
    cuda.set_device(opt.gpuid[0])
    if opt.seed > 0:
        torch.cuda.manual_seed(opt.seed)

It does not work.

[srush]

Huh.. Is this a pytorch issue? What else should we be doing to make sure the seed is used?

[Maggione]

I think it maybe the seed of torchtext.data.Iterator is not fixed? Do you know how to fix it?

[Maggione]

Yes, it is the problem of IO.py. Because torchtext.data.Iterator shuffle the data by the package "random". You can insert the code:
import random
random.seed([your seed])
It works.

[srush]

Oh that's a good catch. Can you send a PR?

[JianyuZhan]

436

[xinyadu]

randomness still exist, after setting the seed for ``random'' in train.py

[fuzihaofzh]

Seems still not work due to the torchtext problem.

[DianeBouchacourt]

Same, it does not seem to work for me either. Setting the random.seed() does not change a seed. It would be great to be able to control the order of the samples drawn by the iterator at each epoch for reproducibility.