Opennmt-py: Azure instance with GPU recommendation?

[paulkp]

Can u recommend an Azure instance with GPU to EXTENSIVELY TRAIN OpenNMT-py models?
@guillaumekln et al

[guillaumekln]

Looks like this instance is appropriate: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-gpu#ncv3-series

[francoishernandez]

@paulkp
Regarding https://forum.opennmt.net/t/does-opennmt-need-double-precision-fp64/3438/2
Does opennmt need double precision (FP64)?
--> no
Best I think is to use mixed precision (FP16/FP32), enabled by setting -model_dtype fp16 (and using optim fusedadam if you're training a Transformer).
See #1208 for some experiments on FP32 vs FP16 speed.

[paulkp]

In light of @francoishernandez 's comment re FP64, @guillaumekln , shouldn't the ND series be sufficient (it's single precision) but otherwise same specs as NCv3 AND SIGNIFICANTLY CHEAPER (like 2/3rds the hourly-price)? I think the P40 is comparable to the P100?

https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-gpu#nd-series

[guillaumekln]

The NCv3 instance comes with a V100 which is faster than a P40 and supports FP16 training. But of course it all comes down to how much you can spend on your project.

[paulkp]

And @guillaumekln:

Q1. If we can somehow get our hands on an 8 GPU (XP) instance will ONMT-py make use of it in practice (ie roughly 8x performance of single GPU)?

Q2. I notice that my TRAINING runs always use a SINGLE CPU out of 6 at 100%. That makes it seem like it is CPU limited? And can't use more than 1 CPU? Are those true statements? Can you explain?

[guillaumekln]

Q1. OpenNMT-py can make use of multiple GPUs but the scaling factor is usually a bit less than 8 because of the cost of cross GPU communication.
Q2. If your GPU usage is still high, this should not be an issue.