Pytorch-cyclegan-and-pix2pix: Question about input_A = input_A.cuda(self.gpu_ids[0], async=True)
In models/cycle_gan_model.py, line 83: input_A = input_A.cuda(self.gpu_ids[0], async=True).
Why do we only use a single GPU (gpu_ids[0]) and why do we set acync to True?
d1ngn1gefe1
👍1
>All comments
It's not about single GPU... A tensor lives on either CPU memory or a GPU's memory. There is no such thing of having a tensor on multiple GPUs. You can always copy a tensor from one GPU to another though. Here we just put the tensor onto one of the GPUs. MultiGPU training is done via DataParallel.
Since our dataloader uses pinned memory, we can load asynchronously. Setting async to true avoids cuda synchronization. So it's mainly for the slight speed gain.
SsnL
on 2 Feb 2018
👍9
Was this page helpful?
0 / 5 - 0 ratings
Related issues
wjx2
路
3Comments
Overtown
路
3Comments
nootfly
路
4Comments
HectorAnadon
路
4Comments
AlexanLee
路
4Comments
Most helpful comment
It's not about single GPU... A tensor lives on either CPU memory or a GPU's memory. There is no such thing of having a tensor on multiple GPUs. You can always copy a tensor from one GPU to another though. Here we just put the tensor onto one of the GPUs. MultiGPU training is done via DataParallel.
Since our dataloader uses pinned memory, we can load asynchronously. Setting async to true avoids cuda synchronization. So it's mainly for the slight speed gain.