Ignite: auto_dataloader can auto set pin_memory if appropriate

[vfdev-5]

🚀 Feature

Idea is to automatically set pin_memory=True if idist.backend() is nccl or default device is cuda.

In the code we can omit this param instead of
https://github.com/pytorch/ignite/blob/6b88eb89cf40e3c4ebe05e0fe67f3c8013ecc5c2/examples/contrib/cifar10/main.py#L199

[InCogNiTo124]

Hey, me again :)

So there is only one line that should be changed? Did I understand that right?

[vfdev-5]

Hey @InCogNiTo124 :)

1) It is about adding some code on L91
https://github.com/pytorch/ignite/blob/6b88eb89cf40e3c4ebe05e0fe67f3c8013ecc5c2/ignite/distributed/auto.py#L90-L92

check if kwargs contains pin_memory and if not set to True if idist.device() is cuda.
Probably, more than one line.

2) update examples

• cifar10

[InCogNiTo124]

Sounds doable. And what about docs or tests?

[vfdev-5]

Sounds doable. And what about docs or tests?

Just say here : https://github.com/pytorch/ignite/blob/6b88eb89cf40e3c4ebe05e0fe67f3c8013ecc5c2/ignite/distributed/auto.py#L26-L28
that we auto set pin_memory=True for cuda devices if user does not provide pin_memory arg.

For tests, we can check pin_memory attribute of DataLoader here :
https://github.com/pytorch/ignite/blob/6b88eb89cf40e3c4ebe05e0fe67f3c8013ecc5c2/tests/ignite/distributed/test_auto.py#L15
if provided instance is DataLoader (can be MpDeviceLoader for XLA)...