Incubator-mxnet: GPUDeviceStorage is not used? why?
I found that GPUDeviceStorage is not used, however CPUDeviceStorage is used, why is that?
hpsoar
All 4 comments
Are you sure you compiled with CUDA support enabled? What is it that you are trying to do?
ptrendx
on 16 Aug 2017
I didn't compile with CUDA, I'm reading the code, I see that when CUDA is used, it actually use GPUPooledStorageManager to do the allocation, on the other hand, CPUDeviceStorage is used no where.
I'm just curious why CPUDeviceStorage is there, and not in use.
I'm on the master branch.
hpsoar
on 17 Aug 2017
Oh, ok. This is because CUDA allocations and deallocations are much more expensive (since they deal with pinned memory), especially with multiple GPUs per node. They also introduce stalls in the GPU pipeline, which is not something we want for maximum performance. That is why MXNet uses caching allocator (GPUPooledStorageManager) instead of a naive one.
ptrendx
on 17 Aug 2017
@szha please close
ptrendx
on 29 Sep 2017
Related issues
sbodenstein
路
3Comments
realbns2008
路
3Comments
ranti-iitg
路
3Comments
xzqjack
路
3Comments
Fzz123
路
3Comments
Most helpful comment
Oh, ok. This is because CUDA allocations and deallocations are much more expensive (since they deal with pinned memory), especially with multiple GPUs per node. They also introduce stalls in the GPU pipeline, which is not something we want for maximum performance. That is why MXNet uses caching allocator (GPUPooledStorageManager) instead of a naive one.