Hello,I have two servers:
Server 1: System is Ubuntu16.04, hard disk israid5, use command "hdparm-t /dev/sdb" test hard disk reading speed is 900M/s, GPU is NVIDIA GTX 1080 8g, training Insightface speed is 80samples/s.
Server 2: System is Ubuntu16.04, system hard disk is raid5, use command "hdparm-t /dev/sda" test hard disk reading speed is 350M/s; Data hard disk SSD is raid0, using command "hdparm-t /dev/sdb" test hard disk reading speed is 1600M/s; GPU is NVIDIA V100 32G, training Insightface speed is also about 80 samples/s (training data on raid0 SSD disk).
Server 2, whether GPU or data hard disk, performance is much better than server 1, why Insightface training speed is not improved?
Thanks for your attentions.
Try set MXNET_CPU_WORKER_NTHREADS=1 and use the code in recognition/
Most helpful comment
Try set
MXNET_CPU_WORKER_NTHREADS=1and use the code inrecognition/