Insightface: About the issue of single GPU training speed

Created on 13 Mar 2019  路  1Comment  路  Source: deepinsight/insightface

Hello,I have two servers:

Server 1: System is Ubuntu16.04, hard disk israid5, use command "hdparm-t /dev/sdb" test hard disk reading speed is 900M/s, GPU is NVIDIA GTX 1080 8g, training Insightface speed is 80samples/s.

Server 2: System is Ubuntu16.04, system hard disk is raid5, use command "hdparm-t /dev/sda" test hard disk reading speed is 350M/s; Data hard disk SSD is raid0, using command "hdparm-t /dev/sdb" test hard disk reading speed is 1600M/s; GPU is NVIDIA V100 32G, training Insightface speed is also about 80 samples/s (training data on raid0 SSD disk).

Server 2, whether GPU or data hard disk, performance is much better than server 1, why Insightface training speed is not improved?

Thanks for your attentions.

Most helpful comment

Try set MXNET_CPU_WORKER_NTHREADS=1 and use the code in recognition/

>All comments

Try set MXNET_CPU_WORKER_NTHREADS=1 and use the code in recognition/

Was this page helpful?
0 / 5 - 0 ratings

Related issues

AnhVPB picture AnhVPB  路  4Comments

WIll-Xu35 picture WIll-Xu35  路  3Comments

mdv3101 picture mdv3101  路  5Comments

1frey picture 1frey  路  4Comments

zys1994 picture zys1994  路  3Comments