Darknet: CUDA Error: invalid device ordinal (Ubuntu 16, NVIDIA Quadro P4000)

[MattCollins84]

I'm getting this when I try to run the GPU enabled example from the website, successfully ran make with GPU=1, no OpenCV.

Running on Ubuntu 16, with NVIDIA Quadro P4000 on nvidia-387 drivers

``` $ ./darknet -i 1 imagenet test cfg/alexnet.cfg alexnet.weights CUDA Error: invalid device ordinal darknet: ./src/cuda.c:36: check_error: Assertion0' failed.
Aborted (core dumped)

$ nvidia-smi
Mon Dec 18 11:50:55 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 387.34 Driver Version: 387.34 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Quadro P4000 Off | 00000000:65:00.0 On | N/A |
| 46% 37C P0 30W / 105W | 199MiB / 8110MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1160 G /usr/lib/xorg/Xorg 149MiB |
| 0 1834 G compiz 46MiB |
+-----------------------------------------------------------------------------+
````

I think CUDA is installed fine, I can run deviceQuery ok:

````
./deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "Quadro P4000"
CUDA Driver Version / Runtime Version 9.1 / 8.0
CUDA Capability Major/Minor version number: 6.1
Total amount of global memory: 8111 MBytes (8504868864 bytes)
(14) Multiprocessors, (128) CUDA Cores/MP: 1792 CUDA Cores
GPU Max Clock rate: 1480 MHz (1.48 GHz)
Memory Clock rate: 3802 Mhz
Memory Bus Width: 256-bit
L2 Cache Size: 2097152 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 101 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 9.1, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = Quadro P4000
Result = PASS
````

And bandwidthTest gets me:

````
[CUDA Bandwidth Test] - Starting...
Running on...

Device 0: Quadro P4000
Quick Mode

Host to Device Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(MB/s)
33554432 12072.5

Device to Host Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(MB/s)
33554432 12875.6

Device to Device Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(MB/s)
33554432 190374.6

Result = PASS
````

I'm a bit of a newbie with this so if you need any more info let me know! Any ideas?

[gtatiya]

You have only one GPU.
So, if you want to use it with Darknet replace 1 with 0 like:
./darknet -i 0 imagenet test cfg/alexnet.cfg alexnet.weights

[MattCollins84]

ah! So daft of me - thank you very much!

[jcleberre22]

Hello guys i know this post is closed but I'm having the same issue and I tried your solution but the command doesn't works :
~/darknet/darknet$ ./darknet -i 0 imagenet test cfg/alexnet.cfg alexnet.weights
Not an option: imagenet

~/darknet/darknet$ ./darknet -i 1 imagenet test cfg/alexnet.cfg alexnet.weights
CUDA Error: invalid device ordinal
darknet: ./src/cuda.c:36: check_error: Assertion `0' failed.
Abandon (core dumped)

Any Idea?

[MattCollins84]

@jcleberre22 I am not an expert, but I think you are experiencing 2 different issues.

Using -i 0 is "working" in that darknet is running, but has an issue with your use of imagenet

Using -i 1 is not working, because you don't have multiple GPUs

[jcleberre22]

Thanks for your answer Matt,
You was right the -i 0 was working i tried with a different name and images and it works
now I have complete the installation process and everything is ok now I have to check how i can integrate YOLO in my project
Have a good day

[89douner]

Help me jcleberre22!
I have the same problem when I use "./darknet -i 0 imagenet test cfg/alexnet.cfg alexnet.weights" this command. --> Not an option: imagenet

Please let me know your solution (with a different name and images).
(Could you write the command? or procedure?) Thanks !

[jcleberre22]

Hello @89douner,

I've changed the command to :
./darknet -i 0 test predictions.png cfg/alexnet.cfg alexnet.weights
You can of course use any image instead of predictions.png.

Anyway, If you are just following this procedure to run YOLO like I was, you don't need this command and can simply run the following command:
./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights
you are already all set.

[89douner]

Thanks a lot!!

[me6ran]

I had the same issue and it is solved by changing OPENCV=0 to OPENCV=1 in the make file the Makefile, and remake it again:
make

[LolikaPadmanbhan]

Hi,
Am using GUP compilation, and getting this error. kindly help me whats is wrong here?
IGNKO-Train:~/Darket/darknet$ ./darknet -i 0 person.jpg test cfg/alexnet.cfg alexnet.weights
Not an option: person.jpg
Thanks

[jcleberre22]

Hi,
you have to put "test" before the image name like this:
./darknet -i 0 test person.jpg cfg/alexnet.cfg alexnet.weights
Have a nice day.

[LolikaPadmanbhan]

@jcleberre22
Hi,
Thank you so much.. i could test it.:-)

[LolikaPadmanbhan]

@jcleberre22
Hey where can i get pre trained darknet.data file? because am trying to use pre trained darknet.data and see for my dataset so.
Kindly help me with that.
Thanks

[jcleberre22]

Hi @LolikaPadmanbhan,
I have no idea my friend, google is your friend :-)
Good luck in your searches and share if you find something it could be usefull.
Cya

[me6ran]

@LolikaPadmanbhan
Look at link below, it gives you all information you need.
https://www.youtube.com/watch?v=eFJOGsQ_YTA

[LolikaPadmanbhan]

@jcleberre22 @me6ran
Thanks guys.