Yet-another-efficientdet-pytorch: CUDA error: invalid device ordinal

Created on 28 Apr 2020  ·  2Comments  ·  Source: zylo117/Yet-Another-EfficientDet-Pytorch

I am running a custom training with product checkout dataset (coco)
I am using google colab with GPU and the following is the error

CUDA error: invalid device ordinal
0% 11/6717 [00:24<1:31:06, 1.23it/s][Error] Traceback (most recent call last):
File "train.py", line 227, in train
cls_loss, reg_loss = model(imgs, annot, obj_list=params.obj_list)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/data_parallel.py", line 148, in forward
inputs, kwargs = self.scatter(inputs, kwargs, self.device_ids)
File "/content/Yet-Another-EfficientDet-Pytorch/utils/utils.py", line 199, in scatter
for device_idx in range(len(devices))], \
File "/content/Yet-Another-EfficientDet-Pytorch/utils/utils.py", line 199, in
for device_idx in range(len(devices))], \
RuntimeError: CUDA error: invalid device ordinal

Training command that I have used for the same is

!python train.py -c 1 -p productDataset --batch_size 8 --lr 1e-5

Most helpful comment

change num_gpu=1 in project.yml

All 2 comments

change num_gpu=1 in project.yml

I changed line 194 in utils.py to devices = ['cuda:0']
and yes your suggestion would have worked the same way

Thank you for the response I was going to close the issue :)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

tdn670000 picture tdn670000  ·  10Comments

CuiZhiying picture CuiZhiying  ·  8Comments

LiangTinting picture LiangTinting  ·  6Comments

wangning7149 picture wangning7149  ·  5Comments

mad-fogs picture mad-fogs  ·  5Comments