Yolov3: test.py", line 211, in test : initialize COCO ground truth api IndexError: list index out of range

[jingweirobot]

(base) jingwei@jingwei:~/yolov3$ python3 train.py --batch-size 8
Apex recommended for faster mixed precision training: https://github.com/NVIDIA/apex
Namespace(adam=False, batch_size=8, bucket='', cache_images=False, cfg='cfg/yolov3-spp.cfg', data='data/coco2017.data', device='', epochs=1, evolve=False, img_size=[320, 640], multi_scale=False, name='', nosave=False, notest=False, rect=False, resume=False, single_cls=False, weights='weights/yolov3-spp-ultralytics.pt')
Using CUDA device0 _CudaDeviceProperties(name='GeForce RTX 2080 Ti', total_memory=11016MB)

Start Tensorboard with "tensorboard --logdir=runs", view at http://localhost:6006/
Model Summary: 225 layers, 6.29987e+07 parameters, 6.29987e+07 gradients
Optimizer groups: 76 .bias, 76 Conv2d.weight, 73 other
Caching labels (117266 found, 1021 missing, 0 empty, 0 duplicate, for 118287 ima
Caching labels (4952 found, 48 missing, 0 empty, 0 duplicate, for 5000 images):
Image sizes 320 - 640 train, 640 test
Using 8 dataloader workers
Starting training for 1 epochs...

 Epoch   gpu_mem      GIoU       obj       cls     total   targets  img_size
   0/0     9.99G      2.45      2.72     0.928       6.1        72       384
           Class    Images   Targets         P         R   [email protected]        F1
             all     5e+03  3.63e+04     0.373     0.672     0.566     0.474

Speed: 13.9/1.4/15.2 ms inference/NMS/total per 640x640 image at batch-size 8

COCO mAP with pycocotools...
Traceback (most recent call last):
File "train.py", line 411, in
train(hyp) # train normally
File "train.py", line 317, in train
dataloader=testloader)
File "/home/jingwei/yolov3/test.py", line 211, in test
cocoGt = COCO(glob.glob('/home/jingwei/yolov3/data/annotations/instances_val*.json')[0]) # initialize COCO ground truth api
IndexError: list index out of range

[jingweirobot]

Dear Author
When I train this model, I got the issue above. Could you help me fix this? Thanks a lot.

[glenn-jocher]

@jingweirobot your model trained correctly. This error happens when trying to use pycocotools on your results.

[jingweirobot]

@glenn-jocher do you know how to fix this issue? Thanks a lot

[glenn-jocher]

@jingweirobot if you are training coco, then you do want pycocotools to run, which means you need to install numpy == 1.17, otherwise this error will occur.

if you are training a custom dataset, then you do not want pycocotools to run, and you can either ignore this error, or rename your *.data file to remove the word coco from it.

[glenn-jocher]

See https://github.com/cocodataset/cocoapi/issues/356

[jingweirobot]

(base) jingwei@jingwei:~/yolov3$ python3 train.py --batch-size 8
Apex recommended for faster mixed precision training: https://github.com/NVIDIA/apex
Your branch is behind 'origin/master' by 4 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)

Namespace(adam=False, batch_size=8, bucket='', cache_images=False, cfg='cfg/yolov3-spp.cfg', data='data/coco2017.data', device='', epochs=1, evolve=False, img_size=[320, 640], multi_scale=False, name='', nosave=False, notest=False, rect=False, resume=False, single_cls=False, weights='weights/yolov3-spp-ultralytics.pt')
Using CUDA device0 _CudaDeviceProperties(name='GeForce RTX 2080 Ti', total_memory=11016MB)

Start Tensorboard with "tensorboard --logdir=runs", view at http://localhost:6006/
Model Summary: 225 layers, 6.29987e+07 parameters, 6.29987e+07 gradients
Optimizer groups: 76 .bias, 76 Conv2d.weight, 73 other
Caching labels (117266 found, 1021 missing, 0 empty, 0 duplicate, for 118287 ima
Caching labels (4952 found, 48 missing, 0 empty, 0 duplicate, for 5000 images):
Image sizes 320 - 640 train, 640 test
Using 8 dataloader workers
Starting training for 1 epochs...

 Epoch   gpu_mem      GIoU       obj       cls     total   targets  img_size
   0/0     9.99G      2.46      2.75     0.967      6.18        72       384
           Class    Images   Targets         P         R   [email protected]        F1: 100%|███████████████████████████████████████████████████████████████████████████████████| 625/625 [01:55<00:00,  5.41it/s]
             all     5e+03  3.63e+04     0.378     0.666     0.556      0.47

Speed: 13.7/1.4/15.1 ms inference/NMS/total per 640x640 image at batch-size 8

COCO mAP with pycocotools...
Traceback (most recent call last):
File "train.py", line 411, in
train(hyp) # train normally
File "train.py", line 317, in train
dataloader=testloader)
File "/home/jingwei/yolov3/test.py", line 211, in test
cocoGt = COCO(glob.glob('/home/jingwei/yolov3/data/annotations/instances_val*.json')[0]) # initialize COCO ground truth api
IndexError: list index out of range
(base) jingwei@jingwei:~/yolov3$ python -c "import numpy; print(numpy.__version__)"
1.17.5
(base) jingwei@jingwei:~/yolov3$

[jingweirobot]

@glenn-jocher
numpy =1.17.5
I still met the same issue.

[jingweirobot]

@glenn-jocher solved this issue. That is not Numpy version issue.
Using
cocoRoot = "/home/jingwei/yolov3/data/coco/"
dataType = "val2017"

        File = os.path.join(cocoRoot, f'annotations/instances_{dataType}.json')

        cocoGt=COCO(File)
   to replace 
         cocoGt = COCO(glob.glob('/home/jingwei/yolov3/data/coco/annotations/instances_val*.json')[0])

Thanks a lot. If anyone meets this problem, please mention and try this.

[glenn-jocher]

@jingweirobot ah I see, thanks for the update! Yes the current implementation is designed for coco and yolov3 folders next to each other:

/dir
    /coco
    /yolov3

[Rajat-Mehta]

@glenn-jocher I have trained yolov3 on my custom dataset. Now I want to generate AP scores for different scales (AP scores across scales) that are available in pycocotools.

I tried to run test.py but getting this error:

COCO mAP with pycocotools...
Traceback (most recent call last):
  File "test.py", line 261, in <module>
    opt.augment)
  File "test.py", line 214, in test
    cocoGt = COCO(glob.glob('../coco/annotations/instances_val*.json')[0])  # initialize COCO ground truth api
IndexError: list index out of range

any idea on how to get these scores for custom datasets?

[glenn-jocher]

@Rajat-Mehta recommend yolov5. mAP by size is not available outside of pycocotools.

[Rajat-Mehta]

@glenn-jocher is it available in yolov5?