Maskrcnn-benchmark: ValueError during loading of dataset

[maxsenh]

❓ Questions and Help

Hi
I tried loading a custom dataset in coco format and followed the descriptions regarding the symlink for the dataset. As a background, I labelled the images on Labelbox, exported the annotation file in JSON WKT format, downloaded the images saved as URL in the annotation file, split the annotation file and the images into train, validation and test data and converted the annotation files to JSON COCO format.
For using maskrcnn_benchmark, I configured my own .yaml file so that it has the correct paths:

...
DATASETS:
  TRAIN: ("coco_nuclei_train", "coco_nuclei_val")
  TEST: ("coco_nuclei_test",)
...

I also configured my own paths_catalog.py file, where I specified the correct paths to the symlinks in maskrcnn-benchmark/datasets/coco_nuclei which looks like this:

...
DATASETS = {
        "coco_nuclei_train": { 
            "img_dir": "coco_nuclei/train",
            "ann_file": "coco_nuclei/annotation/train_coco.json"
        },
        "coco_nuclei_val": { 
            "img_dir": "coco_nuclei/train",
            "ann_file": "coco_nuclei/annotation/val_coco.json"
        },
        "coco_nuclei_test": { 
            "img_dir": "coco_nuclei/test",
            "ann_file": "coco_nuclei/annotation/test_coco.json"
        }
    }
...

Still when I run the training, it reads in the annotation files without problem but apparently cannot load the training/validation data as it raises a ValueError:

loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
loading annotations into memory...
Done (t=0.03s)
creating index...
index created!
[<maskrcnn_benchmark.data.datasets.concat_dataset.ConcatDataset object at 0x2acd4fd43160>]
Traceback (most recent call last):
  File "tools/train_net.py", line 172, in <module>
    main()
  File "tools/train_net.py", line 165, in main
    model = train(cfg, args.local_rank, args.distributed)
  File "tools/train_net.py", line 60, in train
    start_iter=arguments["iteration"],
  File "/home/max/github/maskrcnn-benchmark/maskrcnn_benchmark/data/build.py", line 159, in make_data_loader
    sampler = make_data_sampler(dataset, shuffle, is_distributed)
  File "/home/max/github/maskrcnn-benchmark/maskrcnn_benchmark/data/build.py", line 63, in make_data_sampler
    sampler = torch.utils.data.sampler.RandomSampler(dataset)
  File "/home/max/anaconda3/envs/pt_mask_Rcnn_env/lib/python3.6/site-packages/torch/utils/data/sampler.py", line 64, in __init__
    "value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integeral value, but got num_samples=0

I checked the paths multiple times, but maybe this issue occurred with someone else as well, help will be MUCH appreciated! If I am missing important information please let me know.
Thank you in advance!

[fmassa]

I believe that the dataset that you loaded might end up being empty.

Here is what I'd recommend:
add a

from IPython import embed; embed()

in this part of the code
https://github.com/facebookresearch/maskrcnn-benchmark/blob/d3fed42afe59910dd650308c426640d183d044b5/maskrcnn_benchmark/data/build.py#L46
and check that the dataset that is returned there is actually valid. For example, check that len(datasets[0]) is what you would expect, and also that datasets[0][0] returns the image and the annotation in the BoxList format properly.
Also, do this without torch.distributed.launch, i.e., in a single GPU.

My guess is that your dataset is empty for some reason.

[maxsenh]

Thank you very much for your answer! The dataset is indeed empty, len(datasest[0]) gives 0 with single and multiple GPU training. Will now try to dismantle the code and find out why the images could not be loaded. Could the image size be a reason (my images are 2048x2048)?
I will obviously post updates if I find a solution.

[fmassa]

@maxsenh no, the image size wouldn't be a problem.

It's probably because your JSON is not in the format that COCO is expecting

[maxsenh]

Thank you again for your answer! I found that the issue was because of wrong IDs given by Labelbox. After I changed them, it worked fine.

[fmassa]

Cool, glad to know that you found the problem!

Closing this issue, but let me know if you have further questions

[Idolized22]

Thank you again for your answer! I found that the issue was because of wrong IDs given by Labelbox. After I changed them, it worked fine.
Hey,
Can you please tell me what worng ID has been given by Labelbox ,
I am having a similar problem.

[maxsenh]

Sure, so what I did is I downloaded the annotation file in JSON WKT format and then used the tool labelbox2coco to convert the JSON file into coco format. Then I changed in 'images' per image the value of the 'file_name' to the correct filename and removed the URL given by Labelbox (I downloaded all the images first) and changed 'id' to an integer corresponding to the filename. Then I changed per image in 'annotations' the 'image_id' to the integer corresponding to the filename.
It is maybe not the best way but it worked! Hope it helps