Detectron2: AssertionError: You must register a function with `DatasetCatalog.register`!

[Samjith888]

here i have used custom dataset and i wrote a script to get the values from and return the 'list[dict]' Then i have used the following code to register the dataset,

from detectron2.data import DatasetCatalog, MetadataCatalog
DatasetCatalog.register('train', get_kitti_data_dict(dataset_root_dir, 'train'))
MetadataCatalog.get("train").thing_classes = ["person"]

But i got following error

Traceback (most recent call last):
File "tools/get_dataset_from_kitti.py", line 94, in
DatasetCatalog.register('train', get_kitti_data_dict(dataset_root_dir, 'train'))
File "/opt/anaconda3/envs/Detectron2/lib/python3.8/site-packages/detectron2/data/catalog.py", line 37, in register
assert callable(func), "You must register a function with DatasetCatalog.register!"
AssertionError: You must register a function with DatasetCatalog.register!

The 'list[dict]' is the following
[{'file_name': '/home/samjith/0000180.jpg', 'height': 788, 'width': 1400, 'image_id': 1, 'annotations': [{'bbox': [250.0, 675.0, 23.0, 17.0], 'bbox_mode': <BoxMode.XYWH_ABS: 1>, 'area': 391.0, 'segmentation': [], 'category_id': 0}, {'bbox': [295.0, 550.0, 21.0, 20.0], 'bbox_mode': <BoxMode.XYWH_ABS: 1>, 'area': 420.0, 'segmentation': [], 'category_id': 0},..

[iskode]

Hi, I was able to reproduce your error locally as I'm trying to work with the detectron2 also.
You must provide a ' lambda function' to the 2nd argument like in the custom dataset use custom dataset tutorial:
for d in ['train']:
DatasetCatalog.register(d, lambda d: d = get_kitti_data_dict(dataset_root_dir, d))
MetadataCatalog.get(d).thing_classes = ["person"]
Normally it should work. But really I haven't gone about how to do it without the loop.
Hope it fixes your issue

[ppwwyyxx]

@iskode 's answer is correct.
see https://detectron2.readthedocs.io/tutorials/datasets.html

[Samjith888]

@iskode 's answer is correct.
see https://detectron2.readthedocs.io/tutorials/datasets.html

Do you know how to solve this without using loop !?

[ppwwyyxx]

Based on the python language, the loop posted above is equivalent to

DatasetCatalog.register('train', lambda: get_kitti_data_dict(dataset_root_dir, 'train'))
MetadataCatalog.get('train').thing_classes = ["person"]

[Samjith888]

Based on the python language, the loop posted above is equivalent to

DatasetCatalog.register('train', lambda: get_kitti_data_dict(dataset_root_dir, 'train'))
MetadataCatalog.get('train').thing_classes = ["person"]

By using the above code , i can register the dataset and trained the model by using the registered custom dataset. How to do the evaluation on the trained model and get the AP values for custom dataset?

[ppwwyyxx]

https://colab.research.google.com/drive/16jcaJoc6bCFAQ96jDe2HwtXj7BMD_-m5

[Samjith888]

https://colab.research.google.com/drive/16jcaJoc6bCFAQ96jDe2HwtXj7BMD_-m5

I got the following code for evaluation

from detectron2.evaluation import COCOEvaluator, inference_on_dataset
from detectron2.data import build_detection_test_loader
evaluator = COCOEvaluator("balloon_val", cfg, False, output_dir="./output/")
val_loader = build_detection_test_loader(cfg, "balloon_val")
inference_on_dataset(trainer.model, val_loader, evaluator)

Actually i don't have COCO json file . In my case, I just getting the data from kitti files and put it into a dictionary in the coco format, then registering the dataset. Is the above COCOEvaluator will search for a coco json file ?

I found a warning 'WARNING [02/08 14:06:13 d2.evaluation.coco_evaluation]: json_file was not found in MetaDataCatalog for 'val''

[ppwwyyxx]

https://detectron2.readthedocs.io/modules/evaluation.html#detectron2.evaluation.COCOEvaluator.__init__

[andresviana]

Hi guys, could you please check my case:
My challenge is try to use Detectron2 Beginner's Tutorial with my own project inference detection.

I'm working in colab, I want to register my own custom dataset in detectron2 and there is an AssertionError.

This is the notebook:
https://colab.research.google.com/drive/1qGNs7FJfzndGgaNpzhopcgt-IaISufNG?usp=sharing

This is the error.
AssertionError Traceback (most recent call last)
in ()
59 from detectron2.data import DatasetCatalog, MetadataCatalog
60 for d in ["train", "val"]:
---> 61 DatasetCatalog.register("streets_" + d,lambda d=d: get_street_dicts("/content/potholes/", d))
62 street_metadata = MetadataCatalog.get("streets_train")
63 dataset_dicts = get_street_dicts("/content/potholes/train")
/content/detectron2_repo/detectron2/data/catalog.py in register(name, func)
38 assert callable(func), "You must register a function with DatasetCatalog.register!"
39 assert name not in DatasetCatalog._REGISTERED, "Dataset '{}' is already registered!".format(
---> 40 name
41 )
42 DatasetCatalog._REGISTERED[name] = func
AssertionError: Dataset 'streets_train' is already registered!

Thanks for check it!

[iskode]

Hi @andresviana,
apparently you ran twice this instruction so you can't register a Dataset with an already registered Dataset name.
I suppose you're experimenting so that you'll run multiple times the same cell.
In this case, you can restart the notebook, memory will be freed and you could register the Dataset again without
any problem. But that can slow you down further.
My suggestion is to put the code below in a separate cell:

`

for i, d in enumerate(random.sample(dataset_dicts, 2)):
     # read the image with cv2
     img = cv2.imread(d["file_name"])
     visualizer = Visualizer(img[:, :, ::-1], metadata=street_metadata, scale=0.5)
     vis = visualizer.draw_dataset_dict(d)
     cv2_imshow(vis.get_image()[:, :, ::-1])
     # if you want to save the files, uncomment the line below, but keep in mind that
     # the folder inputs has to be created first
     # plt.savefig(f"./inputs/input_{i}.jpg")

Now you run once the cell with the dataset registration: DatasetCatalog.register("streets_" + d,lambda d=d: get_street_dicts("/content/potholes/", d))`
then you can run multiple times the above visualization code to see or save dataset images.
Hope it solves your problem.

[andresviana]

Thank you, very kind explanation!!
When I run

for i, d in enumerate(random.sample(dataset_dicts, 2)):
# read the image with cv2
img = cv2.imread(d["file_name"])
visualizer = Visualizer(img[:, :, ::-1], metadata=street_metadata, scale=0.5)
vis = visualizer.draw_dataset_dict(d)
cv2_imshow(vis.get_image()[:, :, ::-1])
# if you want to save the files, uncomment the line below, but keep in mind that
# the folder inputs has to be created first
# plt.savefig(f"./inputs/input_{i}.jpg")

I got this Error:
NameError: name 'dataset_dicts' is not defined

What could be wrong?

[iskode]

An advice:
don't hesitate to go over an error yourself to figure it out ! As you'll encounter so many in
your journey and you'll not always find people to answer you.

Let's analyze the error:
dataset_dicts is not defined, is the problem. So the question is where do you define it in the first place
in your code. Go there and check if it has been really defined, re-run the cell.
In your case here, I can see in the colab notebook that you got an error in the previous cell:

   ----> 2     DatasetCatalog.register("streets_" + d,lambda d=d: get_street_dicts("/content/potholes/"+ d))
  3 street_metadata = MetadataCatalog.get("streets_train")
  4 dataset_dicts = get_street_dicts("/content/potholes/train")

In most cases, the error 'Traceback' is very clear. You see that the error occurred before dataset_dicts = get_street_dicts("/content/potholes/train"). Thus dataset_dicts = get_street_dicts("/content/potholes/train") was not executed and so dataset_dicts is not defined !
You can re-start your notebook and re-run it should be fine.
I could have said "restart your notebook and it'll be fine". But my point here is: take a little time to understand what's going on.
Don't hesitate to dispatch each line in a cell and look at variables before and after running each cell to deeply understand
each step.
And I'm sure if you do that, you'll not ask these questions again as you'll find the answer.
Best of luck.

[andresviana]

You are absolutely right, I'll take your advice, and again thank you so much for be patient.

[QiyaoWei]

Hi,

I have a similar issue, and I didn't manage to resolve it with any of the methods above. I did not change the code at all, just starting off and trying to run apply_net.py

Traceback (most recent call last):
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/apply_net.py", line 20, in
from projects.DensePose.densepose import add_densepose_config
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/__init__.py", line 12, in
from .utils.transform import load_from_cfg
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/utils/transform.py", line 6, in
from densepose import DensePoseTransformData
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/__init__.py", line 2, in
from .data.datasets import builtin # just to register data
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/data/__init__.py", line 3, in
from .build import (
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/data/build.py", line 23, in
from .datasets.coco import DENSEPOSE_KEYS_WITHOUT_MASK as DENSEPOSE_COCO_KEYS_WITHOUT_MASK
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/data/datasets/__init__.py", line 3, in
from . import builtin # ensure the builtin datasets are registered
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/data/datasets/builtin.py", line 10, in
register_coco_datasets(COCO_DATASETS, DEFAULT_DATASETS_ROOT)
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/data/datasets/coco.py", line 324, in register_datasets
register_dataset(dataset_data, datasets_root)
File "/Users/qiyaowei/Desktop/detectron2/projects/DensePose/densepose/data/datasets/coco.py", line 303, in register_dataset
DatasetCatalog.register(dataset_data.name, load_annotations)
File "/Users/qiyaowei/Desktop/detectron2/detectron2/data/catalog.py", line 40, in register
name
AssertionError: Dataset 'densepose_coco_2014_train' is already registered!

Process finished with exit code 1

I would like to point out that I tried changing
DatasetCatalog.register(dataset_data.name, load_annotations)
to
DatasetCatalog.register(dataset_data.name, lambda: load_annotations())
as the previous answers suggest, but still encountered the same error. The code is run in pycharm.
Any help would be appreciated. Thanks!