Cvat: CVAT Images Validator

[Rub21]

Our team at Development Seed has been working with the CVAT platform for a while to label very large image sets (~60k street view images in the past few months). We found that it works quite well for our purposes, and we wanted to compile some feedback that could help make it even better.

The quality of the labeling is very important for our work, so we wanted to validate our labels as efficiently as possible. We created a tool to speed up the validation process, which works with the same images stored in the CVAT server. It allowed our labeling team to increase the speed of this step by about 10X.

The validator tool overlays the bbox labels on top of the CVAT images and then allows users to scroll through many images very quickly (rather than frame by frame). Users can set the range of images they wish to validate, and some simple parameters about the visualization. If a user sees any incorrect labels, they can click the image thumbnail and be taken to the CVAT labeling platform to fix that individual image.

E.g

The Github repo for this tool is available here: https://github.com/developmentseed/cvat-images-validator

Currently, installing the tool in the server is still a bit difficult. We hope this could be included as a plugin to CVAT sometime in the future. Let us know if that sounds interesting to you, and maybe we can meet to discuss this.

[nmanovic]

@Rub21 , thanks for the feedback. Really great to see that you have found the tool useful for your tasks.

It is a really awesome example how to improve CVAT for the specific task. I believe if we can draw a grid of images inside cvat-canvas (https://github.com/opencv/cvat/tree/develop/cvat-canvas) we can easily implement similar functionality inside CVAT without an external application.

The same feature can be used (and it is in our plans) for classification task (e.g. filtering bad images). One more way to speed up your task is to use DL models for validation (use responses with high confidence to validate some objects on an image).

@Rub21, please let me know if you can contribute the grid feature into cvat-canvas js library.

I will keep the issue open till it is resolved. In v1.0.0 we are going to update our UI (use cvat-ui, cvat-core, cvat-canvas components). Next release will be concentrated on validation scenarios like you already mentioned.

[Rub21]

Hey @nmanovic !

@Rub21, please let me know if you can contribute the grid feature into cvat-canvas js library.

Having more time these weeks would be good from my side helping to add that functionality directly in the cvat tool. Any suggestion where could we start it. ?