Mask_rcnn: AttributeError: 'list' object has no attribute 'values'

[eyildiz-ugoe]

As described in the balloon example, I've used VIA tool to annotate my images. I've used circles, polygons, etc. and gave each class its respective name.

Now that I want to load the dataset, again, using the same code, and I get the following error:

AttributeError                            Traceback (most recent call last)
<ipython-input-9-048a8da58a4c> in <module>()
      1 # Load validation dataset
      2 dataset = components.ComponentsDataset()
----> 3 dataset.load_components(COMPONENTS_DIR, "val")
      4 
      5 # Must call before using the dataset

~/workspace/Mask_RCNN/samples/components/components.py in load_components(self, dataset_dir, subset)
    122             # the outline of each object instance. There are stored in the
    123             # shape_attributes (see json format above)
--> 124             polygons = [r['shape_attributes'] for r in a['regions'].values()]
    125 
    126             # load_mask() needs the image size to convert polygons to masks.

AttributeError: 'list' object has no attribute 'values'

What could be the problem? Has anyone tried loading the annotated data which contains circles? (not only polygons)

[benjamin-taheri]

The samples only accept the polygons. That's why you get this error. You can either re-annotate the circles or use something like this to convert the circles to polygons in your json file:

import json
from pprint import pprint

with open('via_region_data(val).json') as f:
    data = json.load(f)

    for attr, val in data.items():
        for attr2, val2 in val.items():
            if attr2 == 'regions':
                for attr3, val3 in val2.items():
                    if val3['shape_attributes']['name'] == 'circle':
                        cx = val3['shape_attributes']['cx']
                        cy = val3['shape_attributes']['cy']
                        r = val3['shape_attributes']['r']
                        all_points_x = [cx, cx - 1.5 * r, cx, cx + 1.5 * r, cx]
                        all_points_y = [cy - 1.5 * r, cy, cy + 1.5 * r, cy, cy - 1.5 * r]
                        val3['shape_attributes']['cx'] = all_points_x
                        val3['shape_attributes']['cy'] = all_points_y

                        val3['shape_attributes']['all_points_x'] = val3['shape_attributes'].pop('cx')
                        val3['shape_attributes']['all_points_y'] = val3['shape_attributes'].pop('cy')
                        val3['shape_attributes']['name'] = 'polygon'


pprint(data)

with open('via_region_data-val.json', 'w') as f:
    json.dump(data, f)

[eyildiz-ugoe]

@sajjad-taheri thank you for the code, but when I run I get the following error:

Traceback (most recent call last):
  File "polygon_fixer.py", line 10, in <module>
    for attr3, val3 in val2.items():
AttributeError: 'list' object has no attribute 'items'

[skt7]

VIA has changed JSON formatting in later versions. Now instead of a dictionary, "regions" has a list.

Older Version Format:
"regions":{ "0":{<data_0>}, "1":{<data_1>}, . . . }

Newer Version Format:
"regions":{ [{<data_0>},{<data_1>},...] }

So you can do is, change _line 10_ from for attr3, val3 in val2.items(): to for val3 in val2:

[eyildiz-ugoe]

For some reason the data is not getting loaded no matter what I try. I keep getting:

polygons = [r['shape_attributes'] for r in a['regions'].values()]

AttributeError: 'list' object has no attribute 'values'

Here is my function that is supposed to load the dataset:

class ComponentsDataset(utils.Dataset):

    def load_components(self, dataset_dir, subset):
        """Load a subset of the Balloon dataset.
        dataset_dir: Root directory of the dataset.
        subset: Subset to load: train or val
        """
        # Add classes. We have only one class to add.
        self.add_class("components", 1, "screw")
        self.add_class("components", 2, "lid")

        # Train or validation dataset?
        assert subset in ["train", "val"]
        dataset_dir = os.path.join(dataset_dir, subset)

        # Load annotations
        # VGG Image Annotator saves each image in the form:
        # { 'filename': '28503151_5b5b7ec140_b.jpg',
        #   'regions': {
        #       '0': {
        #           'region_attributes': {name:'screw'},
        #           'shape_attributes': {
        #               'all_points_x': [...],
        #               'all_points_y': [...],
        #               'name': 'polygon'}},
        #       ... more regions ...
        #   },
        #   'size': 100202
        # }
        # We mostly care about the x and y coordinates of each region
        annotations = json.load(open(os.path.join(dataset_dir, "via_region_data.json")))
        annotations = list(annotations.values())  # don't need the dict keys

        # The VIA tool saves images in the JSON even if they don't have any
        # annotations. Skip unannotated images.
        annotations = [a for a in annotations if a['regions']]

        # Add images
        for a in annotations:
            # Get the x, y coordinaets of points of the polygons that make up
            # the outline of each object instance. There are stored in the
            # shape_attributes (see json format above)
            polygons = [r['shape_attributes'] for r in a['regions'].values()]
            names = [r['region_attributes'] for r in a['regions'].values()]
            # load_mask() needs the image size to convert polygons to masks.
            # Unfortunately, VIA doesn't include it in JSON, so we must read
            # the image. This is only managable since the dataset is tiny.
            image_path = os.path.join(dataset_dir, a['filename'])
            image = skimage.io.imread(image_path)
            height, width = image.shape[:2]

            self.add_image(
                "components",
                image_id=a['filename'],  # use file name as a unique image id
                path=image_path,
                width=width, height=height,
                polygons=polygons,
                names=names)

Even after converting them to polygons with the suggested piece of code and the modification, I still cannot load my dataset, hence I cannot train or do anything useful with the network. It's really frustrating to have the entire dataset annotated using the _suggested_ annotation tool (VIA) and then not being able to load it. At this point the developers should really release a fix or stop suggesting that tool to annotate, or underline specifically that users should not use anything else than polygons. This is such a time waste as of now since the dataset cannot be loaded.

[skt7]

You didn't read the last comment I clearly mentioned that

VIA has changed JSON formatting in later versions. Now instead of a dictionary, "regions" has a list

change
polygons = [r['shape_attributes'] for r in a['regions'].values()]
to
polygons = [r['shape_attributes'] for r in a['regions']]

and it will work.

And yes you are right they should update that line of code to make it work on the latest update of VIA or better make it compatible for all the versions.

@waleedka What say?

[eyildiz-ugoe]

@skt7 Yes, now the dataset is loaded. Also this had to be changed:

names = [r['region_attributes'] for r in a['regions'].values()]

to this:

names = [r['region_attributes'] for r in a['regions']]

Thank you for the contribution!

[mymultiverse]

The samples only accept the polygons. That's why you get this error. You can either re-annotate the circles or use something like this to convert the circles to polygons in your json file:
Thanks for the script I notices one thing it the number of polynomial vertex is not sufficient while conversion from circle, It raises error during checking data set. I modified your script with math a bit so that N vertex can be chosen on circle.

import json
from pprint import pprint
import numpy as np

N = 20

thita = np.linspace(-np.pi, np.pi, N)
with open('via_region_dataCir.json') as f:
    data = json.load(f)

    for attr, val in data.items():
        for attr2, val2 in val.items():
            if attr2 == 'regions':
                for attr3, val3 in val2.items():
                    if val3['shape_attributes']['name'] == 'circle':
                        cx = val3['shape_attributes']['cx']
                        cy = val3['shape_attributes']['cy']
                        r = val3['shape_attributes']['r']

                        all_points_x = [cx+r*np.cos(i)for i in thita]
                        all_points_y = [cy+r*np.sin(i) for i in thita]

                        val3['shape_attributes']['cx'] = all_points_x
                        val3['shape_attributes']['cy'] = all_points_y

                        val3['shape_attributes']['all_points_x'] = val3['shape_attributes'].pop('cx')
                        val3['shape_attributes']['all_points_y'] = val3['shape_attributes'].pop('cy')
                        val3['shape_attributes']['name'] = 'polygon'


pprint(data)

with open('via_region_data.json', 'w') as f:
    json.dump(data, f)

[girijesh97]

@mymultiverse @sajjad-taheri Hi, How can I convert polyline and rect to ploygon?