Vision: Accounting for class_id in cityscapes dataset

[prabhuteja12]

Hi,

In the dataset class for Cityscapes, the classes with their meta-data is mentioned as a namedtuple. However, this information is not used anywhere. Especially for semantic segmenation, the train_id should be used to convert the pixel value (of the label image) to the training label.

Is this planned to be added anytime soon?

Edit: There seems to have been a commit that does something like this and that hasn't been merged.
https://github.com/pytorch/vision/pull/1025/commits/39a7a2fb6015792b6c32fd7ee3eeac9774f712a4

[fmassa]

For now, I'd recommend adding a transform that modifies the label to the desired format.

Before merging something like the PR you mentioned, we need to have a more structured way of addressing this.

[fmassa]

also cc @TheCodez , as you have sent the original PRs

[TheCodez]

I would also recommend using a transform even when having some helper methods available.
For example like here: https://github.com/TheCodez/pytorch-GoogLeNet-FCN/blob/master/googlenet_fcn/datasets/transforms/transforms.py#L32-L37

[prabhuteja12]

@TheCodez Thanks for the link! Thats looks great for my use!

@fmassa What do mean by structured way that you mentioned?

[RJT1990]

Seconding @TheCodez approach.

I've been doing something similar-ish (but keeping the ID logic within the transform) - also using dual transforms logic.

class ConvertCityscapesIds(object):
    """Converts Cityscape masks - adds an ignore index"""

    def __init__(self, ignore_index):
        self.ignore_index = ignore_index

        self.id_to_trainid = {-1: ignore_index, 0: ignore_index, 1: ignore_index, 2: ignore_index, 3: ignore_index,
                              4: ignore_index, 5: ignore_index, 6: ignore_index, 7: 0, 8: 1, 9: ignore_index,
                              10: ignore_index, 11: 2, 12: 3, 13: 4, 14: ignore_index, 15: ignore_index, 16: ignore_index,
                              17: 5, 18: ignore_index, 19: 6, 20: 7, 21: 8, 22: 9, 23: 10, 24: 11, 25: 12, 26: 13, 27: 14,
                              28: 15, 29: ignore_index, 30: ignore_index, 31: 16, 32: 17, 33: 18}

    def __call__(self, image, target):

        target_copy = target.clone()

        for k, v in self.id_to_trainid.items():
            target_copy[target == k] = v

        return image, target_copy

[fmassa]

Yes, the way @RJT1990 is how I do it myself as well.

@prabhuteja12 by structured way, I meant a similar comment as in https://github.com/pytorch/vision/issues/1067#issuecomment-507599030