Mask_rcnn: How to set NUM_CLASSES 14 instead of NUM_CLASSES 81......? Where to modify the code including model.py and anyother file.....?

[matiqul]

[MPForte]

following the steps of #161 and #104.

1. In coco.py in load_coco function change class_ids=None with the ids that you want to train, e.g. class_ids=[0,43,77] (BG, fork, scissors).
2. In config.py change NUM_CLASSES with the number of classes you want to train. But actually here it is NUM_CLASSES=1. So I'm stuck at this point. Changing only step 1 makes no difference since my model continues to train everything. If you manage to go on, let me know.

[toby-coleman]

Currently in the process of testing this, but if I understand correctly:

1. Edit coco.py with class_ids=[43, 77] (no need to include background class);
2. Edit NUM_CLASSES = 3 again in coco.py; then
3. Either:
   
   
   
   • Start training from the imagenet weights, i.e. python coco.py train --dataset=/path/to/data --model=imagenet; or
   
   
   • To train from the pre-trained COCO weights, edit coco.py on line 469, adding exclude=['mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask'] to the load_weights call. Then run using python coco.py train --dataset=/path/to/data --model=coco
   
   

[MPForte]

Do you mean: model.load_weights(model_path, by_name=True, exclude=['mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask']) right?

And do you know how to do python coco.py train --dataset=/path/to/data --model=coco using the ROS version of this code? Should it be somehow integrated into the launch file?https://github.com/akio/mask_rcnn_ros

[toby-coleman]

Yes, correct on model.load_weights.

I've not used the ROS version, but I should think the following would work:

• Use this repo to re-train the model via the steps above;
• Edit the ROS version to use your new, re-trained weights, i.e. edit this file and change COCO_MODEL_PATH to point to your new weights file, and CLASS_NAMES to be your reduced set of objects.

[anju54]

in addition with training three classes only, i want to draw the prediction box or mask on three class only.
how to do that.
in other word i can say, let the model detect all the class but while sowing it should draw box over this three classes only.
please help me to do that?

[Yiman-GO]

@paolaforte there is no need to change the num_classes in config.py. Because when you set num_classes in coco.py, you rewirte it.

[endluo]

@paolaforte how did you know the [0,43,77] is the (BG, fork, scissors) ? where can i find the class_id? THX

[mohammedayub44]

@anju54 I have the same issue, did you happen to get a solution to your query ?
@paolaforte class names displayed are in order (starting from 0) hence the mapping 0-80 (total 81)

[anju54]

Yeah what u want to know. I have executed this algorithm successfully

[mohammedayub44]

@anju54 I don't want to retrain the model but just want to display selected classes. Can you please post your changes ?

@waleedka
I was working with this
https://github.com/Tony607/colab-mask-rcnn/blob/master/Colab_Mask_R_CNN_Demo.ipynb
I can run this successfully using 80 classes, but having trouble changing no of classes to just 30.

The code runs after including "exclude= ..." but it It result jpg's have classe ID's that I have not included, like [cow, tie etc..]. Not sure what is going wrong here.

I had to modify the local samples/coco.py file include only related classes:

def load_coco(self, dataset_dir, subset, year=DEFAULT_DATASET_YEAR, class_ids=[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,26,27,58,59,60,61,62,63,64,70,71,72,73,74,78],
              class_map=None, return_coco=False, auto_download=False):
...

Below is my notebook snippet:

# Import COCO config
sys.path.append(os.path.join(ROOT_DIR, "samples/coco/"))  # To find local version
print(ROOT_DIR)
import coco

#We use a K80 GPU with 24GB memory, which can fit 3 images.
batch_size = 5

ROOT_DIR = os.getcwd()
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
VIDEO_DIR = os.path.join(ROOT_DIR, "videos")
VIDEO_SAVE_DIR = os.path.join(VIDEO_DIR, "save_2")
COCO_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
if not os.path.exists(COCO_MODEL_PATH):
    utils.download_trained_weights(COCO_MODEL_PATH)

class CocoConfig(Config):
    """Configuration for training on MS COCO.
    Derives from the base Config class and overrides values specific
    to the COCO dataset.
    """
    # Give the configuration a recognizable name
    NAME = "coco"

    # We use a GPU with 12GB memory, which can fit two images.
    # Adjust down if you use a smaller GPU.
    IMAGES_PER_GPU = 1

    # Uncomment to train on 8 GPUs (default is 1)
    # GPU_COUNT = 8

    # Number of classes (including background)
    #NUM_CLASSES = 37  # COCO has 80 classes
    NUM_CLASSES = 30

#class InferenceConfig(coco.CocoConfig):
class InferenceConfig(CocoConfig):
    # Set batch size to 1 since we'll be running inference on
    # one image at a time. Batch size = GPU_COUNT * IMAGES_PER_GPU
    GPU_COUNT = 1
    IMAGES_PER_GPU = 5
    #MAX_GT_INSTANCES = 100
    #TRAIN_ROIS_PER_IMAGE = 50
    #BACKBONE = "resnet50" #not working at all!
    #RPN_ANCHOR_STRIDE = 2
    #POST_NMS_ROIS_TRAINING = 1000
    #POST_NMS_ROIS_INFERENCE = 500
    #IMAGE_MIN_DIM = 400 #really much faster but bad results
    #IMAGE_MAX_DIM = 512
    #DETECTION_MAX_INSTANCES = 50 #a little faster but some instances not recognized

config = InferenceConfig()
config.display()

model = modellib.MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=config)
model.load_weights(COCO_MODEL_PATH, by_name=True,exclude=['mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask'])

[anju54]

if you simply want to draw that prediction bounding box over class = 30 then just modify ur visualize.py file
In my case, I wanted it for only person class so I fixed it for class = 1 in visualize.py.
here is the modification in visualize.py ( only in display_instances() method)

def display_instances(image, boxes, masks, class_ids, class_names,
scores=None, title="",
figsize=(16, 16), ax=None):
"""
boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates.
masks: [height, width, num_instances]
class_ids: [num_instances]
class_names: list of class names of the dataset
scores: (optional) confidence scores for each box
figsize: (optional) the size of the image.
"""
# Number of instances
N = boxes.shape[0]
if not N:
print("\n* No instances to display * \n")
else:
assert boxes.shape[0] == masks.shape[-1] == class_ids.shape[0]

if not ax:
    _, ax = plt.subplots(1, figsize=figsize)

# Generate random colors
colors = random_colors(N)

# Show area outside image boundaries.
height, width = image.shape[:2]
ax.set_ylim(height + 10, -10)
ax.set_xlim(-10, width + 10)
ax.axis('off')
ax.set_title(title)

masked_image = image.astype(np.uint32).copy()
t = []
t2 = []
t3 = []
t4 = []
for i in range(N):
    color = colors[i]
    class_id = class_ids[i]

    # Bounding box
    if not np.any(boxes[i]):
        # Skip this instance. Has no bbox. Likely lost in image cropping.
        continue
    if (class_id == 1):

        y1, x1, y2, x2 = boxes[i]
        p = patches.Rectangle((x1, y1), x2 - x1, y2 - y1, linewidth=2,
                              alpha=0.7, linestyle="dashed",
                              edgecolor=color, facecolor='none')
        ax.add_patch(p)
        '''arr = [y1, x1, y2, x2]
        a = np.array(arr)
        # rows, cols = a.shape
        t = np.append(t, a[0])
        t2 = np.append(t2, a[1])
        t3 = np.append(t3, a[2])
        t4 = np.append(t4, a[3])
        # print(t)'''

    # Label
    #class_id = class_ids[i]
    #print(class_id)
    score = scores[i] if scores is not None else None
    label = class_names[class_id]
    #print(label)
    if (class_id == 1):
        x = random.randint(x1, (x1 + x2) // 2)
        caption = "{} {:.3f}".format(label, score) if score else label
        ax.text(x1, y1 + 8, caption,
                color='w', size=11, backgroundcolor="none")

    # Mask
    if (class_id == 1):
        mask = masks[:, :, i]
        masked_image = apply_mask(masked_image, mask, color)

        # Mask Polygon
        # Pad to ensure proper polygons for masks that touch image edges.
        padded_mask = np.zeros(
            (mask.shape[0] + 2, mask.shape[1] + 2), dtype=np.uint8)
        padded_mask[1:-1, 1:-1] = mask
        contours = find_contours(padded_mask, 0.5)
        for verts in contours:
            # Subtract the padding and flip (y, x) to (x, y)
            verts = np.fliplr(verts) - 1
            p = Polygon(verts, facecolor="none", edgecolor=color)
            ax.add_patch(p)
'''array1 = [t, t2, t3, t4]
a1 = np.array(array1)
rr = np.transpose(a1)
d = pd.DataFrame(rr, columns=['y1', 'x1', 'y2', 'x2'])
d.to_csv('/home/anju/Documents/Mask_RCNN/a_f.csv', index=None)
print(a1)'''
ax.imshow(masked_image.astype(np.uint8))
plt.show()

[xuyanging]

How to train maskrcnn with both coco dataset and external dataset? I mean in terms of some specific classes, coco may be not enough for training.

[Master-HM]

Yeah what u want to know. I have executed this algorithm successfully

@anju54 You said you've executed this algorithm successfully, Your problem was also related to mine, so please have a look at my problem.

I'm doing a predictions for cotton plant field. I've three classes, (BG, Cotton, Weed). I don't want my model to detect or label anything else so my dataset consist of 100 Cotton field images. Which has both cotton plant and weed plant in it, then I annotated them with VIA tool. So I modified almost all code files including coco.py model.py config.py I think I also modified utils.py, and trained model. After so many bugs, and three weeks of debugging my model started training on google colab.

Above Image is copied from Inspect_plant.ipynb which is actually modified form of inspect_ballon_data.ipynb. But After training, When I started detection, I didn't got expected results. Below Images has splashed from my plant model.

It has gray-scaled whole image and returned original color pixel for detected classes, It also has significant accuracy issue, It is splashing cotton accurately but neglecting weed most of times. My major concern is weed, I need to get all weed plant coordinates from images, because I'll then drive robotic arm to pluck weeds from field.
Now I want it to return results as it did in above inspect_plant.ipynb image, that I've attached above.

Kindly help me with step-wise instruction to reach my target.
@waleedka I've seen your name in every code file, kindly reply me to get into this.