Mask_rcnn: InvalidArgumentError: scores has incompatible shape

[yerzhik]

I'm running prediction using pretrained model, images coming from camera. I'm testing the program and after a minute or so the following error appears:

File "mask_rcnn/main.py", line 83, in execute_recognition
results = model.detect([image], verbose=1)
File "/root/mask_rcnn/mrcnn/model.py", line 2524, in detect
self.keras_model.predict([molded_images, image_metas, anchors], verbose=0)
File "/opt/conda/lib/python3.6/site-packages/keras/engine/training.py", line 1169, in predict
steps=steps)
File "/opt/conda/lib/python3.6/site-packages/keras/engine/training_arrays.py", line 294, in predict_loop
batch_outs = f(ins_batch)
File "/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2715, in __call__
return self._call(inputs)
File "/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2675, in _call
fetched = self._callable_fn(*array_vals)
File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1439, in __call__
run_metadata_ptr)
File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: scores has incompatible shape
[[{{node mrcnn_detection/map/while/non_max_suppression/NonMaxSuppressionV3}} = NonMaxSuppressionV3[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mrcnn_detection/map/while/GatherV2/_4235, mrcnn_detection/map/while/GatherV2_1/_4237, mrcnn_detection/map/while/non_max_suppression/NonMaxSuppressionV3/max_output_size, mrcnn_detection/map/while/non_max_suppression/iou_threshold, mrcnn_detection/map/while/non_max_suppression/score_threshold)]]
[[{{node roi_align_mask/GatherV2/_4279}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_3071_roi_align_mask/GatherV2", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Any ideas why and how to fix it?

[h-ann]

Have the same error.
Loading model with cpu device
with tf.device("/cpu:0"): model = modellib.MaskRCNN(mode="inference", config=config, model_dir=LOGS_DIR)
and updating the tensorflow version to newer one (1.13.1) did the trick.

[yerzhik]

In my case I also was experiencing another issue, which was related with tensorflow. My version is 1.12.0 and what seems to be solving this issue is the following code:

config = tensorflow.ConfigProto() 
config.inter_op_parallelism_threads = 1 
keras.backend.set_session(tensorflow.Session(config=config))

[yerzhik]

Have the same error.
Loading model with cpu device
with tf.device("/cpu:0"): model = modellib.MaskRCNN(mode="inference", config=config, model_dir=LOGS_DIR)
and updating the tensorflow version to newer one (1.13.1) did the trick.

That is another way to solve it. Does it work well with cuda 10?

[huangh12]

I encounter this error as well. It seems like a bug of NonMaxSuppressionV3 in tf1.12.
Simply replacing the NonMaxSuppressionV3 with NonMaxSuppressionV2 solve the problem!

from tensorflow.python.ops import gen_image_ops
tf.image.non_max_suppression = gen_image_ops.non_max_suppression_v2