Mask_rcnn: mrcnn expects wrong shape for input image

[rbavery]

Hello! I'm trying to train mrcnn on 8 channel imagery to detect agriculture. (channels are RGBN for on/off growing season). Here a notebook with my code: https://github.com/rbavery/deepimagery/blob/master/notebooks/Train_WV2_MRCNN.ipynb

I've followed the steps on the wiki for adjusting the mrcnn config and dataset classes for more than 3 channels, except for the last step since I don't want to use pregenerated weights and want the model to initialize those randomly: https://github.com/matterport/Mask_RCNN/wiki

The strange error is below, somewhere, something is expecting an array of shape (None,None, 3) but gets shape (256, 256, 8) instead which is the shape it should be expecting. You'll see that I've set the IMAGE_SHAPE channel to accept N, changed MEAN_PIXEL for 8 channels, and changed load_image to accept 8 channels.

I've tried to dig through the stacktrace to find why it would expect (None,None, 3) but am quite lost. Any help/suggestions are appreciated. I'm also confused why it is throwing RuntimeError: sequence argument must have length equal to input rank and if this issue is seperate or not from the other issue.

Please let me know if you need anymore information.

Train all layers
WARNING:tensorflow:From /opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.

Starting at epoch 0. LR=0.001

Checkpoint Path: /home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/models/wv2-gridded20180624T1338/mask_rcnn_wv2-gridded_{epoch:04d}.h5
Selecting layers to train
conv1                  (Conv2D)
bn_conv1               (BatchNorm)
res2a_branch2a         (Conv2D)
bn2a_branch2a          (BatchNorm)
res2a_branch2b         (Conv2D)
bn2a_branch2b          (BatchNorm)
res2a_branch2c         (Conv2D)
res2a_branch1          (Conv2D)
bn2a_branch2c          (BatchNorm)
bn2a_branch1           (BatchNorm)
res2b_branch2a         (Conv2D)
bn2b_branch2a          (BatchNorm)
res2b_branch2b         (Conv2D)
bn2b_branch2b          (BatchNorm)
res2b_branch2c         (Conv2D)
bn2b_branch2c          (BatchNorm)
res2c_branch2a         (Conv2D)
bn2c_branch2a          (BatchNorm)
res2c_branch2b         (Conv2D)
bn2c_branch2b          (BatchNorm)
res2c_branch2c         (Conv2D)
bn2c_branch2c          (BatchNorm)
res3a_branch2a         (Conv2D)
bn3a_branch2a          (BatchNorm)
res3a_branch2b         (Conv2D)
bn3a_branch2b          (BatchNorm)
res3a_branch2c         (Conv2D)
res3a_branch1          (Conv2D)
bn3a_branch2c          (BatchNorm)
bn3a_branch1           (BatchNorm)
res3b_branch2a         (Conv2D)
bn3b_branch2a          (BatchNorm)
res3b_branch2b         (Conv2D)
bn3b_branch2b          (BatchNorm)
res3b_branch2c         (Conv2D)
bn3b_branch2c          (BatchNorm)
res3c_branch2a         (Conv2D)
bn3c_branch2a          (BatchNorm)
res3c_branch2b         (Conv2D)
bn3c_branch2b          (BatchNorm)
res3c_branch2c         (Conv2D)
bn3c_branch2c          (BatchNorm)
res3d_branch2a         (Conv2D)
bn3d_branch2a          (BatchNorm)
res3d_branch2b         (Conv2D)
bn3d_branch2b          (BatchNorm)
res3d_branch2c         (Conv2D)
bn3d_branch2c          (BatchNorm)
res4a_branch2a         (Conv2D)
bn4a_branch2a          (BatchNorm)
res4a_branch2b         (Conv2D)
bn4a_branch2b          (BatchNorm)
res4a_branch2c         (Conv2D)
res4a_branch1          (Conv2D)
bn4a_branch2c          (BatchNorm)
bn4a_branch1           (BatchNorm)
res4b_branch2a         (Conv2D)
bn4b_branch2a          (BatchNorm)
res4b_branch2b         (Conv2D)
bn4b_branch2b          (BatchNorm)
res4b_branch2c         (Conv2D)
bn4b_branch2c          (BatchNorm)
res4c_branch2a         (Conv2D)
bn4c_branch2a          (BatchNorm)
res4c_branch2b         (Conv2D)
bn4c_branch2b          (BatchNorm)
res4c_branch2c         (Conv2D)
bn4c_branch2c          (BatchNorm)
res4d_branch2a         (Conv2D)
bn4d_branch2a          (BatchNorm)
res4d_branch2b         (Conv2D)
bn4d_branch2b          (BatchNorm)
res4d_branch2c         (Conv2D)
bn4d_branch2c          (BatchNorm)
res4e_branch2a         (Conv2D)
bn4e_branch2a          (BatchNorm)
res4e_branch2b         (Conv2D)
bn4e_branch2b          (BatchNorm)
res4e_branch2c         (Conv2D)
bn4e_branch2c          (BatchNorm)
res4f_branch2a         (Conv2D)
bn4f_branch2a          (BatchNorm)
res4f_branch2b         (Conv2D)
bn4f_branch2b          (BatchNorm)
res4f_branch2c         (Conv2D)
bn4f_branch2c          (BatchNorm)
res5a_branch2a         (Conv2D)
bn5a_branch2a          (BatchNorm)
res5a_branch2b         (Conv2D)
bn5a_branch2b          (BatchNorm)
res5a_branch2c         (Conv2D)
res5a_branch1          (Conv2D)
bn5a_branch2c          (BatchNorm)
bn5a_branch1           (BatchNorm)
res5b_branch2a         (Conv2D)
bn5b_branch2a          (BatchNorm)
res5b_branch2b         (Conv2D)
bn5b_branch2b          (BatchNorm)
res5b_branch2c         (Conv2D)
bn5b_branch2c          (BatchNorm)
res5c_branch2a         (Conv2D)
bn5c_branch2a          (BatchNorm)
res5c_branch2b         (Conv2D)
bn5c_branch2b          (BatchNorm)
res5c_branch2c         (Conv2D)
bn5c_branch2c          (BatchNorm)
fpn_c5p5               (Conv2D)
fpn_c4p4               (Conv2D)
fpn_c3p3               (Conv2D)
fpn_c2p2               (Conv2D)
fpn_p5                 (Conv2D)
fpn_p2                 (Conv2D)
fpn_p3                 (Conv2D)
fpn_p4                 (Conv2D)
In model:  rpn_model
    rpn_conv_shared        (Conv2D)
    rpn_class_raw          (Conv2D)
    rpn_bbox_pred          (Conv2D)
mrcnn_mask_conv1       (TimeDistributed)
mrcnn_mask_bn1         (TimeDistributed)
mrcnn_mask_conv2       (TimeDistributed)
mrcnn_mask_bn2         (TimeDistributed)
mrcnn_class_conv1      (TimeDistributed)
mrcnn_class_bn1        (TimeDistributed)
mrcnn_mask_conv3       (TimeDistributed)
mrcnn_mask_bn3         (TimeDistributed)
mrcnn_class_conv2      (TimeDistributed)
mrcnn_class_bn2        (TimeDistributed)
mrcnn_mask_conv4       (TimeDistributed)
mrcnn_mask_bn4         (TimeDistributed)
mrcnn_bbox_fc          (TimeDistributed)
mrcnn_mask_deconv      (TimeDistributed)
mrcnn_class_logits     (TimeDistributed)
mrcnn_mask             (TimeDistributed)
/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/keras/engine/training_generator.py:44: UserWarning: Using a generator with `use_multiprocessing=True` and multiple workers may duplicate your data. Please consider using the`keras.utils.Sequence class.
  UserWarning('Using a generator with `use_multiprocessing=True`'
Epoch 1/40
ERROR:root:Error processing image {'id': 'ZA0488124', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0488124/image/ZA0488124_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rankERROR:root:Error processing image {'id': 'ZA0116365', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0116365/image/ZA0116365_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank

ERROR:root:Error processing image {'id': 'ZA0493712', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0493712/image/ZA0493712_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rankERROR:root:Error processing image {'id': 'ZA0557249', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0557249/image/ZA0557249_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank

ERROR:root:Error processing image {'id': 'ZA0161611', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0161611/image/ZA0161611_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0488132', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0488132/image/ZA0488132_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0485502', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0485502/image/ZA0485502_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0809727', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0809727/image/ZA0809727_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rankERROR:root:Error processing image {'id': 'ZA0811581', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0811581/image/ZA0811581_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank

ERROR:root:Error processing image {'id': 'ZA0807883', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0807883/image/ZA0807883_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0158188', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0158188/image/ZA0158188_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0489761', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0489761/image/ZA0489761_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0159037', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0159037/image/ZA0159037_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0793314', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0793314/image/ZA0793314_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0806260', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0806260/image/ZA0806260_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0807887', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0807887/image/ZA0807887_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
ERROR:root:Error processing image {'id': 'ZA0116365', 'source': 'wv2', 'path': '/home/rave/tana-crunch/waves/deepimagery/data/raw/wv2/train/ZA0116365/image/ZA0116365_OSGS_ms.tif'}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1695, in data_generator
    use_mini_mask=config.USE_MINI_MASK)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 1218, in load_image_gt
    mask = utils.resize_mask(mask, scale, padding, crop)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/utils.py", line 513, in resize_mask
    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/interpolation.py", line 606, in zoom
    zoom = _ni_support._normalize_sequence(zoom, input.ndim)
  File "/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/scipy/ndimage/_ni_support.py", line 65, in _normalize_sequence
    raise RuntimeError(err)
RuntimeError: sequence argument must have length equal to input rank
0
7
(4, 256, 256, 8)
[]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-cd5e9087bf26> in <module>()
      3 model = modellib.MaskRCNN(mode="training", config=config,
      4                                   model_dir=MODEL_DIR)
----> 5 train(model, ROOT_DIR, "train")

<ipython-input-5-03b570344448> in train(model, dataset_dir, subset)
     30                 epochs=40,
     31                 augmentation=augmentation,
---> 32                 layers='all')
     33 
     34 

/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py in train(self, train_dataset, val_dataset, learning_rate, epochs, layers, augmentation)
   2326             max_queue_size=100,
   2327             workers=workers,
-> 2328             use_multiprocessing=True,
   2329         )
   2330         self.epoch = max(self.epoch, epochs)

/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name +
     90                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1427             use_multiprocessing=use_multiprocessing,
   1428             shuffle=shuffle,
-> 1429             initial_epoch=initial_epoch)
   1430 
   1431     @interfaces.legacy_generator_methods_support

/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    189                 outs = model.train_on_batch(x, y,
    190                                             sample_weight=sample_weight,
--> 191                                             class_weight=class_weight)
    192 
    193                 if not isinstance(outs, list):

/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/keras/engine/training.py in train_on_batch(self, x, y, sample_weight, class_weight)
   1215             x, y,
   1216             sample_weight=sample_weight,
-> 1217             class_weight=class_weight)
   1218         if self._uses_dynamic_learning_phase():
   1219             ins = x + y + sample_weights + [1.]

/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
    751             feed_input_shapes,
    752             check_batch_axis=False,  # Don't enforce the batch size.
--> 753             exception_prefix='input')
    754 
    755         if y is not None:

/opt/anaconda3/envs/maskrcnn/lib/python3.6/site-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
    134                             ': expected ' + names[i] + ' to have shape ' +
    135                             str(shape) + ' but got array with shape ' +
--> 136                             str(data_shape))
    137     return data
    138 

ValueError: Error when checking input: expected input_image to have shape (None, None, 3) but got array with shape (256, 256, 8)

[rbavery]

More info on this issue. It looks like in envs/maskrcnn/lib/python3.6/site-packages/keras/engine/training.py, the train_on_batch function is receiving an argument "x" that represents the following (from the func docstring):

    def train_on_batch(self, x, y,
                       sample_weight=None,
                       class_weight=None):
        """Runs a single gradient update on a single batch of data.

        # Arguments
            x: Numpy array of training data,
                or list of Numpy arrays if the model has multiple inputs.
                If all inputs in the model are named,
                you can also pass a dictionary
                mapping input names to Numpy arrays.

In my case x is a list of arrays (which for some reason is length 7 even though I set batch size to 4). I expected each of these to be shape (256 256 8) but here is the output when I print the shape of each element of x:

(4, 256, 256, 8)
(4, 14)
(4, 16368, 1)
(4, 256, 4)
(4, 100)
(4, 100, 4)
(4, 256, 256, 100)

It looks like the first element is my training data, I'm not sure what the rest is.

[rbavery]

I found the issue. In mrcnn/model.py I had to set

# Inputs
        input_image = KL.Input(
            shape=[None, None, 8], name="input_image") ### changed from 3

[rbavery]

Might want to put this extra step in the wiki for multi channel inputs.

[fastlater]

@rbavery I am using 1024x1024 png IMAGES and I get the same error than you: RuntimeError: sequence argument must have length equal to input rank. It is strange because I am using the same format and same shape, [1024,1024,3] for the image and the masks (black and white) just like it is used for the nucleus demo.
Any advise how to solve it.

[fastlater]

Solved. I forgot to change the load_mask method

[zengqiang12]

@fastlater I am experiencing similar issues that seem caused by mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0) this line in util resize_mask function. Does your problem caused by this as well? May I know how did you solve this?

[keacifer]

It ' been solved by reshape the input mask in load_mask , the shape should be (W, H ,Channels), but in the code it is （W， H， Channels ， 1）， so use numpy.squeeze() to remove the 1 and it works.
if the mask is （W，H，1）e.t the black and white image, it will work without changing anything.

[yogesh660]

Solved. I forgot to change the load_mask method

can you tell me what did you change?

[SpicySyntax]

Solved. I forgot to change the load_mask method

Can you elaborate? what did you change to get this to work?

[lijo381]

you are reading the mask as a 3 channel image but in mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0) it is expecting a single channel image, so you need to convert the mask image into single channel image when you read it by changing to
m = skimage.io.imread(os.path.join(mask_dir, f), as_gray=True).astype(np.bool) from
m = skimage.io.imread(os.path.join(mask_dir, f)).astype(np.bool)
This should do the trick

[cj-l]

I found the issue. In mrcnn/model.py I had to set

# Inputs
        input_image = KL.Input(
            shape=[None, None, 8], name="input_image") ### changed from 3

You can now change config.IMAGE_SHAPE[2] to equal your number of channels to do this

[Ostyk]

@fastlater I am experiencing similar issues that seem caused by mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0) this line in util resize_mask function. Does your problem caused by this as well? May I know how did you solve this?

I'm still struggling with this despite trying all the fixes in this forum. However, when I perform the same action on the same image, outside the mask r cnn code, it works.

[patankar-anant123]

I found the issue. In mrcnn/model.py I had to set

# Inputs
        input_image = KL.Input(
            shape=[None, None, 8], name="input_image") ### changed from 3

After Doing This I am getting Error: "ValueError: Dimension 0 in both shapes must be equal, but are 7 and 64 for 'Assign' (op: 'Assign') with input shapes: [7,7,8,64], [64,3,7,7]."