Incubator-mxnet: DataLoader Operator stack inferring shapes failed

[mikethreeacer]

Description

I tried to use custom dataset with dataloader to train a model.
The error appears when use dataloader to get a batch.
Images are loaded in dataset correctly but all images and labels are None in batch.

import glob
from PIL import Image

import mxnet
from mxnet import nd
from mxnet.gluon.data import Dataset


class data(Dataset):
    def __init__(self, root_dir):    
        self.files = sorted(glob.glob(root_dir + '/*/*'))
        self.label = [nd.array(0) if x.split('/')[-1] == 'test' else nd.array(1) for x in self.files]

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        image = Image.open(self.files[idx])
        image = nd.array(image)
        label = self.label[idx]
        return image, label

dataset = data(data_root)
test_dataset = dataset
test_loader = mxnet.gluon.data.DataLoader(test_dataset, batch_size = 4)

for (image, label) in train_loader:
    print(image)
    print(label)

Error Message

mxnet.base.MXNetError: [11:21:44] src/imperative/./imperative_utils.h:146: Operator stack inferring shapes failed.
input shapes:
None
None
None
None
output shapes:
None
operator attributes:
num_args : 4

Stack trace:
  [bt] (0) /home/***/.virtualenvs/torch/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x6d554b) [0x7fed620f254b]
  [bt] (1) /home/***/.virtualenvs/torch/lib/python3.6/site-packages/mxnet/libmxnet.so(mxnet::imperative::SetShapeType(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, mxnet::DispatchMode*)+0x363b) [0x7fed6538de8b]
  [bt] (2) /home/***/.virtualenvs/torch/lib/python3.6/site-packages/mxnet/libmxnet.so(mxnet::Imperative::Invoke(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&)+0x1db) [0x7fed65395dcb]
  [bt] (3) /home/***/.virtualenvs/torch/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x3839f1f) [0x7fed65256f1f]
  [bt] (4) /home/***/.virtualenvs/torch/lib/python3.6/site-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x62) [0x7fed652574e2]
  [bt] (5) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fedacfc9dae]
  [bt] (6) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x22f) [0x7fedacfc971f]
  [bt] (7) /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2b4) [0x7fedad20a5c4]
  [bt] (8) /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x11c33) [0x7fedad20ac33]

Environment

mxnet-cu101mkl 1.6.0
ubuntu 18.04
cuda 10.1
python 3.6.9

[wkcn]

could you please print the image.shape and label.shape in __getitem__?

[wkcn]

The cause is that nd.stack doesn't support zero-shape array.
replacing self.label with self.label = [nd.array([0]) if x.split('/')[-1] == 'test' else nd.array([1]) for x in self.files] works.

[mikethreeacer]

Thanks a lot!