Coremltools: Incorrect input shape

[andriikrupka]

I have keras model like that:

inputlayer = Input(shape=(126,12))

model = BatchNormalization()(inputlayer)
model = Conv1D(16, 25, activation='relu')(model)
model = Flatten()(model)
model = Dense(output_size, activation='sigmoid')(model)

model = Model(inputs=inputlayer, outputs=model)

Which I convert to coreml:

coreml_model = coremltools.converters.keras.convert(model,
                                                    class_labels=classes)
coreml_model.save('speech_model.mlmodel')

So, I expect to see MultiArray (Double 126x12), but I see MultiArray (Double 12)

Could you help to say what I'm doing wrong?

[G-mel]

since your input dimension length is 2, according to https://github.com/apple/coremltools/blob/master/coremltools/converters/keras/_keras2_converter.py#L227, it assumes your input shape is [Seq, D], and therefore returns (dim[1],) where dim would be (126,12) in your case.

To get around this, you could try:

inputlayer = Input(shape=(126 * 12,))

model = Reshape((126,12))(inputlayer)
model = BatchNormalization()(model)
model = Conv1D(16, 25, activation='relu')(model)
model = Flatten()(model)
model = Dense(output_size, activation='sigmoid')(model)

model = Model(inputs=inputlayer, outputs=model)

where in your coreML app, input a flattened input thats of shape (126 * 12,), and the mlmodel will handle the reshaping to (126, 12) within the model.

[srikris]

Looks like this was fixed.

[brodney]

This seems to be happening still. I worked around the problem with the Reshape layer suggested above, but I get errors printed to the console for every prediction call at runtime.

BNNS FULLY: input batch stride doesn't make sense (32 < 32576)

For context, my first layer without the workaround is a Conv1D with input shape (4096,1).

[vellamike]

@srikris @slin07 this is definitely not fixed. I tried with the latest (master) branch and the problem appears to still be occurring. The workaround suggested by @G-mel does work but I think that the reshape operation is unnecessarily expensive on GPU.

Could this issue be reopened?

[aseemw]

@andriikrupka (126,12) is interpreted as (Seq, Channel) by CoreML. CoreML treats Seq. dimension similar to a batch dimension hence it does not hard code it into the mlmodel. So the mlmodel for the input just says 12 corresponding to the number of channels. This way you can change the value of the sequence dimension at run time when you call predict in your app. At that time CoreML can be given a multi array of 3 dimensions, which maps to (Seq,B,C). You just need to create an array of shape (126,1,12) and pass it to the predict call. In fact you can also call predict on an input which has a different seq dimension (say 100 instead of 126) by passing in (100,1,12) sized input. You do not need to change the mlmodel by adding a reshape layer.

Please see discussion in issue #142 for more details.

If in you case you only work on 126 length sequences, you can do a 2D conv such that the height dimension is 1 and the width dimension corresponds to the sequence dimension. Something like this:

inputlayer = Input(shape=(1,126,12)) #Keras interpretation: (H,W,C)
model = BatchNormalization()(inputlayer)
model = Conv2D(16, (1,25), activation='relu')(model)
model = Flatten()(model)
model = Dense(10, activation='sigmoid')(model)
model = Model(inputs=inputlayer, outputs=model)
print(model.summary())

[vellamike]

@aseemw I tried passing an input of size (100,1,12,1,1) to @andriikrupka's model and the result was an error complaining that rank-5 inputs were not supported.

inputlayer = Input(shape=(126,12))
output_size=30
model = BatchNormalization()(inputlayer)
model = Conv1D(16, 25, activation='relu')(model)
model = Flatten()(model)
model = Dense(output_size, activation='sigmoid')(model)

model = Model(inputs=inputlayer, outputs=model)
coreml_model = coremltools.converters.keras.convert(model)

X = np.random.rand(126,1,12,1,1)
coreml_model.predict({'input1':X})

RuntimeError                              Traceback (most recent call last)
<ipython-input-62-692012ab6531> in <module>()
----> 1 coreml_model.predict({'input1':X})

~/dev/coreml_experiments/.venv/lib/python3.6/site-packages/coremltools/models/model.py in predict(self, data, useCPUOnly, **kwargs)
    262 
    263         if self.__proxy__:
--> 264             return self.__proxy__.predict(data,useCPUOnly)
    265         else:
    266             if _macos_version() < (10, 13):

RuntimeError: {
    NSLocalizedDescription = "Input input1 is an array of rank 5, but this model only supports single vector inputs (rank 1) or a sequence of batches of vectors (rank 3).";
}

[aseemw]

@vellamike I erred in my description of the input shapes. Please see comments in issue #142.