Onnxruntime: [Loading Model] Matlab Exported LSTM Model

[prometheusDE]

We encountered an issue with loading a onnx model generated in a different learning framework - Matlab Deep Neural Network Toolbox.

Basically we followed Matlab's tutorial on sequence-to-sequence regression models using LSTM Networks Link

We exported the model using the plugin ONNX exporter Link

Now we tried to load the model using onnxruntime build from source release version 0.4.0

We get the following error:

Load model from Turbfoan_Engine_Degradation.lstmmodel.onx failed:Node:fc_2 Output:fc_2 [ShapeInferenceError] Mismatch between number of source and target dimensions. Source=2 Target=3

The onnx file (zipped): Turbfoan_Engine_Degradation.lstmmodel.zip

The model file loads just fine in Netron Link

Any help would be appreciated.

[raymondxyang]

I am looking into it..

[prometheusDE]

Thanks a lot! :D

[raymondxyang]

I traced the shape inference of the model:

1. the output of lstm node is 4-dimensions
2. the output of first gemm node (fc-1) is 2-dimensions
3. the output of first gemm node (fc-2) is 2-dimensions, which is different with the defined [1,1,1] shape.
   From my limited understanding and the definitions on https://github.com/onnx/onnx/blob/master/docs/Operators.md#Gemm , those inferred results looks good to me. Probably we can include the exporter team for the discussion.

[tsu12]

We noticed that some LSTM models exported by MATLAB ONNX Converter don't work well with ONNX Runtime, although they could be loaded into other frameworks, as ONNX Runtime strictly follows ONNX spec for the shape requirement. A new release of MATLAB ONNX converter will be released soon and it will work with ONNX Runtime better.

[prometheusDE]

Thanks a lot @raymondxyang and @tsu12

@tsu12 Is there another C/C++ Framework that can load and score onnx models?

[tsu12]

@prometheusDE, can you try to load this model to CNTK?

[cuixing158]

@tsu12 @raymondxyang Does the current matlab onnx version support the export of target detection networks(use exportONNXNetwork function), such as the Yolov2 network（export to yolov2.onnx）? It seems that only support 'classification' ,'regression','pixelclassification' layer type network...

[prometheusDE]

@tsu12

I tried:

pip install cntk

in python:

import cntkz 

z = cntk.Function.load("Turbfoan_Engine_Degradation.lstmmodel.onnx", device=cntk.device.cpu(), format=cntk.ModelFormat.ONNX)

I get the following error message (basically the same as in the onnxruntime).

About to throw exception 'Failed to load model: 'Node:fc_2 Output:fc_2 [ShapeInferenceError] Mismatch between number of source and target dimensions. Source=2 Target=3''
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\Python36\lib\site-packages\cntk\internal\swig_helper.py", line 69, in wrapper
    result = f(*args, **kwds)
  File "C:\Python\Python36\lib\site-packages\cntk\ops\functions.py", line 1635, in load
    return cntk_py.Function.load(str(model), device, format.value)
RuntimeError: Failed to load model: 'Node:fc_2 Output:fc_2 [ShapeInferenceError] Mismatch between number of source and target dimensions. Source=2 Target=3'

[CALL STACK]
    > CNTK::TrainingParameterSchedule<double>::  GetMinibatchSize
    - CNTK::  XavierInitializer
    - CNTK::Function::  Load
    - PyInit__cntk_py (x2)
    - PyCFunction_FastCallDict
    - PyObject_GenericGetAttr
    - PyEval_EvalFrameDefault
    - PyErr_Occurred
    - PyTuple_Size
    - PyEval_EvalFrameDefault
    - PyErr_Occurred
    - PyObject_GenericGetAttr
    - PyEval_EvalFrameDefault
    - PyErr_Occurred
    - PyEval_EvalCodeEx

[raymondxyang]

@cuixing158 Probably @tsu12 can better answer this problem.

Meanwhile, I would like to close the problem since it is followed up by a partner organization. But feel free to keep the discussion ongoing if necessary.

[prasanthpul]

Being discussed here: https://www.mathworks.com/matlabcentral/fileexchange/67296-deep-learning-toolbox-converter-for-onnx-model-format