Glow: Unsupported Cast Operator in ONNXModelLoader::loadOperator

[sri-oc]

Trying to use glow to emit bundle for image classifier using resnet50 model, which is converted from tf to onnx.

Steps followed for conversion:

1. Freeze the graph - sample.pb
2. Convert tf graph to onnx using tf2onnx.convert

When I used the converted model with the glow to generated a bundle using the following command
./bin/image-classifier tests/images/imagenet/cat_285.png -image-mode=0to1 -m=resnet50/tf2onnx.onnx -model-input-name=gpu_0/data_0 -cpu -emit-bundle /tmp -dump-graph-DAG=/tmp/resnet50.dot -network-name=static_version -relocation-model=pic

I'm getting an error saying,
Encountered an error, exiting.
file: /root/srinivas/Glow_Repo/Glow_Code/glow/lib/Importer/ONNXModelLoader.cpp line: 896 error code: MODEL_LOADER_UNSUPPORTED_OPERATOR message: Failed to load operator.

When I tried to debug the error, it is cast operator is causing an issue.

[jfix71]

Hi @srinivas491-oneconvergence, can you provide us with the proto containing the operator that isn't supported? Or more details on the cast that is causing the problem?

[nickgg]

As an aside, do you need to convert from TF? Can you use the resnet50 model from the Caffe2 Model Zoo?

[sri-oc]

@nickgg, I'm trying to use the glow for the TF models by converting them to ONNX instead of going through Caffe2.

[tlepley-cadence]

We've got the same problem too. There is a discussion ongoing with tf2onnx people (https://github.com/onnx/tensorflow-onnx/issues/322) because in many situations, Cast could be optimized out by tf2onnx. But since Cast is anyway a valid ONNX operator, it's IMO relevant to support it in Glow. We have already added a partial support for the ONNX Cast in our private Glow repo. I am planning to upstream it in a near future.

[sri-oc]

@jfix71
Just an overview of the cast operation in the graph.

[sri-oc]

@tlepley-cadence

By any chance, is there a way for you share your private repo which has partial support for Cast operator? Since I need this operator support to continue my training and inference of tf models.

It would be great if you share that.

[tlepley-cadence]

@srinivas491-oneconvergence I can't share the private repo, but I've just created a WIP PR here: #2655

I still need to write few tests before this PR can land, but at least, you can make a try with the branch of this PR. I'll certainly find the time to complete the PR with tests tomorrow.

[tlepley-cadence]

@srinivas491-oneconvergence in your graph, it's not clear to me what the Cast operation takes as input. It is supposed to have an input tensor, and it's not clear on your graph. Also, there is no type information (what is the source and destination type of the cast ?).

Edit: I see on the right that this Cast converts float to int. Looking at the constant tensor value, it looks strange to me. because you'll get value 0 by casting your tensor to int. Is it really a Cast node that you want ? Isn't it more a Quantize node ? In my use case, the cast is used to convert a int32 scalar to int64. This cast is introduced due to a difference of type between TensorFlow and ONNX.