Onnxruntime: python API to get intermediate layer output

[vera121]

I found it inconvenient that onnxruntime can't display the intermediate results during model inference.
I checked the similar issue #1455, but found it couldn't solve the question for model such as yolov3 from ONNX model zoo.

I tried the code mentioned at https://github.com/microsoft/onnxruntime/issues/1455#issuecomment-514805365 and also this:

model = onnx.load(model_path)
for w in model.graph.value_info:
    model.graph.output.extend([w])
print(model.graph.output)

But the results show still the original 3 outputs. The intermediate layers can't be added and displayed.

Can you add some APIs to show these results?
It is not easy to analyze some large models only with the last outputs.

System information

• ONNX Runtime version: 0.5.0

[hariharans29]

You need to know the name of the intermediate tensor from the model and you need to put the exact name here intermediate_tensor_name and re-load the saved model in OnnxRuntime. Unless the required intermediate output is part of the graph outputs, currently OnnxRuntime, does not support fetching the results.

model = onnx.load(model_path)
intermediate_layer_value_info = helper.ValueInfoProto()
intermediate_layer_value_info.name = intermediate_tensor_name
model.graph.output.append(intermediate_layer_value_info)
onnx.save(model, model_path)

[vera121]

Thanks @hariharans29 . I found it work when I change the code to:

model = onnx.load(model_path)
intermediate_tensor_name = "convolution_output74"
intermediate_layer_value_info = helper.ValueInfoProto()
intermediate_layer_value_info.name = intermediate_tensor_name
model.graph.output.extend([intermediate_layer_value_info])
onnx.save(model, model_path)

And then reload the modified model.

[ranc]

The solution does not work anymore, there is a type check for the dtype of the output tensor...
ORT should have a proper API to fetch it