Onnxruntime: Session::Run() function doesn't support multiple outputs

[elevir]

Describe the bug
When I'm trying to infer two outputs of model at the same time an exception of access violation occurs.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 1809
• ONNX Runtime installed from (source or binary): source
• ONNX Runtime version: 0.4.0

To Reproduce
Code is same as example, the difference is that a model has two outputs and two output node names passed into Run method.

Expected behavior
In the screenshot below output[0] is nullptr, but 'output[1]' is 0xcccccccccccccccc which is not equals to nullptr. If such using of inference is not supposed, then there must be an exception, otherwise it is bug and output must be checked for nullptr value before indexing.

This is code snippet of method
ORT_API_STATUS_IMPL(OrtRun, _In_ OrtSession* sess, _In_ OrtRunOptions* run_options, _In_ const char* const* input_names, _In_ const OrtValue* const* input, size_t input_len, _In_ const char* const* output_names1, size_t output_names_len, _Out_ OrtValue** output)
from onnxruntime_c_api.cc

[snnn]

Show me your code please.
For example,

  Ort::Value output_tensor = session.Run(nullptr, input_node_names.data(), input_tensors, output_node_names.data(), 1);

Did you change the last parameter from 1 to 2?

[elevir]

@snnn the code of Run looks like

Ort::Value output_tensor = session.Run(nullptr, input_node_names.data(), input_tensors, output_node_names.data(), output_node_names.size());

The size of output_node_names equals to 2.

[linkerzhang]

is it ok for you to share the model with us please?

[elevir]

@snnn @linkerzhang there is full code of model using.

#define CUDA
//#define TENSORRT

#include <iostream>
#include <onnxruntime/core/session/onnxruntime_cxx_api.h>

#ifdef CUDA
#include <onnxruntime/core/providers/cuda/cuda_provider_factory.h>
#elif defined TENSORRT
#include <onnxruntime/core/providers/tensorrt/tensorrt_provider_factory.h>
#endif


void main()
{
    Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "env");
    Ort::SessionOptions sessOptions;

#ifdef CUDA
    OrtSessionOptionsAppendExecutionProvider_CUDA(sessOptions, 0);
#elif defined TENSORRT
    OrtSessionOptionsAppendExecutionProvider_Tensorrt(sessOptions);
#endif

    sessOptions.SetGraphOptimizationLevel(0);

    const wchar_t* modelPath = L"super_point.onnx";
    Ort::Session sess(env, modelPath, sessOptions);
    auto allocator = Ort::Allocator::Create_Default();
    // print number of model input nodes
    std::vector<const char*> inputNodeNames(1);
    std::vector<int64_t> inputNodeDims;
    char* inputName = sess.GetInputName(0, allocator);
    inputNodeNames[0] = inputName;
    Ort::TypeInfo typeInfo = sess.GetInputTypeInfo(0);
    auto tensorInfo = typeInfo.GetTensorTypeAndShapeInfo();
    auto tensorShape = tensorInfo.GetShape();

    size_t dataSize = OrtGetTensorShapeElementCount(tensorInfo);
    std::vector<float> inputData(dataSize, 0);
    auto allocInfo = Ort::AllocatorInfo::Create_Cpu(OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
    Ort::Value inputTensor[1] = { 
        Ort::Value::CreateTensor(
            allocInfo,
            inputData.data(), dataSize * sizeof(float), 
            tensorShape.data(), tensorShape.size(), 
            ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT) 
    };


    size_t outputsCount = sess.GetOutputCount();
    std::vector<const char*> outputNodeNames(outputsCount);
    for (size_t i = 0; i < outputsCount; ++i) {
        char* outputNodeName = sess.GetOutputName(i, allocator);
        outputNodeNames[i] = outputNodeName;
    }
    //std::vector<const char*> outputNodeNames = { sess.GetOutputName(1, allocator) };

    Ort::Value output = sess.Run(nullptr, inputNodeNames.data(), inputTensor, outputNodeNames.data(), outputNodeNames.size());
        for (const auto& dim : output.GetTensorTypeAndShapeInfo().GetShape())
        std::cout << dim << " ";
    std::cout << std::endl;
}

And there is the model: super_point.zip

[snnn]

I hit an error of:
/data/src/onnxruntime/onnxruntime/core/framework/execution_frame.cc:64
onnxruntime::common::Status onnxruntime::IExecutionFrame::GetOrCreateNodeOutputMLValue(int, const onnxruntime::TensorShape *, onnxruntime::MLValue *&) shape && tensor.Shape() == *shape was false. MLValue shape verification failed. Current shape:{2,1,480,640} Requested shape:{2,256,60,80}

[elevir]

@snnn is it thrown before running inference? Because I can succesfully infer model with one node (including output node with shape { 2, 256, 60, 80 })

[snnn]

No. It's inside sess.Run.

[elevir]

@snnn strange. I have built Ort with Debug config and program still fails on access violation without errors related on Shape.
Just in case, I've updated message above with actual code.

[xgirones]

I think I ran into a similar problem. I solved it by supplying an array of NULL initialized pointers to OrtValue, one for each expected output:

`std::vector vpOutputTensor(outputNodeNames.size() );

pSt = OrtRun( apS.get(), NULL, inputNodeNames.data(), (const OrtValue* const*)&pInputTensor, 1, outputNodeNames.data(), outputNodeNames.size(), vpOutputTensor.data() );`

[snnn]

It's a bug in the cxx wrapper.