Onnxruntime: Segmentation fault when running a session as attribute of a class

[walbermr]

Describe the bug
There is a strange behavior when handling a section: if a section is created and saved as a variable (inside a class, for example) to be used further, calling Session.Run results in segmentation fault. When you call Session.Run inside the function you create it, the call that used to result in segmentation fault works fine.

I am annexing an example in which this can be reproduced, when commenting line 40 you get the segmentation fault (forward call inside PassOnnx constructor). I am also using cmake to compile.

System information

• OS Platform and Distribution: Linux Ubuntu 18.04
• ONNX Runtime installed from (source or binary): Binary
• ONNX Runtime version: 1.4

To Reproduce

#include <vector>
#include <string>
#include <iostream>

#include <assert.h>
#include <onnxruntime_cxx_api.h>

class PassOnnx
{
private:
    /* data */
    Ort::Session *session;
    size_t num_input_nodes;
    std::vector<int64_t> input_node_dims;
    Ort::AllocatorWithDefaultOptions allocator;
    std::vector<char *> input_node_names;
public:
    PassOnnx(const char* model_path, int numberofThreads=1)
    {
        // Set environment for our session
        Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "test");
        Ort::SessionOptions session_options;
        session_options.SetIntraOpNumThreads(1); //number of threads should be a define, we may experience issues to change that in the future
        session_options.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_EXTENDED);
        // Load session
        session = new Ort::Session(env, model_path, session_options);
        // Get the properties of our model
        this->num_input_nodes = session->GetInputCount();
        this->input_node_names = std::vector<char *>(this->num_input_nodes);
        for (int i = 0; i < this->num_input_nodes; i++)
        {
            char *input_name = session->GetInputName(i, allocator);
            this->input_node_names[i] = input_name;
            Ort::TypeInfo type_info = session->GetInputTypeInfo(i);
            auto tensor_info = type_info.GetTensorTypeAndShapeInfo();
            ONNXTensorElementDataType type = tensor_info.GetElementType();
            input_node_dims = tensor_info.GetShape();
        }
        float dumbInput[13] = {1,2,3,4,5,6,7,8,9,10,11,12,13};
        std::cout << this->forward(std::vector<float>(dumbInput, dumbInput + 13)) << std::endl;
    }
    double forward(std::vector<float> frame)
    {
        // Init the input tensor
        std::cout << "pass 1" << std::endl;
        size_t input_tensor_size = frame.size();
        auto memory_info = Ort::MemoryInfo::CreateCpu(OrtArenaAllocator, OrtMemTypeDefault);
        Ort::Value input_tensor = Ort::Value::CreateTensor<float>(memory_info, frame.data(), input_tensor_size, this->input_node_dims.data(), 2);
        assert(input_tensor.IsTensor());
        std::cout << "pass 2" << std::endl;
        // Get our model output
        std::vector<const char *> output_node_names;
        for(int i = 0; i < this->session->GetOutputCount(); i++)
        {
            output_node_names.push_back(this->session->GetOutputName(i, allocator));
        }
        std::cout << "pass 3" << std::endl;
        auto output_tensors = this->session->Run(Ort::RunOptions().UnsetTerminate(), this->input_node_names.data(), &input_tensor, 1, output_node_names.data(), 1);
        std::cout << "pass 4" << std::endl;
        assert(output_tensors.size() == 1 && output_tensors.front().IsTensor());
        double *arr = output_tensors.front().GetTensorMutableData<double>();
        double res = arr[0]; //get output
        return res;
    }
};

int main(int argc, char const *argv[])
{
    float dumbInput[13] = {1,2,3,4,5,6,7,8,9,10,11,12,14};
    PassOnnx network("pass_model.onnx");
    std::cout << network.forward(std::vector<float>(dumbInput, dumbInput + 13)) << std::endl;
    return 0;
}

• ONNX MODEL:

https://drive.google.com/file/d/1QGJ6_rY_MZyWIv_i30U6KnXMjENxj6gA/view?usp=sharing

Expected behavior
I suppose that the expected behavior is not crashing when calling Session.Run for the first time after creating a session

[hariharans29]

I quickly tried this on Windows 10 (not Ubuntu 18.04) to see if there was something obviously problematic. The program seemed to run fine -

This is with the forward call commented inside the constructor. With that uncommented, I get two such result blocks (one for the call from the constructor and one from the call from main).

Are you sure the program finds the model ?

I can try on Ubuntu.

[hariharans29]

I could repro the segfault on Ubuntu, but the problem is with the app code:

Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "test"); - The Ort::Env instance is scoped to just the constructor and it is destroyed when the constructor is exited. The Ort::Env instance MUST BE VALID whenever invoking an ORT API and with the current setup, it may not be valid which manifests as the segfault.

I made some a minor adjustment to your code by adding another member to your class: Ort::Env* env and I initialized it with new Ort::Env(..., ...) in the ctor and a minor change in this line: session = new Ort::Session(*env, model_path, session_options);, the app compiled and ran fine without a segfault with line 40 commented on Ubuntu.

Please re-open if you have further questions.

[walbermr]

Thanks for the help, worked fine in the example and on the application! I was using the sample C++ API code provided in the documentation and due to it, I was facing slowdowns during multiple inferences as result of the re-creating multiple times Ort::Env and Ort::Session. Changing that to this discussed method improved significantly the execution time. Thank you very much for the help!