One: [onert] Move ruy context from cker to runtime

[periannath]

Related : #2929 #3225

Ruy context has a collection of utilities and it is needed to run kernels in ruy library.
Currently, cker has ruy context and it is defined as thread_local.

Issue

This implementation runs well on most cases but produces error when one runtime thread is re-created after destruction.

At the first run, the weight value for hybrid fully connected is cached by ruy library and weight value is freed.
In thread deconstruction, cached weight value in ruy context is freed.
In the second run on a re-created thread, the runtime doesn't have weight value and ruy context also doesn't have cached weight value.
ONE Runtime will produce segfault for this case.

Todo

To solve this problem, I'll move ruy context from cker to runtime.
Each session will have its ruy context and the above problem will be solved.

/cc @chunseoklee @YongseopKim @dhdh-oh

[periannath]

How to implement?

• Introduce ExternalContext

• Backend has ExternalContext

  Option 1

• Classes from Backend to cker::FullyConnectedHybrid has ExternalContext

  • Backend, KernelGenerator, FullyConnectedLayer gets ExternalContext during its initialization
  • Pass ExternalContext at kernel run
• Pros
  
  
  
  • Easy to implement
  
  

• Cons

  • Some classes has unnecessary ExternalContext as its class member
    

    Option 2

• Pass ExternalContext as argument

  • LinearExecutor -> FunctionSequence -> FullyConnectedLayer
• Pros
  
  
  
  • ExternalContext is maintained only at Backend
  
  
• Cons
  
  
  
  • Ruy dependency in core
  
  
  • Many layers are affected
  
  
  • XXLayer::run() -> XXLayer::run(ExternalContext *)
  
  

[YongseopKim]

This will touch runtime's execution. cc/ @Samsung/nnfw

[periannath]

I realized several things via offline discussion with @ragmani.

Option 2 makes onert_core depends on ruy library since onert_core has LinearExecutor.
Each subgraph has Backend and BackendContext and models with control flow can create multiple subgraphs. In this case, multiple ruy context will be created.

Managing ExternalContext inside BackendContext would be an option.

[ragmani]

And @periannath talked about multiple ruy contexts. A flaw of multiple ruy contexts is that they create multiple thread pools unnecessary. Multiple thread pools create many alive threads at the same time.

[periannath]

I'll move ExternalContext to BackendContext and do some mult-thread testing on nnpackage_run. After testing is done, I'll upload PR.

[periannath]

I just found that BackendContext is not removed by nnfw_close_session. Cached weight allocated inside ruy context is leaked. I'll fix this error.

• This is fixed by adding virtual keyword to destructor of BackendContext.

[periannath]

Done.