One: [onert] Renaming controlflow backend

[wateret]

Now that controlflow backend does more than just supporting control-flow operations, what about renaming it?

Here's what it does:

• Has kernels for If and While - enables control-flow operations
• Has a kernel for Permute operation
  
  
  
  • Permute operation is a special operation that runtime can generate(cannot exist in models) for copying tensor data between different backends
  
  
• Manages tensor objects of model inputs and outputs
• (Possibly manages tensor objects of If, While and Permute)

Name suggestions are welcome, or even one can say to keep the current name.

[wateret]

I would suggest util or builtin.

[s-barannikov]

I'm not very aware of the current status of the runtime. Could you tell the difference between the cpu and controlflow backends as of now? It the controlflow supposed to be a replacement for cpu? If not, why?

[s-barannikov]

Permute operation is a special operation <...> for copying tensor data <...>

What about renaming it to Copy? :)

[hyunsik-yoon]

+1 for builtin

[wateret]

@s-barannikov

Permute operation is a special operation <...> for copying tensor data <...>

What about renaming it to Copy? :)

Yeah, I also think Permute is not a very good name for it, let's think about it later. 😄

I'm not very aware of the current status of the runtime. Could you tell the difference between the cpu and controlflow backends as of now? It the controlflow supposed to be a replacement for cpu? If not, why?

Basically backends are dynamically loaded plugins, and they provide operation kernels and memory management for the tensors they manage. controlflow is a special backend that is part of runtime(not dynamically loaded) which supports special operations only - Permute, If and While. (If other backends implement these operations, it is no problem but it would never be used.) Plus, it does things I stated above.

cpu backend is a backend that we provide as a bundle which is not essential.

[s-barannikov]

@wateret Thank you for the explanation.
It is just something bugs me about this controlflow backend. It is special in so many ways that I'm not sure it can be called a "backend" at all.
This is just a thought, I don't suggest anything.

[wateret]

I see, but this backend works just like other backends. It has to manage ITensor objects and provide kernels for If, While and Permute.

And here's just few exceptions when it is specially treated:

• The runtime can change input/output tensor buffers (entry point of an execution)
• To make control flow ops have control of the execution flow, runtime passes Executor objects to controlflow::KernelGenerator

[s-barannikov]

@wateret

• The runtime can change input/output tensor buffers (entry point of an execution)
• To make control flow ops have control of the execution flow, runtime passes Executor objects to controlflow::KernelGenerator

So, the controlflow backend:

• prepares input tensors for the subgrah (If / While);
• launches Executor::execute for the selected subgraph;
• passes outputs of one subgraph to inputs of another (Permute).

It seems that its repsonsibility is managing the execution process - it chooses what to execute and prepares data for that.
Unlike ordinary backends, it also does not need its own TensorManager, because no tensors belong to it.
From that point of view, the controlflow backend is more "execution manager" rather than a backend. It is a level higher that other backends.

It's something to think about, just forget for the moment of the current implementation (KernelGenerator, IfLayer etc.) 😉
Suppose we have a set of subgraphs, and If / While / Permute operations connecting them (consider these operations "virtual", they _do not_ belong to any subgraph). This hypotetical "execution manager" knows how the subgraphs are connected and behaves accordingly. If, for instance, after executing some subgraph we've figured out the cond input of the If operation, and we also have all other input tensors for the If operation, this "execution manager" can write the input tensors and schedule this subgraph for execution.

---

Point is, if it is diffucult to choose a name for a class, then it probably means that the class has unclear (mixed) responsibility.
When the purpose of the class is clear, it is easy to choose a name by answering the question "what is it responsible for".

[wateret]

@s-barannikov Thank you for sharing your thoughts. 😃

Suppose we have a set of subgraphs, and If / While / Permute operations connecting them (consider these operations "virtual", they do not belong to any subgraph). This hypotetical "execution manager" knows how the subgraphs are connected and behaves accordingly. If, for instance, after executing some subgraph we've figured out the cond input of the If operation, and we also have all other input tensors for the If operation, this "execution manager" can write the input tensors and schedule this subgraph for execution.

At first I personally wanted to treat those operations specially, and an "execution manager" would process those operations. I guess it may be similar to what you've mentioned. Anyway just in case you would be interested, #154 is the context how we ended up with the current way.

Plus, I still think it is worth trying switching to that way.

[s-barannikov]

I guess it may be similar to what you've mentioned.

Thanks for the link, this sounds like exactly what I meant