Stride: Script and Task improvements Epic

[xen2]

Various ideas this Epic covers:

• Make our task system alloc-free (possible with recent C# version) so that people are not afraid to create thousands of scripts with await
• Reorganize task system so that we can wait various points of engine. So far we only have Update/Draw, but I suspect many scripts wants much more control when they want to run (after physics, etc...).
  Not sure yet if we want some kind of high-level sync point graph editor, or something more programmatic.

[velhaco20000]

Is there anyone working on this task ?

[velhaco20000]

You proposed 2 features:

• Minimize allocation in a task start, possibly with a minimum thread local storage;
• Create several points to start a script in order to achieve a precondition.

Is that right ?

[xen2]

Yes, that's two separate features (probably done together in the same feature branch).
I started some prototyping on this task: https://github.com/xen2/xenko/tree/microthread2
I have made a simple library to test part of the concepts, next step is to decide exactly how to integrate it into Xenko and switch various part of Xenko to use it.

[velhaco20000]

Do you want to create a mini batch system ?

[velhaco20000]

I will read the code and make a proposal.

[xen2]

FYI the code is still a prototype/playground, it is not yet integrated and lot of things will change.

[velhaco20000]

Just a little update, and correct me if I'm wrong :

• You wrote a mini batch system , with a scheduler and a synchronization system;
• Normally, users don't access directly such systems. There is a layer in the engine breaking great tasks into little taks and this layer and the little tasks weight depends on system configuration (I.E. core counts, core capabilities, etc) ;
• If those assumptions are correct, the next step is construct this layer to break great tasks in little tasks and create the synchronization points;
  -DX12 and Vulkan helps with this job;
• I don't know if you are thinking on transparency on this layer, permitting the user/programmer to control it beyond creating custom sync points;
  Let me hear your observations and corrections.

[velhaco20000]

Little Update 2: (correct me if I'm wrong)

• Xenko uses SDL as Window System, SharpDX as DX11/12 Binding, SharpVulkan as Vulkan Binding , OpenTK (Xamarin) as an OpenGL/OpenGL ES/Metal binding/layer ;

• Seems the engine was implemented in a way to be more compatible with older APIs ;

• One simple feature is implement async compute in order to use MicroThreads and Vulkan/DX12/Metal .Use MicroThreads to populate the queue and get the response. This will break compatibility in components;

• This imply in divide the engine implementation in high perfomance devices and low performance(mobile) devices .

Let me hear your observations and corrections.

[velhaco20000]

Little Update 3: ( corrections and observations are welcome)

• I will create an async compute language subset using ANTLR4 and generating the parser in C# (Any restrictions/directions for grammar?);
• The subset will produce METAL/Vulkan-OpenGL/HLSL compute shaders;
• The compute shader will be processed by MicroThread using the scheduler ;
• A use scenario will be created and the solution could be better evaluated.
  Any restriction to this course of action ?

[velhaco20000]

Little Update 4:

• There is a Metal API in Xamarin, so the compute queue could be enable for testing purpose;

[fdrobidoux]

@velhaco20000 I would suggest you make a draft pull request to a new branch based on master from a fork of this repo and talk about that over there. This is mostly to reduce spam on epics, and you'll be able to explain yourself better using the PR template.

[velhaco20000]

@fdrobidoux , you are right. I just don't want to create something useless or something somebody else is doing right now.

[HeadClot]

Hey @xen2 - got a question about this Epic what do you mean by a "high-level sync point graph editor"? Do you mean visual Scripting because that would be nice to get. :)

[Shpendicus]

Has anything happend anymore in regard of this topic since i see that the tasks in stride 4.0.01 are still .NET 4.0 tasks and nott valuetasks afaik.

[tebjan]

@Shpendicus did you check the changes in the linked WIP branch?

[Shpendicus]

@tebjan what is WIP branch? where can i check it out?

Do u mean this one? https://github.com/xen2/stride/tree/microthread2

but isnt this more or less depricated and replaced now officially by stride?