Stride: [Asset] Slightly complex prefabs and scenes are effectively unusable

[Eideren]

Release Type: Github
Version: Latest
Platform(s): Windows editor
Describe the bug
TL;DR: slightly complex scenes and prefabs are pretty much unusable because of the time the editor takes to save changes and add them to scenes.

• Details
  Modifying scenes and prefabs, drag&drop and other operations with those features takes an absurdly large amount of time and often ends in crashes in more complex scenarios, just dropping a prefab containing a couple of children in a scene can take more than a minute and will crash the game studio once the amount of children increases further.

• Profiling
  When profiling a drag and drop we can see that a lot of time is taken by reflection and fighting the GC with allocations but both of those doesn't take most of the time anyway (around 20%), the rest is spent all over the place in quantum so this is most likely a design issue.
  some of the functions where called more than 4.000.000 times and it was barely a fourth of the way to completion.

• Question
  All of that work is to setup binding instance to the source prefab's for changes to cascade over, right ? I don't really understand why this needs to happen for a drag and drop operation, shouldn't this happen once a modification to the source has been made ?

• Potential Solution
  Loading from disk works fairly well even though that would require accessing files and parsing them which seems like it should take longer but clearly doesn't here, perhaps we should only write the changes to disk on save and propose to users to reload their opened scene if they contain changed prefabs ?

To Reproduce
Steps to reproduce the behavior:

1. Extract this zip: MyGame7.zip
2. Load the solution in game studio
3. Drag and drop MediumPrefab in the scene view/hierarchy
4. Gamestudio will now block for the next couple of minutes or might just crash before successfully adding that prefab to the scene.

Additional context
I initially started investigating this because I had a prefab made out of around 2000 fairly simple items, the scene editor opened that prefab in a couple of second at most but drag and dropping it inside another scene led to a seemingly never ending load. I understand that 2000 items is perhaps too large for one prefab to hold but I don't think we should expect such an operation to either block for more than 20 minutes and/or crash afterward.

---

If you don't have time to @xen2 I can work on this but I only have a couple of days to do so and I need some pointers:

1. How is this whole system architectured ?
2. Where is it used ?
3. Why do we need all of this quantum reflection stuff when we actually generate serializers for binary serialization ?
4. Is what I proposed above in Potential Solution feasible

[Kryptos-FR]

It is possible some steps are required in order of copy/paste and drag&drop to work between two instance of the GameStudio (like between two unrelated projects).

I don't remember the details or whether it was actually implemented to work in that case, but I do remember having some discussions about it and at least attempted an implementation.

[xen2]

@Eideren Sorry for the late answer (I was off last 3 days) -- it might be too late now.

Sadly I am not too familiar with the exact detail/status of Quantum as I didn't work on it directly except at the very beginning.

Here's some thoughts/idea:

• There's probably something in O(n^2) or worse, and I don't think it should be the case for prefabs. This is probably something that was not detected or impacted us at the time. so it was not investigated/fixed.
  If this is the case, it might be possible that it's simple to fix it once we find where/why this bad complexity happens (or why it's called so many times) -- but not sure it's the case and it might be difficult to find it.
• Quantum is not about serialization but to create a kind of "view model"-compatible objects from existing asset or runtime objects (i.e. TextureAsset orModelComponent`). The idea is that they can then easily work with undo/redo, change notifications, prefab changes, etc. (and possibly more advanced cases such as sync objects asynchronously between different threads/processes).
  The biggest downsides are increased allocations, lack of IntelliSense and extra indirection/complexity when using it.
• I remember there was some discussion about providing alternatives modes for updating prefabs (i.e. on demand on button click in inheriting asset rather than all the time automatically). That would likely be close to your proposed potential solution.
  Note: https://github.com/stride3d/stride/issues/178 is loosely related

[xen2]

As a follow-up to the third point, I also remember Quantum was designed to cope with live-reconciliation disabled and possibly done in a second step since we store all the necessary information in Yaml (i.e. a deleted item is not simply removed from the list but marked with a special ~(Deleted) node).
We discussed implementing this mode (where prefab is synchronized on user request and/or only for open assets, not sure of the exact details) but unfortunately we didn't get around to it.