Entt: Towards EnTT v2.5/v3

[skypjack]

Road to EnTT v2.5/v3 (it depends on the types of changes).
Here is a list of the new features and major changes planned for the next version.

• Performance improvements for multi component standard views
• Serialization/Deserialization
• Spaces (not sure about this actually)
• ...

Changes are merged on master step by step. They could also be breaking ones in terms of API.
Please, keep on using the latest tag if you want to switch to the new version at once.

[ArnCarveris]

@skypjack Add some workload inside benchmark lambdas

[skypjack]

@ArnCarveris

Yep, benchmarks would be more meaningful actually. The ones I'm using are the same of EntityX so as to be able to compare the two projects. But they are poorly written, I agree.
Now that iterations are highly optimized by the compiler (see the README on the space branch for more details), those benchmarks don't make sense anymore actually.

I'm putting this in the TODO list. Good point.

[ArnCarveris]

@skypjack Any chance to have view/each with optional entity range? can be useful for iterating grouped entities (like parents children).

[skypjack]

@ArnCarveris What's the use case? I mean, as an example, if I'm iterating transforms where I've parent-child relationships, I've already a view to use to get the components for the children of an entity. Something like this:

view.each([&view](auto entity, auto &ptransform) {
    for(auto child: transform.children) {
        auto &ctransform = view.get(child);
        // ... Logic ...
    }
});

What's the use case for which this isn't enough?

[ArnCarveris]

@skypjack transform.children is std::vector or any other container?

[skypjack]

I don't know, it was an example for the purpose of the question, I would say yes. Probably a fixed size array or something like this.
Honestly I usually do it the other way around using component.parent, so rarely going from parents to children.
Anyway, set aside it, what's the use case for a range-each that we cannot solve using views/registry already?

[DomRe]

What about a way to store 'systems'. Since most of the time users would operate on entities / views in a system.

Maybe a sparse set that can hold Systems so user's don't have to rewrite system management code for every project using entt.

And whatever structure holds the systems can call methods like event() or update() for you for all systems.

So for example in a game:

void runGame(EventObject ev, DeltaTime dt)
{
    entt::DefaultRegistry er;
    entt::SystemRegistry sr;

    sr.event(ev); // Pass event object to all systems.

    // later
    sr.update(er, deltaTime); // Update all systems
}

Functions would probably look like:

void SystemRegistry::event(const EventType& ev);
void SystemRegistry::update(const entt::Registry<entt::entity_type>& registry, TimeType deltaTime);

Which would mean user systems would derive from a type like:

class System
{
public:
    virtual void event(const EventType& ev) = 0;
    virtual void update(const entt::Registry<entt::entity_type>& registry, TimeType deltaTime) = 0;
};

And EventType / TimeType could be provided by the user from either templates or specified in macros, or a typedef such as:

typedef double TimeType;

[AndrzejKozik]

About parent/children reletionship between entities - I prefer to use parentID in child along with its depth (int), eg. parent.depth + 1; then I'm using view sorted on depths to ensure parents are evaluated before their children.

I like the idea of maneged bus with its registration of whole systems (all events processing methods in one go), but on the same time it makes trouble in establishing proper ordering of systems responses to an event. If I register systems (objects) S_1, S_2, ..., S_n to the bus, there is a fixed ordering of their invocation in response of any event (consider pre-PhysX_Update and post-PhysX_Update). Then, I've trouble changing that order in a simple way (without introducing fake event types).

Maybe some way of registration of individual member functions could be done?

[skypjack]

@reworks

I've this bullet in the TODO list:

to analyze, long term feature: systems organizer based on dependency graphs for implicit parallelism

You are proposing to make an intermediate step to register systems without organizing them, right?
It could make sense actually. However, note that usually I provide systems with the event bus through a locator, so here we already have two different approaches to the problem. The way you do it is more (let me say) _entityx-oriented_, if I remember correctly how it works.

@AndrzejKozik

About parent/children reletionship between entities - I prefer to use parentID in child along with its depth (int), eg. parent.depth + 1; then I'm using view sorted on depths to ensure parents are evaluated before their children.

Yeah, more or less what I did in my last work, children point to the parent and the whole thing is ordered on a per-parent basis.

Maybe some way of registration of individual member functions could be done?

Can you elaborate a bit more on this?
Note also that it sounds really strange to me that the order of registration to the bus can influence your runtime.

[AndrzejKozik]

My issue is connected with @reworks approach - instead of manually call systems, make an engine publish an 'update' event to all the systems. The question is how to force ordering of systems calls individually for each event type.

The order can be either implicit, given by order of registration (it depends on containers under the hood) or given by dependency graph You mentioned, but in this case it will order whole systems, for all event types.

One solution would be possibility to override 'global' graph for some events, but it will incure performance penalty. On the other hand, assuming there is some implicit ordering of listeners, one way to go is not registering whole systems, but individual methods as listeners for certain event-types.

Alternativelly, some method to alter ordering of listeners of certain event-type (based on list of systems pointers) would be great.

I know that both solutions are rather error-prone and low-level, but such functionality is needed for advanced use cases.

[skypjack]

@AndrzejKozik Oh, I got the comment from @reworks a bit differently.
I guess he meant to register systems in a specific order, then invoke them exactly in the same order and not through events. Instead, if I got it right, he wants to provide systems with a dispatcher to use to send events if required.

[AndrzejKozik]

Oh, I see a part of the idea. If we can have a few registries, each with different order of systems, then it'll do the job. Registry updates have to be manually scattered in an engine - I prefer events from engine.

I don't get however the part of placing registry in the role of bus, with event dispatcher for each registry. Instead of just publish event to the bus, You have to publish it to each registry separately?

[skypjack]

@AndrzejKozik Probably it's worth waiting for more details from @reworks :-)
It looks like we didn't get his idea correctly and we are just trying to figure out what it was...

[DomRe]

Oh. I meant registering systems in a specific order in an ordered data structure so that when event() and update() are called it calls the appropriate system functions in that order. Basically just a container holding orderered pointers / objects / references to systems.

The event thing was passing an object containing events to the systems to process. For example, a movesystem might want to know if the player moved, so it would check the event object for a PLAYER_MOVED event or something.

[AndrzejKozik]

@reworks, have You considered single registry or multiple registries, each with different order of systems?

In the second part, is Your idea to have separate Bus functionality inside each system registry?

[skypjack]

I'm taking in consideration the idea of creating a sort of _extended registry_ that offers built-in support for (let me say) _runtime features_ like blueprint, spaces (I don't like much the current implementation actually) and so on.
In other terms, a way to support all those features that want to know when entities and components are created or destroyed.
Currently, only underlying data structures for persistent views observe the pools of components somehow.

What about?

My main concern is if it's worth extending the registry with a derived class or if these features are of common use and thus they should be part of the current API of a registry.

Any feedback is appreciated.

[gale93]

I upvote for a class System where you can derive from and you can:

• register and "receive()" any event you want
• implement a generic update() method

and the you add it to the SystemRegistry..

Imo, give an order to the Systems update() it's not a good move, unless you make it somehow filterable.
Because you may have Systems (like in my case) that want to be updated in different phases of the loop, like on normal update, on fixed update or on rendering..

So in my mind the only possible way would be somewhat like

enum {
    UPDATE,
    FIXED_UPDATE,
    RENDER
    ...
}; 

SystemRegistry::update(dt, FIXED_UPDATE);

[skypjack]

@gale93 If you had a way to register systems somehow within a runner, using multiple runners wouldn't solve your problem?

[gale93]

@skypjack oh well, as long as the systems continue to receive events from all runners and not only from the runner they are linked to, yes ! ^^"

P.s.: I just saw that you too are from Florence! Cool thing! And very cool project !

[skypjack]

@gale93 If I was you and had to design something like this, I'd put a dispatcher in a locator and let the systems refer to it directly. If you don't like the idea of using a locator, well, I'd pass the dispatcher as an argument during construction and probably during update. In no case I'd make it part of a _runner_, whatever it is.

That being said, probably I'll work on it soon for it seems that almost everybody wants this feature in EnTT.
Let's see...

---

Yep, Florence, Rifredi. I didn't know you were from Italy, nice to know. ;-)

[AndrzejKozik]

@gale93 I feel I do the same things in my game. But instead of generic update method in systems, I make dedicated event receivers (fixedUpdate, renderUpdate, prePhysXUpdate, postPhysXUpdate, and so on). In this way, instead of direct calling SystemRegistry.update(...) from engine I just publish an appropriate event. In this way I don't need system registries at all. The only problem with such an approach is the same order (or undefined order) of event receiver calls for each event type - I think that the goal of systems registries is to overcome this issue, but in my opinion it will unnecessarily complicate the game structure.

[skypjack]

Probably most of the features of which we are talking shouldn't be part of an ECS-centric framework like EnTT.
Instead, it could make sense to create an almost full-featured yet minimal engine based on EnTT that compiles as a library (header-only doesn't fit well in this case) and extends the ECS itself with all those functionalities required to create such a tool.

EnTT can be used in a wide variety of applications.
On the other side, all these requests are meant almost exclusively for the gaming sector (spaces, systems organizer with a delta, prefab, etc).
Of course, the goal could be to start a new project and port general purpose features back into this framework.

What about?

[skypjack]

Follow-up of my last message.

I created a new repo (mini) where to make experiments with EnTT so as to create a minimal engine.
I'll put there all the features I want to see in an _engine_ and that wouldn't make sense within this framework. Similarly, I'll place here all those tools that can find their way in a header-only library like this.

Stay tuned and keep on participating in both the projects!!
You're welcome.

---

Side note: I'll be out of home until next Wednesday, so I won't push anything on mini at least for a while. Don't worry, I'll be back!! :-)

[vblanco20-1]

Hello skypjack.
First of all, great work with your library.
Inspired by the new Unity ECS library, ive decided to try to implement something of the sort, but for Unreal Engine. Im evaluating multiple different C++ ECS libraries for possible creation of such integration (wich will be open source too). When creating my test project (a simple bullet hell, with a bunch of systems and multiple components, i found something Entt lacked that Entitas and EntityX dont, wich was very useful.

This feature was default events. In both entitas and entityX i can have systems register themselves to events for Component Add/Remove, and Entity Add/Remove. These default events are extremelly useful becouse they make the implementation of certain features a lot simpler.
For example, i was using a Render Mesh Component, wich had a Mesh pointer, a Material pointer (for unreal engine mesh and materials), and a Index for its own instanced mesh object (unreal engine way of rendering instances).
The system i have to do this essentially created multiple instanced meshes, one per mesh type, and then keep the index in the component. The index was kept becouse that way i only need to update the exact instance index of a specific object, and not need to update the objects that didnt move (very useful here, as modifying instances is not cheap).

In both of the other entity libraries, i had a "OnComponentRemoved" event, wich signaled the InstancedMesh system, this event added the index of the RenderMeshComponent that was removed into a "free" list, wich i then reused when i have a new RenderMesh component to render, thus keeping the instanced meshes to an absolute minimum and reusing render instances constantly.

As this event does not exist in entt, i had to add "CreateEntity" and "DestroyEntity" functions to my "ECSIntegration" class, wich in turn does call the events, as i did not want to modify the core source code of the library.But this is very error prone, as if i call destroy() on an entity that has a RenderMeshComponent, this render mesh will become garbage and stay there forever, unless i do some kind of garbage collection in the instancing system, by detected wich instance indices arent being used.

An integration of a default signal/event for a entt::Registry shouldnt be any complicated, at least for Entities themselves. a good candidate would be in the "destroy()" function on the Registry class. Just have a default "entityDestroyedEvent" or similar in the registry. (In fact im going to add this myself). But a version for components is a lot more complicated to do, but would be a very useful feature. This way, in a case like the one above, the InstancedRenderSystem just listens for OnComponentRemoved and cleans up when that happens. The downside is that such default event would cause a performance hit, as every time you add/remove an entity or component it would call the events.

In case you are interested, this is one of my benchmarks: https://gfycat.com/EnergeticWeeEagle . There are 3000 bullets there (entities) and 100something turrets (also entities). Rendering is done via instanced mesh and there is no collision, just a distance check.

[skypjack]

Hi @vblanco20-1

I'm glad you enjoy using EnTT.
Thank you for your comment too, contributions are highly appreciated.

Inspired by the new Unity ECS library, ive decided to try to implement something of the sort, but for Unreal Engine.

Wow. It sounds great!!
Is there a public repo where I can have a look at your work or is it still private and under development?

This feature was default events. [...] The downside is that such default event would cause a performance hit

You got one of the main reasons for which EnTT doesn't offer such a feature out-of-the-box.
Moreover I never had the same requirement and thus a performance hit for nothing was to be avoided my side.

A possible _alternative_ could be to replace events with components.
As an example, instead of destroying your entities and thus their RenderMeshComponents, couldn't you just add them another component (something like CleanupAndDestroy or whatever)? This way you can create a dedicated system that iterates all the entities that have that component plus those that require a cleanup step, releases things (in your case, the instanced mesh) and thus destroys entities.

Less intrusive, sort of _component way_ to do it, performance hit avoided.
= delete the destroy member function in your ECSIntegration class and do everything through components.

Of course, there are also other valid approaches to the problem that aren't based on events as well.

As an example, if you accept to have components the data members of which aren't necessarily primitive types, you can store a sort of handler instead of a _raw instance index_.
A tiny wrapper with a cast operator so as to be able to use it exactly as you would use your index, plus a mechanism similar to the one of the deleter introduced with an std::unique_ptr.
This way, whenever the handler is destroyed, the instance is automatically released under the hood.

And so on... Have you considered to use something different from events for that, so as to avoid performance hits for those operations where events aren't required?
If you have 100 components and events are used only for five of them, does it worth it at the end of the day?

These are more or less the reasons for which EnTT hasn't events on create, destroy, assign, remove and the others.
Not exactly something it lacks, that had been a choice (maybe wrong? Let's discuss it!).

An integration of a default signal/event for a entt::Registry shouldnt be any complicated, at least for Entities themselves. [...] But a version for components is a lot more complicated to do, but would be a very useful feature.

Actually it's quite simple to integrate events in both the cases.
Did you see the signalling stuff that is part of the EnTT framework? I've already all what is needed to easily add support for them.
However I'm not convinced (yet) that it's a key feature. As I said, I didn't consider it something that EnTT lacks, mainly because I succeeded in working around it somehow so far and the performance hit would be probably greater than the benefit.

---

As a side note, feel free to contact me privately if you want to go deeper through the topic.
You can find my mail in the profile.
Your project sounds great and I'm willing to help if you need.

Also, do not forget to star the project so as to spread it more and more.
More users means more comments, more help, more participation and hopefully a greater library with more and more features. ;-)

[vblanco20-1]

Thank you for your answer.

Your idea of using a "handle" with an automatic deleter is definitely a lot better of an idea than implementing an event system. Im going to try that for the specific components that need to update something when they get deleted.
At the moment my experiments are barely more than very basic experiments, but i will try to publish the experiments once they are a bit better.

[skypjack]

@vblanco20-1

_Better_ depends on the actual problem, so let's see if it works for you. ;-)
Events are a nice tool and I developed a lot of stuff about it. However, when it comes to working with an entity-component system, users are tempted to introduce events where they shouldn't because they are the easiest solution (actually the more _OOP-ish_, aren't we using components now?), often without considering drawbacks in real world cases (someone said entityx or entitas?). :-)
As a rule of thumb, if there is a more _component-oriented_ way to do it, have a go with it and you won't be disappointed. ;-)
It doesn't mean that the idea of using events is wrong, but you must carefully consider pros and cons for they aren't for free. That's all.

---

Let me know when you publish your software, I'm really interested in it.
Feel free also to contact me whenever you want if you need.

[skypjack]

Time to close this issue.
I suspect there are a few changes to do to the serialization stuff so as to compile it also on VS.
Once I did it, I'll create the v2.5.0 tag with raw views, save/restore, performance improvements and lower memory usage.
Probably it will happen during the next week, when I'll be back to home.

All the other things (if any) will be part of a future release.

Thank you all for your comments.
Really appreciated. Stay tuned!!

[skypjack]

Created tag v2.5.0 - enjoy. :-)