Entt: What should the shared components API look like?

[Kerndog73]

At the moment, it seems unclear as to what shared components will actually look like when implemented in EnTT. I thought it would be a good idea to open an issue so that the discussion is all in one place and we can refer to it later. I find that conversations get lost and forgotten on Gitter.

I came up with a few things that I think could be part of the shared components API. This issue is for discussing that API and the use cases for shared components.

• attach<C>(e, ...)
  
  
  
  • Construct C and attach it to e
  
  
  • e will have its own copy of C
  
  
  • This is valid even if e is already sharing C
  
  
• attach_copy<C>(e)
  
  
  
  • Copy construct the existing C and attach it to e
  
  
  • e is given its own copy of C
  
  
  • Assert that e is sharing C
  
  
• attach_from<C>(e, s)
  
  
  
  • Take the C that is attached to s and share it with e
  
  
  • This is valid even if e is already sharing C
  
  
  • Assert that s is sharing C
  
  
• sharing<C>(a, b)
  
  
  
  • Are a and b both sharing the same C?
  
  
  • Assert that both a and b are sharing possibly different instances of C
  
  
• has<C>(e)
  
  
  
  • Is e sharing C?
  
  
• remove<C>(e)
  
  
  
  • Detach C from e
  
  
  • If e was the last entity to share C, we must destroy C.
  
  
  • Assert that e is sharing C
  
  
• unique<C>(e)
  
  
  
  • Does e have just one C all to itself?
  
  
  • No use cases have popped into my head so maybe it's useless. Also, I'd imagine that you'd have to iterate the whole sparse set to find this information so it's probably not a good idea.
  
  
• sharing_with<C>(e)
  
  
  
  • Get the list of entities that share that same C shared by e. Not sure if we should also include e.
  
  
  • I'm not sure how efficient this would be. It might also involve iterating the whole sparse set.
  
  
  • We could return an iterator range so that the user could copy the list to a collection or just iterate the entities if they want to. This is faster and more flexible than returning a std::vector.
  
  
  • The iterators could iterate the sparse set and return only the entities that share C with e. Maybe they could do something more efficient.
  
  
• view<C>()
  
  
  
  • Single component view of shared C
  
  
  • Iterations will potentially show the same instance of C multiple times
  
  
  • Raw iterations will provide access to each instance of C only once
  
  
• view<C, R>()
  
  
  
  • Multi-component view of shared C and regular R
  
  
  • Iterations will potentially show the same instance of C multiple times
  
  
  • Raw iterations are not allowed. It wouldn't make sense anyway.
  
  

The names I've chosen are just placeholders. Maybe I'm missing some functions. Maybe I've included something I shouldn't have. Maybe something behaves differently to what users will expect.

Discuss!

[ArnCarveris]

@Kerndog73

sharing_with<C>(e) -> each_shared<C, Func>(e, func)

[Kerndog73]

@ArnCarveris That probably fits better with the existing API. I don’t think there are any functions that return iterator ranges. There are quite a few eachs so it feels familiar.

[ArnCarveris]

Iterator ranges are needed in general.

[ArnCarveris]

@Kerndog73

attach<C>(e, ...) -> assign_shared<C>(e, ...)
attach_copy<C>(e) -> assign_copy<C>(e, s)
attach_from<C>(e, s) -> assign_from<C>(e, s)

[Kerndog73]

@ArnCarveris So assign_copy will copy the C that s shares and assign it to e? Seems to make more conceptual sense since we’re actually assigning something.

I’m not sure about assign_from though. We might need to include share in the name to contrast between copying and sharing.

Since shared components have a trait to identify them as shared components (was that the consensus?), we could use assign instead of assign_shared if we want.

[ArnCarveris]

@Kerndog73
attach<C>(e, ...) -> assign_shared<C>(e, ...)
attach_from<C>(e, s) -> assign_shared_from<C>(e, s)

attach_copy<C>(e) -> clone<C>(e, s) - overload

[Kerndog73]

How about assign(e, ...), assign_share(e, s), assign_copy(e, s)?

[ArnCarveris]

@Kerndog73

auto& a0 = reg.assign<A>(e1);
auto& a1 = reg.assign<A>(e2);

// a0 and a1 are unique instances?

[Kerndog73]

@ArnCarveris Yes, they are unique

[ArnCarveris]

@Kerndog73

assign_share<A>(e, s) - will assert if A not shared?
assign_copy<A>(e, s) - will work with non-shared component type also?

[Kerndog73]

@ArnCarveris

assign_share will assert(has<A>(s)). e may already share A with another group of entities. We can static_assert that A is a shared component.

assign_copy is basically just a convenience wrapper for assign<A>(e, get<A>(s)) so it could work with regular components too. We don't actually need it but it might make code clearer.

[ArnCarveris]

@Kerndog73

sharing<C>(a, b) - better have to assert only for checking if a and b is valid

[Kerndog73]

@ArnCarveris

It might be a teeny tiny bit faster if we assert that both entities have the component. However, what you’re saying might fit more closely with the use case so it makes sense.

[ArnCarveris]

@Kerndog73

unique<C>(e) -> shared_count<C>(e), or shares<C>(e) - Returns the number of shared component instances of the given entity. (There should be grabbed from internal, no useless cycles are needed)

[Kerndog73]

@ArnCarveris

Is there a use case for that function? The implementation will have to keep track of the shared count to avoid iterating the whole sparse set. Does it really make sense to extend the data structure to optimise a function that might not be used?

[Kerndog73]

@skypjack Someone with a better understanding of the implementation details (like yourself) might help in justifying some of these functions.

[ArnCarveris]

Well now I have to count it manually, there use case:

        template<typename... Types>
        struct node {};

        template<typename... Types>
        struct link
        {
            entity_type target;
        };
        template<typename... Types>
        struct cluster
        {
            entity_type first;
            count_type    count;
        };

        struct root
        {
            registry_type    registry;


            template<typename... Types>
            auto make_node(entt::type_list<Types...>&&)
            {
                auto ret = registry.create();

                registry.assign<node<Types...>>(ret);

                return ret;
            }

            template<typename... Types>
            auto& make_link(entity_type from, entity_type to, entt::type_list<Types...>&&)
            {
                return registry.assign<link<Types...>>(from, to);
            }

            template<typename Type, typename... Args>
            auto make_node(Args&&... args)
            {
                auto ret = make_node(entt::type_list<Type>{});

                registry.assign<Type>(ret, std::forward<Args>(args)...);

                return ret;
            }

            template<typename Type, typename... Args>
            auto make_cluster_node(entity_type cluster, Args&&... args)
            {
                auto node = make_node<Type>(std::forward<Args>(args)...);

                update_cluster<Type>(cluster, node);

                return node;
            }
            template<typename... Types>
            auto& update_cluster(entity_type owner, entity_type node)
            {
                make_link(node, owner, entt::type_list<cluster<Types...>>{});

                if (auto ptr = registry.try_get<cluster<Types...>>(owner))
                {
                    ++ptr->count;

                    return *ptr;
                }
                else
                {
                    return registry.assign<cluster<Types...>>(owner, node, count_type{ 1 });
                }
            }

            template<typename Func, typename... Types>
            bool each_cluster_link(const cluster<Types...>& ref, Func&& func) {
                auto ret = false;

                auto link_view = make_link_view(entt::type_list<cluster<Types...>>{});

                auto it = link_view.find(ref.first);

                if (it != link_view.end())
                {
                    auto a = it;
                    auto b = it - ref.count;

                    for (it = a; it != b; it--)
                    {
                        if (func(entity_type(*it)))
                        {
                            ret = true;

                            break;
                        }
                    }
                }

                return ret;
            }

            template<typename... Types>
            auto make_link_view(entt::type_list<Types...>&&) {
                return registry.view<link<Types...>>();
            }
        };

[skypjack]

I'm trying to write down the (almost) _final API_ I'd expect to see.
I'll add it here as soon as I finish it. In the meantime, please continue to discuss your idea. They are really useful for the purpose. :+1:

[Kerndog73]

@ArnCarveris

I think shared_count could be implemented more efficiently by the user so I don’t think it makes sense to be part of EnTT.

I’m a bit on-the-fence about each_shared. The user could use a std::vector which is faster but less memory efficient. EnTT might have to iterate the whole set (but I’m not sure) so it will be slower but more memory efficient. It sounds like a useful function though.

EnTT should minimise the size of the API and maximise the number of things you can do with it.

[Kerndog73]

@skypjack I thought about ways to optimize each_shared to avoid iterating the whole sparse set. There might be a way to sort the set to make entities that are sharing the same component contiguous. We could call it sort_shared. Then we'd have a fast_each_shared function that avoids iterating the whole set by assuming this contiguous layout. If the entities are contiguous, it might even be possible to get a pointer and size to the entity IDs (a very fast_ function).

Is this actually possible?

[skypjack]

The main problem I see with this is that a shared component cannot be sorted nor be part of a group, because in both cases you want to get the _ownership_ of the pool and induce the order you need. If this is acceptable, then yes, it's possible.

[Kerndog73]

The user could probably use signals to maintain a std::vector<entt::entity> owners stored in the shared component. Then the user can iterate that instead of using each_shared. Would the existing construction and destruction signals work for this purpose or would we need new signals? The user can do it better so we can scrap each_shared.

[skypjack]

What I don't understand yet is what are the real benefits of this. I'll try to explain what I mean, so that we can discuss it.

Today, if I wanted to use shared components, I'd use a handle to an external data structure as a component and assign it to my entities (actually I do it for eg textures). This way I can sort pools and assign them to groups.
If we remove the each_shared (and this makes sense because it would ruin a bit everything), I'd implement shared components exactly in the same way: internal handles and a separate data structures where components are laid out.
So, is this only syntactic sugar at the end of the day? Something to avoid implementing it externally? Let's discuss it.

---

As a side note and if you are following the discussion about paged pools on gitter, that feature would ease a lot shared components and make possible an efficient each_shared instead, other than some other fancy things.

[joaosausen]

Today, if I wanted to use shared components, I'd use a handle to an external data structure as a component and assign it to my entities (actually I do it for eg textures). This way I can sort pools and assign them to groups.

What do you mean? I'm looking for a way to reference entities to other entities, like "units" to a "player". In your example, how would you do that?

[skypjack]

@joaosausen do you mean parent-child relationship? That's another beast but quite simple indeed.

[joaosausen]

yes @skypjack Is there a expected way to handle this on entt? I'm not sure what I should do if I'm iterating on a entity and I need all child entities to make decisions.

[Kerndog73]

@skypjack I think it’s better to store components in the registry. For the same reason we have context variables: the registry is the one _source of truth_.

[skypjack]

It makes sense. Have you read the paging stuff I'm discussing on gitter? I think we could use pages to solve also the problem of the each_shared - a shared instance per page and that's all. Probably it's worth to hold up this issue until pages proved to be the right choice (or the wrong one, but hopefully this won't be the case).

[skypjack]

I've thought to this request. I think there are two cases that should be treated separately.
One case is that in which I've N entities and I want all of them to share the same component. Here it's share as in std::shared_ptr, that is there is not an explicit owner and the component is deleted when the last entity that refers to it is destroyed.
The other case is that in which I want to define a (let me sat) prefab class with its own components and all the instances that generated from cloning this object refer initially to the components of the prefab itself (until at least I explicitly replace the component). In this case, the prefab owns the components and its lifetime must overcome that of the instances that generated from it.
Intuitively, the latter collapses on the former if I treat the prefab as if it was a normal entity. However, if we want to allow an ownership model where the components aren't really _shared_, that is a model where there exists an _owner_, these two cases must be probably treated separately. Have you thought of this?

[Kerndog73]

@skypjack Would the ownership model make it easier to iterate instances of a prefab? Maybe I could say "I don't want the owner entity".

If that isn't the case then I don't really see the benefit of the ownership model in EnTT. The user could probably do this by storing the entity ID of the owner in the shared component.

[ArnCarveris]

@skypjack Well second case sounds like component reference:

template<typename Entity, typename Comp>
struct ref
{
  Entity owner;
};

[skypjack]

I started implementing this yesterday.
I was sure I had finally a good design for shared stuff, but still I realized soon that it lacks the possibility to iterate the entities that share a given component.
I'm almost there tho, but built-in shared components with sparse sets aren't that obvious while they are ironically straightforward to implement on top of a registry.

[alanjfs]

Aside from how this can technically work, I'm curious about what usecases people have in mind for shared components? I've found myself wanting shared components, only to realise that it would have introduced additional problem that would be harder to solve.

For example, I wanted to add user input as components.

• MousePressComponent
• KeyReleaseComponent
• ...

And obviously, the inputs would come from the same location and so would be identical. It would make sense to then generate 1 mouse press component and share it amongst entities.

But then..

registry.view<MousePressComponent>().each([](auto entity, const auto& event) {
  // ...
});

1. Should it iterate through all entities, returning the same component over and over?
2. Or should it iterate through only the unique components, like it normally would, which in this case is just one?
3. And if so, what entity should it return?

And if (1), then what about a non-const component? For example, say I've got a shared color amongst a group of entities.

registry.view<Position, Color>().each([](const auto& pos, auto& col) {
  col.r = 1.0f / pos.x;
});

What do we expect to have happen?

So from where I'm standing, the problem doesn't seem technical, but practical. Logical. But maybe I'm overlooking something.. What are some examples of problems people are looking to solve with shared components? My guess is there are alternative - better - ways of solving any issue that doesn't involve sharing components.

[skypjack]

Oh, I completely forgot to reply this question @alanjfs I'm sorry. Here I am though.

I'm curious about what usecases people have in mind for shared components?

An example that comes to my mind is to manage resources but in this case I wouldn't suggest to use shared components as described above.
The fact is that I don't allow a registry to store and manage my resources. Instead, I've a dedicated class for each type that can predict, preload, perform a lazy unload and so on. This class returns handles to resources and I store those handles in my components. A sort of flyweight pattern made in terms of components. If two entities refer the same resource, they have two copies of the same handle that will guarantee the lifetime of the resource and all the other fancy things you want to do with resources.

For example, I wanted to add user input as components.

Don't do it. Inputs can easily go on different entities that are intercepted by different systems that run at different frequencies (eg local playing character, network queue, log, hud, etc). Trying to share the input here can have more cons than pros.

My guess is there are alternative - better - ways of solving any issue that doesn't involve sharing components.

I've a slightly different opinion. The problem is that there doesn't exist a single way of _sharing_ something.
I can understand why a framework that offers a single way depicts it as if that's all you need to solve all you problems but everything isn't a nail just because you've a hammer.

Let's consider some models from the point of view of the data only:

• The _flyweight-like_ approach described above.
• A sharing model where you edit a copy and all the entities see the new values.
• A sharing model with a copy-on-write policy
• ...

Now, let's discuss the _ownership_:

• A sharing model where an owner exists and the lifetime of the shared component is the same of its owner.
• A sharing model where an owner exists and a new owner is somehow elected when the former dies.
• An _owner-less_ sharing model, mostly similar to the _flyweight-like_ one described above but without handles.
• ...

And so on. I can mention use cases for all these models, so what?
It doesn't make much sense to implement a single model, then go around and push it as if it's the best thing ever. I'm not this kind of developer or maintainer. If I can't do something that is complete or at least enough general purpose to match most of the use cases, then it's pointless from my point of view.

I thought of closing this issue more than once but I want to write a blog post on this topic, so as to have a good link for the newcomers.

[alanjfs]

Thanks @skypjack

An example that comes to my mind is to manage resources but in this case I wouldn't suggest to use shared components as described above.

Ok, but then what are some use cases for sharing components? I don't think that's been answered here; should it be implemented without at least one usecase in mind?

Don't do it. ... I've a slightly different opinion.

I think I didn't make my point well enough here, which was that I tried doing this and quickly found it wasn't a good fit. I think we've actually got the same opinion on this. :)

So, my question is not "How do we implemented shared components?" but rather "For what purposes(es) is shared components a good idea?"

[skypjack]

Ok, but then what are some use cases for sharing components? I don't think that's been answered here; should it be implemented without at least one usecase in mind?

There are some (few, actually) use cases but not all them are matched by the same model.

An example is for rendering, when you've N (with N pretty high) entities that share eg the same mesh. In this case, what you want is to go from the mesh to the entities to optimize the rendering phase, nothing more and nothing less. You aren't interested in things like - _oh, wow, I can edit the component and all entities see the same changes_. It wouldn't even have much sense here.

Another example is at gameplay level, eg when your characters share a _race_ or similar. If you want to edit a property for all the entities that have a given race, you can do it on a shared component and do it once. However, in this case it's interesting most of the times to have a _copy-on-write_ model or something like this aside the _shared_ component to allow an entity and thus its component to live independently from the others when needed.

That said, I cannot imagine a _one-match-all_ model and the few use cases that come to my mind are pretty different and require a dedicated approach. This is why I'm so skeptical on this topic.
I wouldn't say that it should be implemented without a use case though. Instead, I think the _right_ model strictly depends on the use case and that's more or less the reason behind my previous answer.

I think I didn't make my point well enough here, which was that I tried doing this and quickly found it wasn't a good fit. I think we've actually got the same opinion on this. :)

Indeed. Put aside the few and very specific cases I can come up, I think it's pretty much more a _marketing feature_ than a must-have. This is mainly due to the fact that the risk is that what you have has an eventually built-in feature isn't what you need most of the times.

So, my question is not "How do we implemented shared components?" but rather "For what purposes(es) is shared components a good idea?"

It mostly depends on the application on which you're working. You shouldn't use something just because _it's there_.
When you find that you really need it, ping me and I'll be glad to describe how to implement it _correctly_ in EnTT, where correctly here means _in a way that makes sense for your needs_. :wink:

[skypjack]

Time to close this issue.
I'm about to publish a post where I describe the possible models for sharing data and I explain the reasons why EnTT doesn't offer one of them out-of-the-box.
Feel free to reopen this if the post doesn't satisfy your need. :wink:

[bjadamson]

Here's the link to the post for anyone else who finds this thread through google:
https://skypjack.github.io/2020-02-02-ecs-baf-part-7/

[skypjack]

Good point @bjadamson thank you.