Entt: Basic logic operations for registry and filtering views.

[sunbubble]

It just occurred to me that one could in principle enhance the registry.has<...>(...) and registry.any<...>(...) methods with all the basic logic operations. I admit XOR seems to me to be the most convenient, but in theory all apply.

bool xorExistingComponents = registry.XOR<component1, component2, component3, ...>(entity);

The holy grail would of course be able to generate views with more expressive logical conditions. But that might have other performance trade-offs.

Also, build times may suffer...

[skypjack]

The holy grail would of course be able to generate views with more expressive logical conditions.

How do you envision them? each returns references to actual objects. You cannot return references to things that don't exist. How do you imagine the API for that?

[sunbubble]

The holy grail would of course be able to generate views with more expressive logical conditions.

How do you envision them? each returns references to actual objects. You cannot return references to things that don't exist. How do you imagine the API for that?

Maybe something like:

registry.view<C1>(entt::exclude<C2, entt::XOR<C3,C4>>);

I'm not sure if such is possible in practice.

[skypjack]

No, I mean, how you would use it? The only way I see is to allow only range-for and iterators since each would be tricky to implement.
Another possibility is to turn the or in an std::variant and return it during the each.

[Kerndog73]

You could pass multiple functions to each. Similar to the way std::visit works. BTW, xor is a reserved keyword.

[sunbubble]

consider

registry.view<C1>(entt::exclude<C2, entt::XOR<C3,C4>>).each(...);

as an alternative to

registry.view<C1>(entt::exclude<C2>).each([&](auto entity, auto comp){
    if (!registry.XOR<C3,C4>(entity))
        // do something
});

which itself is an alternative to

registry.view<C1>(entt::exclude<C2>).each([&](auto entity, auto comp){
    if (!((registry.has<C3>(entity) && !registry.has<C4>(entity)) || (!registry.has<C3>(entity) && registry.has<C4>(entity))))
        // do something
});

This last case only gets worse with more components.

@Kerndog73 Thanks for the reserved keyword point, I UPPERCASED all the xors in the previous comments.

edit: another example:
OR corresponds to any:

registry.view<C1>(entt::exclude<entt::OR<C2,C3>>).each(...);

[sunbubble]

How do you envision them? each returns references to actual objects. You cannot return references to things that don't exist. How do you imagine the API for that?

I think that's a very valid point. I think it would be fine if it happened only in the context of entt::exclude.

For the positive case, a variant type would probably work, but that looks like an antipattern at first glance. I don't see how to avoid causing a bunch of cache misses there. You would probably want to do separate iterations when you want to access different combinations of components inside the each method.

[sunbubble]

The major use-case I see is when you have a bunch of empty flag-type components you don't want to access and you effectively want to generate a filtered view.

In this sense a companion to entt::exclude would also be entt::filter.
The concept would be, the template parameters in the view itself are the components you want to access. The ones in entt::filter and entt::exclude just filter the view for existence of other component types.

Sorry for all the edits and thinking out loud.

[skypjack]

No, well, thank you for sharing your guesses.
I'm not sure it's possible (or at least easy to implement) but I'll look into this and let you know. :+1:
I'll try to give you a feedback as soon as possible.

[sunbubble]

Awesome, thanks!

[Kerndog73]

The logical XOR operator isn't really defined for more than two operands. It might make more sense to generalize this to counting how many of the given components are attached. Then to check if only one of a list of components is attached, you'd do something like registry.count<A, B, C>(entity) == 1.

[sunbubble]

XOR is associative, so it’s trivial to generalize to any number of inputs. It’s also commutative so the order of input types wouldn’t matter. I was thinking that the template fold expressions could maybe be used for this feature I proposed.

Then to check if only one of a list of components is attached, you'd do something like registry.count(entity) == 1.

Well you could use this true, which looks quite useful by itself, but I don’t know how this would work in the entt::exclude context.

The whole point would be to be able to generate compact expressive filtered views without having to access the registry within each each function. I don’t know the internals of EnTT too well, but it looks to me this could lead to significant performance increases in some cases, as it looks far more cache coherence friendly. Because you first filter all the components, and then do work on them without accessing the registry to check for conditionals.

It also has the added benefit that you’re more flexible defining the lambdas that go in the each function because you don’t need to append a bunch of empty auto arguments at the end. And could in theory also improve the performance of views when used with the std library algorithms. This is all highly subjective and speculative.

[Kerndog73]

When you generalize XOR for more than two operands, you don't get the desired behaviour. You take XOR to mean "one and only one" but this truth table says different (see the bottom row).

| a | b | c | a XOR b | (a XOR b) XOR c |
|---|---|---|-----|---------|
| 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 1 | 0 | 1 |
| 0 | 1 | 0 | 1 | 1 |
| 0 | 1 | 1 | 1 | 0 |
| 1 | 0 | 0 | 1 | 1 |
| 1 | 0 | 1 | 1 | 0 |
| 1 | 1 | 0 | 0 | 0 |
| 1 | 1 | 1 | 0 | 1 |

If you chain XOR like this, you end up checking for odd parity. That is "an odd number of operands are true" rather than "only one operand is true". So to minimize confusion, I strongly believe that returning the count and letting the user define what XOR means is the better way to go.

---

Well you could use this true, which looks quite useful by itself, but I don’t know how this would work in the entt::exclude context.

Perhaps something like entt::count<1, A, B, C>

---

I don’t know the internals of EnTT too well, but it looks to me this could lead to significant performance increases in some cases, as it looks far more cache coherence friendly. Because you first filter all the components, and then do work on them without accessing the registry to check for conditionals.

For views, the performance would be about the same as checking the condition yourself inside the loop. It might even be a little slower because the optimizer might have difficulty seeing through all the templates. For groups however, this might actually be faster than a condition in the loop.

---

I think the addition to views that you're proposing might make more sense as an extension to entt::observer. The registry.count thing I mentioned seems somewhat useful and very easy to implement.

[sunbubble]

If you chain XOR like this, you end up checking for odd parity.

True, that was a mistake on my part. I’ll correct the name of the bool in the first comment.

I still think the larger point is that it would be very nice to be able to generate pre-filtered views, and as you say groups. And there I’d say both your count proposal and basic logic operations look very useful to me, and as you've shown only equivalent in the trivial case. I believe being able to generate views more expressively might be really nice.

Regarding performance in views, I was basing my assumption of the related fact that you should prefer view.get to registry.get. I know compilers are wizards nowadays but this pattern looks easier to vectorize to me. But performance arguments without benchmarks are a bit pointless, so forgive me for all the speculations.

[sunbubble]

I modified the name of the issue given the direction the discussion has taken. I hope you don't mind.

[Kerndog73]

We already have registry.has<A, B, C>(entity) for A AND B AND C and registry.any<A, B, C>(entity) for A OR B OR C. What else is there?

[sunbubble]

We already have registry.has<A, B, C>(entity) for A AND B AND C and registry.any<A, B, C>(entity) for A OR B OR C. What else is there?

Those are the only ones I know about. Which is why I opened this issue.

---

In theory these could also be enhanced this way:

registry.has<A, entt::XOR<B, C>>(entity)

and

registry.has<A, entt::NOT<B>>(entity)

Possibly not worth the effort unless they share the same implementation with the filtered views. And also possibly harder to optimize.

[Kerndog73]

In theory these could also be enhanced this way:

registry.has<A, entt::XOR<B, C>>(entity)

and

registry.has<A, entt::NOT<B>>(entity)

That sounds like a lot of complex templates for very little gain. I think

registry.has<A>(entity) && registry.count<B, C>(entity) == 1

and

registry.has<A>(entity) && !registry.has<B>(entity)

are more readable. C++ already has logical operators. I don't see the point in reinventing them with templates.

[sunbubble]

That sounds like a lot of complex templates for very little gain.

Agree. I was trying to get both data with a single read. But given how EnTT works, it's probably for no gain at all.

---

C++ already has logical operators. I don't see the point in reinventing them with templates.

Don't get me wrong, I'd much rather prefer this:

registry.view<C1>(entt::filter<~(C2 & (C3 ^ C4))>).each(...);

or

registry.view<C1>(entt::filter<!(C2 && (C3 != C4))>).each(...);

or even

registry.view<C1>(entt::filter<not(C2 and (C3 xor C4))>).each(...);

to what I proposed ;). Sadly I don't think that's valid C++.

[Kerndog73]

The best you can probably do in C++ is registry.view<C1>(~(c<C2> & (c<C3> ^ c<C4>))) where c is a variable template. I still think this is a lot of templates for no good reason.

[sunbubble]

I still think this is a lot of templates for no good reason.

The OR case is already implemented with entt::exclude. I don't know how hard it is to generalize. I still believe it would be very nice to be able to pre-filter views more expressively.

All this you can already do by creating multiple views with different conditions, but that leads to a lot of code repetition. And is probably not as cache friendly to iterate over, unless you somehow merge the views. Which, afaik, could only work if all merged views capture the same arrangement of components.

But yes, definitely lots of templates.

---

The best you can probably do in C++ is registry.view<C1>(~(c<C2> & (c<C3> ^ c<C4>))) where c is a variable template. I still think this is a lot of templates for no good reason.

I think what you describe is very close to what is already there. Consider:

registry.view<C1>(entt::exclude<C2, C3> && entt::has<C4>>);

The only problem in this example is that it's not very explicit whether things are AND or OR. Thus a filter method with some type of component predicate would be ideal...

But as you can see we already have logical operations implemented as templates. To be fair it's the whole technical concept behind EnTT itself.

[skypjack]

What I don't understand is: if you want to iterate all entities that have A and B, not C and either D or E, you can already do that with two views that don't overlap or even better with two partial groups (on D and E) that would have by far better performance than any other view based solution.
My concern is that here we risk to add something that all in all isn't that useful performance wise and will quickly become only a maintenance burden.

Why using two views/groups isn't acceptable? I missed the point, sorry.

[sunbubble]

ATM I'm doing exactly that, generating multiple views.

---

On performance

Maybe I'm misunderstanding the performance consideration, but imagine I create these two views with the complementary logic, as you suggest:

[X, , , , , ,X, ,X, , , , , ,X,X,X,X, , , , , , , , ,X, ]
[ ,X, , , ,X, ,X, , , ,X, , , , , , ,X, , ,X, , , , , ,X]

I can iterate over them individually. But if I could express them in a single view I'd get:

[X,X, , , ,X,X,X,X, , ,X, , ,X,X,X,X,X, , ,X, , , , ,X,X]

And I'd iterate once over these. Wouldn't this be faster because of better cache usage?
I'm of course disregarding the cost of generating the more complex views.

---

On maintenance

If I generate multiple views I need to be careful that the logic of every view is exactly complementary to that one of the others. Maintaining the logical coherence of different complementary queries is in my experience also a non-trivial maintenance burden. Especially if you want the order of the views not to matter.

[skypjack]

Wouldn't this be faster because of better cache usage?

The way views work is way more complex.
A view uses the _shortest_ pool internally, so most likely D and E in my example. Therefore, you iterate two disjoint sets of entities and all them are returned to the caller in both cases.
A partial owning group would even get rid of the test to know if the entity is _valid_ since it's so by construction.

Maintaining the logical coherence of different complementary queries is in my experience also a non-trivial maintenance burden

Wouldn't be a problem as well if you get entities that have D and entities that have E interleaved with no apparent logic?
I usually prefer to elaborate the two sets separately since there is a reason for which they are so.

[sunbubble]

Wouldn't be a problem as well if you get entities that have D and entities that have E interleaved with no apparent logic?

Maybe, I guess this one is subjective.

Anyway, thanks for all the interesting feedback. It's always quite interesting to think about the best ways to use this library :).

[skypjack]

Yeah, sure, this issue turned out to something interesting, so thank you for participating! :wink:

[sunbubble]

I just found another case in my code where this feature might be useful 😉

// f(A&, B&) and g(A&, B&)

reg.view<A, B>(entt::any<C, D>).each(f);

reg.view<A, B>(entt::exclude<C, D>).each(g);

I like the idea that I don't need to add the unused filter types to the function signatures (or use variadic lambdas).

Also this example in particular gives you the complementary set from entt::exclude.

[skypjack]

The expected result for entt::any<C, D> is that you receive an entity also when it has both C and D?
I cannot figure out the api of an each for this particular case tbh. An std::optional isn't suitable since it wouldn't be enough when the entity has both the components. So... a series of pointers that are eventually null? You would add a filter as well thus defeating the purpose.
What else?

[sunbubble]

The expected result for entt::any is that you receive an entity also when it has both C and D?

When it has either C or D. Thus the combinations of both views iterates over all pairs. You apply f on a and b if either C or D exist, if not, you apply g.

—-

Again I think I’m misunderstanding something. I never care to access C or D types, I simply want to filter the view based on whether they exist. So as a user I would never be exposed to any null pointers.

[skypjack]

Oh. So something like this?

reg.view<A, B>(entt::any<C, D>).each([](A &a, B &b) { ... });

I missed this point. I thought all time you wanted to receive instances also for C and D.

[sunbubble]

Oh. So something like this?

reg.view<A, B>(entt::any<C, D>).each([](A &a, B &b) { ... });

I missed this point. I thought all time you wanted to receive instances also for C and D.

Yes exactly this! I explicitly do NOT want to access instances of C and D.

The point is that C and D are often empty structs that work as flags. Therefore I never care to get instances of them. And this is also the reason why I think it's nice to remove them from the view's template parameters because you are exactly stating "I want my view to be created based on the existence of C and D but I never want to access them". Which is nice because it expresses which components you actually want to access.

Which is also why I was speculating that it may improve performance.

[skypjack]

You want something similar to how ranges work in C++20 then.
That is: reg.view<A, B>() is my view, let's filter it somehow. It would be similar to this in C++20:

for (auto entity: reg.view<A, B>() | std::views::filter(entt::any<C, D>)) {
    std::cout << i << ' ';
}

While the each based iteration could be something like:

view.filter(entt::any<C, D>(registry)).each([](auto entity, A &a, B &b) { ... }

Or even:

view.filter(reg.view<C>(), reg.view<D>()).each([](auto entity, A &a, B &b) { ... }

[skypjack]

Since it's not different from:

regi.vew<A, B>().each([c = reg.view<C>(), d = reg.view<D>()](auto entity, A &a, B &b) {
    if(c.has(entity) && d.has(entity)) { ... }
});

[sunbubble]

You want something similar to how ranges work in C++20 then.

Exactly. How the interface looks I didn't think much about yet.

[sunbubble]

Since it's not different from:

regi.vew<A, B>().each([c = reg.view<C>(), d = reg.view<D>()](auto entity, A &a, B &b) {
    if(c.has(entity) && d.has(entity)) { ... }
});

Does this work already? 😄 I never thought of this.
The only disadvantage of this interface is that you couple your function object to the view filtering functionality. If the filtering is done on the view's side, you're more flexible with reusing lambdas.

[skypjack]

If only forward iterators didn't have the retarded constraint of returning actual references, this could be easily implementable using a range-like approach.
Unfortunately, all we can do in that case is returning entities rather than tuple of elements. Btw maybe it's even reasonable in this case but... you'd like to have this _filter_ during an each and not during a range-for loop, right? :smile:

[sunbubble]

you'd like to have this filter during an each and not during a range-for loop, right? 😄

exactly...

---

reg.view<A, B>(entt::any<C, D>).each([](A &a, B &b) { ... });

I suggested this form because it's essentially how exclude<C, D> works.

[skypjack]

From the gitter channel. Probably, this is something doable (even though it would be a breaking change):

reg.group<A, B>(entt::filter<entt::all<C>, entt::any<D, E>, entt::none<F>>)

Similarly for views:

reg.view<A, B>(entt::filter<entt::any<C, D>, entt::none<E>>)

Where the first <...> means _own_ for groups and reg.view<A>() is equivalent to reg.view(entt::filter<entt::all<A>>) here.

Another option:

reg.group<A, B>(entt::all<C> | entt::any<D, E> | entt::none<F>)

Rather than:

reg.group<A, B>(entt::filter<entt::all<C>, entt::any<D, E>, entt::none<F>>)

That is, a more _range-ish_ approach. Similarly, for views we would have:

reg.view<A, B>(entt::any<C, D> | entt::none<E>)

Or equivalently:

reg.view(entt::all<A, B> | entt::any<C, D> | entt::none<E>)

[skypjack]

I think we can have a smooth transition with deprecated functions.
@Kerndog73 do you like this version more? It looks good actually and it doesn't add much complexity probably.

[Kerndog73]

@skypjack I'm all for mimicking the standard library and ranges is no exception

[skypjack]

If only forward iterators didn't have the retarded constraint of returning actual references, this could be easily implementable using a range-like approach.

Now we have iterable objects for views and groups. So, this:

reg.view<A, B>().each([c = reg.view<C>(), d = reg.view<D>()](auto entity, A &a, B &b) {
    if(c.has(entity) && d.has(entity)) { ... }
});

Becomes:

for(auto &&[entt, a, b]: reg.view<A, B>().each()) {
    if(registry.any<C, D>(entity)) {
        // ...
    }
}

And you can easily reuse your lambdas, members, whatever if you want.
Moreover, the object returned by each is suitable for using with C++20 ranges. Something like:

for(auto &&[entt, a, b]: reg.view<A, B>().each() | std::views::filter([&reg](auto curr) { return reg.any<C, D>(std::get<0>(curr)); })) {
    // ...
}

I haven't tried it, the code is out of my mind but it should be something similar to that.
So, all in all, imho there is nothing else to do here. Am I wrong?

[sunbubble]

This is very nice, thanks.

[skypjack]

You're welcome. I'm closing the issue then.
Feel free to reopen it if you spot any error or missing feature. Thanks.