Paper: 1:14.2: getEntities and getLivingEntities is slow

[kangarko]

PaperSpigot 1.14.2 build 59

The problem I am experiencing is that just getting all entities or living entities in a world and doing nothing else takes too long time.

I coded a simple performance meter which will execute the code several time and compound the timings below:

[kangarko]

Here is an explanation for the timings after the first crossed New Timed Spawn Iteration line - there are 888 entities on the world, and limit-world took 85 milliseconds to perform which at present does nothing else than doing the empty for loop as the code above.

[kickash32]

Can you test this on 1.13.2?

[kangarko]

Yep, much better there.

The "Run ->" means how many times the code was run. There are more entities and the timings are considerably lower.

[kangarko]

The 1.13 is indeed much much better.

The normal users will never set their chunk radius to a high number (that influences the "run" amount), so even with insanely high entities loaded in active chunks, the timings on that version are excellent (compared to 1.14 that took over 2 seconds and server was unusable):

[kickash32]

This is likely related to the chunk not unloading issue #2106

[kangarko]

Not sure, we only iterate entities in loaded chunks around the player at the user-specified radius.

[kangarko]

@zachbr any news regarding this? Still seems to affect the most recent Paper releases, some of my customers are still reporting this

[Spottedleaf]

Using our EntityRemoveFromWorldEvent event and Chunk load and unload events you can easily make your own set

(please do not use the entityadd event, it could called for chunks that are not loaded and wont be loaded)

[kangarko]

@Spottedleaf are you suggesting I keep track on living entities within my plugin on my own? Isn't that breaking the purpose of the server's storage and adding unnecessary complexity and overhead?

[electronicboy]

The problem is heavily down to mojangs changes in 1.14, entities in chunks which aren't in a fully loaded state are in the global entity list, all operations around that list, e.g. ticking entities, bukkits returning of the list of entities, ends up having to do additional checks to ensure that it's only ticking/returning entities from those fully loaded chunks

[kangarko]

Thanks for the explanation, I understand that it's been 4 months since this is reported so that must mean there's no easy fix. Has this problem yet been reported to Mojang or should I take initiative?

Lastly, are you so suggesting I keep track of living entities with certain metadata (=Bosses) manually? I must support Spigot not only Paper so I assume handling it with ChunkLoadEvent, ChunkUnloadEvent is sufficient for already spawned ones?

[kickash32]

I doubt Mojang will care about the performance regressions of an unsupported 3rd party API...

[kangarko]

You mean Paper? Alright and if that does not get better on 1.15 are you guys considering putting efforts into fixing it? I am not the only plugin developer experiencing this.

[aikar]

You would first have to have a theory on how to fix it. mojang is unlikely to fix it.
If you have an idea, propose and discuss it on Discord in #paper-dev (not here, as it'll likely be a lot of discussion)

[aikar]

As for your plugin, you could use our better API's to maintain a count as a Paper specific improvement, so if people use your plugin on Paper it runs faster, and fall back to the current slow methods for Spigot.

Then just encourage users to use paper to get better performance.

[kangarko]

I have no idea how to fix this, I did not dig into these parts of NMS before.

What do you mean by using Paper's better API? Is there any method similar to 'getLivingEntities' or did you mean what Spottedleaf talked about earlier?

Thanks!

[aikar]

I was referencing Spotteds suggestion, in that you can use it if you detect paper, and fall back to slow calls on Spigot.

I don't believe you can accurately represent world entity state on Spigot, as it doesn't fire proper events for entity removal and even entity adds is unreliable.

My Entity Add/Remove World events are most accurate.

[Spottedleaf]

Currently entity remove and add events are not consistent with chunk load and unload (and Entity#isValid)

[aikar]

Well yeah they don't relate to chunk events, but if you want to track all active entities then that would be what you need, would need different (but more complicated than it sounds) events for Chunk moves.

[aikar]

But for what it seems is being done, he doesn't need chunk events, he just needs to track active entities.

[Spottedleaf]

Entity remove can't be used alone, the chunk unload event is called in 1.14 even when the chunk remains in memory without removing entities. After unload event is invoked these entities are not returned via bukkit api (getEntities for ex) so the events themselves are not sufficient. So it would be a combination of Entity add/remove and chunk load/unload events for a plugin to maintain a list

[Spottedleaf]

However there are ongoing internal discussions to perform all of that logic for you behind the scenes to bring performance back in-line with 1.13

[kangarko]

That is great news! Is there any ETA for that? Because it would be wasteful to implement such complex workaround if in say 1-2 weeks you guys rewrite the program that will affect all plugins, not just one, I believe I can focus on other things during that period instead.

Thanks!

[Spottedleaf]

no eta currently