Paper: Lots of Armor Stands and Minecarts can cause server lags and TPS drops

[BestMark]

Performance issue

Timings or Profile link:

https://timings.aikar.co/v2/?id=34fe5e1afdcf4d57b2ad12b32b492b3b

Description of issue:

As you know, Armor Stands are falling blocks, so, we can set more than 1000 armor stands per 1 block/chunk and server begins to lag. Same problem with minecarts.

Plugin list:

Independent of plugins.

bukkit.yml, spigot.yml, paper.yml, server.properties

Default, without changes

Other helpful links

http://i.imgur.com/8KogGZL.png

Paper build number:

Version: git-Paper-761 (MC: 1.9.4)

Suggestion

I hope that you will limit this entities per chunk.

[zachbr]

So if we spam lots of entities in a single chunk, they cause lag? You may be onto something here...

We, Spigot, and all the way up to CraftBukkit, already provide limits on natural entity spawns. If a plugin or command spawns the entity, it bypasses those limits. As it should. There are a multitude of plugins that exist to cover all cases and do per-chunk entity limiting, and I advise you use one of them.

(and no, Armor stands are not falling blocks).

[BestMark]

I think that this limit doesn't work properly in PaperSpigot.
If I use Spigot, the problem is not observed. This issue applies only to the PaperSpigot.
I have spawned more than 10000 armor stands using Spigot and there were only client-side lags. It didn't affect the TPS in Spigot.
That fixed in TacoSpigot, because there is option "armor-stand.optimize-movement" that disables gravity for armor stands.
P.S. Sorry for my bad English

[zachbr]

The limit _does not have any effect on things you spawn manually._ In any server implementation. At all, none.

The thing killing you is probably the collisions, and I am unable to reproduce any paper specific issues there. Don't spam a bunch in a single block and it'll be fine. If you think they need optimization, that's fine. But putting lots and lots in a single chunk/single block is not a realistic test case, nor one we will care about.

[ForsMine]

@Zbob750
"But putting lots and lots in a single chunk/single block is not a realistic test cast. nor one we will care about"
So, it means that TacoSpigot's developers care about it, but you not? Very funny.
I have tested this bug. It is not problematic to crash server. If you are the developer, which wants to create optimized version of spigot, you must fix bugs at least.
"Don't spam a bunch in a single block and it'll be fine"
Say this to players, who wants to use this bug to crash servers. I don't understand what prevents you to fix it.

[sgdc3]

@ForsMine The truth is that they don't need this feature on their (the devs) server. This project is becoming selfish as spigot.

[MrPowerGamerBR]

@ForsMine the point is: It isn't a Paper specific issue, and this issue is easily fixed with a plugin.

Yes, TacoSpigot has a patch to optimize armor stands, but it also breaks armor stand gravity (so you will have floating armor stands in your server, if you like that.

[ForsMine]

@MrPowerGamerBR
"It isn't a Paper specific issue"
So, that's why spigot doesn't have this issue?
Why server creators must write plugins to fix it, when you can simply use Spigot?
I am not going to use PaperSpigot, while there are such dangerous issues.

[MrPowerGamerBR]

@ForsMine

The limit does not have any effect on things you spawn manually. In any server implementation. At all, none.

Try going thru every Paper 1.9/1.10 patch to see if there was any armor stand related patch, IIRC there is none related to armor stands.

[Black-Hole]

Why are you not using plugins like Moblimiter? They are configurable in so many details.

Paper is a community project, so if you blame it for being "selfish" you should reflect about yourself:
You are pointing out a case that makes sense for your server only.
You contributed to project exactly nothing.

[sgdc3]

You are pointing out a case that makes sense for your server only.

Yeeeeah sure.

[sgdc3]

Paper is a community project, so if you blame it for being "selfish" you should reflect about yourself.

I'm currently contributing to many community projects, and as team we always try to listen to the feedback of our users. I know that a plugin can fix this but paper should at least limit the max number of colliding entities.

[BestMark]

But this bug not observed using Spigot, so I think, this "limit" doens't work only in PaperSpigot.
Try to manually install more than 1000 armorstands in creative gamemode per chunk using Spigot and PaperSpigot and you will see the difference in performance for the worse.
The problem applies to all servers with creative mode.

[MrPowerGamerBR]

@sgdc3

spigot.yml

Default: 8
Type: Integer
Description: Limit a single entity to colliding x number of configurable times per tick. This setting lowers the performance impact of entities trapped in a 1x1 pen.

https://www.spigotmc.org/wiki/spigot-configuration/

Unless this is broken in Paper, but again, I didn't see any patch affecting the entity collision code.

[MrPowerGamerBR]

@BestMark that's already fixed, it was a issue in Spigot, not in Paper, Paper already pulled the changes in upsteam (the changes in Spigot)

[sgdc3]

@MrPowerGamerBR This limits the number of collisions per tick not the max amout of colliding entities.

[MrPowerGamerBR]

@sgdc3 change it to 0, no more collisions per tick, so no more lag due to a lot of entity collisions.

(well, IIRC setting to 0 removes every collision, so maybe that will help the issue, see this (I tested in 1.7.10 because I had a local server open)):

If you see, collision isn't affecting those pigs (if the collision was set to anything above 0, the pigs would "explode" (getting pushed due collision))

This issue is because the armor stands are generating a lot of lag due to a lot of collisions between them, right? Then set it to 0 and see if does help the issue.

[sgdc3]

Yeah, the main problem is solved in this way, but it's really needed to disable collisions at all? There should be another way :/

[MrPowerGamerBR]

@sgdc3 well, people can also crash a server by spawning a lot of pigs (or any other entity) in a 1x1x1 space, disabling collisions fixes the issue.

Disabling collisions per entity type would be a good solution, but I don't know if it is easy to implement that.

[Techcable]

Why are you giving players permissions to spawn 1000s of entities, then complaining about them crashing the server?
You should consider limiting a player's permission to spawn entities, so they don't cause lag.
Again, this is _not_ a paper issue, this is an issue with you giving your players permissions to spawn 1000s of entities.

You should consider hiring a developer to put limits on the number of entities a player can spawn, thats the _real_ fix for this issue.
Even if you disable collisions, entitiy ticking will still murder your server.
As long as players can spawn 1000s of entities, you aren't safe.

[sgdc3]

@Techcable Ok, but why do we need a plugin? Why shoudn't it be an inbuilt feature of Paper?

[zachbr]

The truth is there are two of us that primarily contribute to this project, and a whole heap of others that chime in occasional fixes and feature requests, that we are grateful for, yes.

This issue is not an example of us "ignoring our community", it's an example of a single feature that isn't a feature and has been done a million times already by others so us duplicating their work is nonsensical. Not selfish. Skip the accusations and the stupid drama.

So, it means that TacoSpigot's developers care about it, but you not? Very funny.
I have tested this bug. It is not problematic to crash server. If you are the developer, which wants to create optimized version of spigot, you must fix bugs at least.
"Don't spam a bunch in a single block and it'll be fine"
Say this to players, who wants to use this bug to crash servers. I don't understand what prevents you to fix it.

So lets start with us not caring, and Taco caring:

[11:37:16] <Techcable> lol, I care even less than Z750 does

If we didn't care owe wouldn't be putting endless amounts of time into a community project for everyone, one that I _don't have any practical purpose for whatsoever, seeing as I no longer maintain a public server_. If we didn't care, I wouldn't be losing money paying for servers to host downloads and maven repos. If we didn't care, I would have blocked you for disagreeing instead of explaining anything here.
This mindset that we don't care, is insulting. You disagree, fine, but if you think we don't care, I kindly ask you to go run something else, _without_ any of our code in it.

Yes, TacoSpigot has a patch to optimize armor stands, but it also breaks armor stand gravity (so you will have floating armor stands in your server, if you like that.

Which is why we didn't pull that one from them, but no we don't care, really it's that simple /s

But this bug not observed using Spigot, so I think, this "limit" doens't work only in PaperSpigot.
Try to manually install more than 1000 armorstands in creative gamemode per chunk using Spigot and PaperSpigot and you will see the difference in performance for the worse.
The problem applies to all servers with creative mode.

as I have said earlier I cannot reproduce any Paper specific performance issues.

I can't speak for everyone here, but I contribute to avoid drama so if we want to create a new issue with less of it that's fine. But if we want to redo this one with blatant accusations and no real solutions being presented, please skip it and go elsewhere.

[aikar]

Wanting to add that the ultimate issue is too many entities, and the only reason you may not see it on spigot is due to a really bad feature called entity tick limiter.

Spigot replaces lag with inconsistent server behavior instead, and we removed that bad system.

The only valid solution to lag because of too many entities, is to control your entities.