Sponge: RCON overloads all CPU cores

[limbo-app]

This issue was originally submitted by @Ewallee in SpongePowered/SpongeForge#2354.

---

• SpongeForge version: spongeforge-1.12.2-2705-7.1.0-BETA-3361
• Forge version: 1.12.2-forge1.12.2-14.23.4.2744
• Java version: 8

• Operating System: ubuntu 18.04

• Plugins/Mods: no mods, no plugins

Everything works as it should, until I start using RCON(sending messages and commands to servers console)
Steps to reproduce the error:
Start a sponge server 1.12.2 (actually all the latest sponge versions) and start sending messages with RCON. After ca 10-12 messages/commands all of the servers CPU cores will be overloaded.

https://i.imgur.com/rgrrJ9D.jpg

The same happens on Windows:

https://i.imgur.com/0kuXLA2.jpg

Server log: https://pastebin.com/gSg81faa

[ryantheleach]

Sounds like the RCON mixin might be somewhat out of date.

[benazarenko]

Same bug happens on Ubuntu 16.04.5, 8Gb DDR3 (1600 Mhz), i3-3220. I hope you will fix it soon.

[Mr-Krab]

I confirm this problem.

• Intel(R) Xeon(R) CPU E3-1246 v3 @ 3.50GHz

• Debian 9

• Java(TM) SE Runtime Environment (build 1.8.0_171-b11)

• SpongeAPI: 7.1.0-SNAPSHOT-6cd2dc3

• SpongeForge: 1.12.2-2705-7.1.0-BETA-3403
• Minecraft Forge: 14.23.4.2747

• Mods (48): Minecraft, Minecraft Coder Pack, Forge Mod Loader, Minecraft Forge, SpongeAPI, SpongeForge, Better Questing, CoFH Core, CoFH World, CodeChicken Lib, Console Filter, Construct's Armory, CraftTweaker JEI Support, CraftTweaker2, CreativeCore, CreativeCoreDummy, CustomNPCs, Ex Nihilo Creatio, Fishing Net Mod, Hammer Core, HoloInventory, Immersive Engineering, InstantUnify, Iron Chest, ItemPhysic, Just Enough Items, LagGoggles, Mantle, McJtyLib, Parachronology, QuestUtils, Quick Leaf Decay, Reborn Core, RebornStorage, Redstone Arsenal, Redstone Flux, Refined Storage, Solar Flux Reborn, Standard Expansion, Thermal Dynamics, Thermal Expansion, Thermal Foundation, Thermal Innovation, Tinkers' Construct, Tree Chopper, UniDict, VanillaFix, WanionLib

• Plugins (27): Minecraft, Minecraft Coder Pack, SpongeAPI, SpongeForge, FastAsyncWorldEdit, BeanCore, CatClearLag, CommandSync, EconomyLite, GriefPrevention, HuskyCrates, HuskyUI, LangSwitch, LuckPerms, Nucleus, PayDay, PlaceholderAPI, ProjectCore, SkyClaims, StackBan, UltimateChat, UniversalMarket, VillagerShops, VirtualChest, VirtualTool, WorldEdit, spark

Also there is a rapid filling of RAM.

[Aaron1011]

I'm completely unable to reproduce this issue, in both development and prodcution. Server memory usage remains constant, and the server CPU usage remains close to 0%.

[Ewallee]

Ok, im not too lazy to point out what you should do to reproduce this issue:
Everything works as it should, until I start using RCON (SENDING !!! (with RCON!) messages and commands to servers console)

[Aaron1011]

That's exactly what I did - the server continued to run properly.

[benazarenko]

@Aaron1011 https://youtu.be/gr0l6k7Qfyg

[phit]

didn't watch the full video yet, but just to clarify straight away: this has nothing to do with not believing any of you!
it simply means @Aaron1011 was unable to reproduce it when following the same steps you guys did, that doesn't mean the issue doesn't exist, possibly this only happens in production and not in a dev environment

[Aaron1011]

I tested both in development and in production - I couldn't reproduce the issue in either.

[Zidane]

nvm, shouldn't fast forward a video lol...

[phit]

@Zidane first in video is without sponge to show the issue starts when sponge is added..

[Zidane]

Oh I completely didn't realize that! Oops.

[benazarenko]

@Zidane I wanted to show the bug so I sped it up until it's "Done".
Btw forgot to mention — JDK 8u181

[masaru09]

me too
https://www.youtube.com/watch?v=mtjGYLHDUZY

[Ewallee]

@Aaron1011
May be you could record a video on how you made it to work with no lag?
I believe several people here would like to know how to run the server properly...
Try to record all the steps you did, if possible.
Thanks in advance

[dualspiral]

Spotted it. Fixing now.

[dualspiral]

Technical explanation: this only started to occur once RCON clients disconnected. We weren't handling the "end of file" read properly, so we ended up with infinite loops running for each disconnected RCON client, after four, that'll take up most of the capacity of a quad core CPU, meaning that the server thread competed for time from an RCON thread, and hence, severe TPS losses.

I put a check in for what happens when an RCON client disconnects and cleaned up our side. It's all OK now, the infinite loop has been averted.