Paper: Large scoreboard.dat results in long Downloading Terrain screen

[BlackBeltPanda]

When players connect to a server with a large scoreboard.dat file (2-3MB), they end up with a long (5-10 seconds) "Downloading Terrain" screen before they're logged in. Deleting the scoreboard.dat file results in instantaneous logins.

• scoreboard.dat file: https://www.dropbox.com/s/afnde0mzuu0cvc6/scoreboard.dat?dl=1

Plugin list:

AntiWorldFly, AsyncWorldEdit, AsyncWorldEditInjector, Auctions, AutomaticInventory, BlockStore, BossBarAPI, BuycraftX, CaseOpening, ChatControl, ChestShop, Citizens, ClaimLevels, CleanroomGenerator, CoreProtect, CraftBook, CS-CoreLib, CustomEnderChest, DeluxeChat, DeluxeTags, DiceFurniture, Essentials, EssentialsKitChest, EssentialsSpawn, FastCraftPlus, FeatherBoard, FurnitureLib, HolographicDisplays, iN_Blocks, LeaderHeads, LibsDisguises, LightAPI, LiveTweak, Lockette, MaintenanceMode, MarriageMaster, mcMMO, mcMMO_Party_Admin, MobArena, Multiverse-Core, Multiverse-NetherPortals, MVdWPlaceholderAPI, NametagEdit, NoteBlockAPI, OpenInv, PandaEssentials, PandaSkyblock, PermissionsEx, PlugMan, ProjectileTrajectory, ProtectionLib, ProtocolLib, PvPManager, Shop, SketchMap, SkQuery, Skript, Slimefun, SortingHopper, SQLibrary, SuperVanish, TerrainControl, TitleManager, UltraCosmetics, uSkyBlock, Vault, VoidSpawn, VoteRoulette, Votifier, WorldBorder, WorldEdit, WorldEditSelectionVisualizer, WorldGuard

bukkit.yml, spigot.yml, paper.yml, server.properties

• server.properties: http://pastebin.com/raw/GF32i4mt
• bukkit.yml: http://pastebin.com/raw/y0LRfKKE
• spigot.yml: http://pastebin.com/raw/C0fNL7va
• paper.yml: http://pastebin.com/raw/SyAQpici
  

  Paper build number: 704

[zachbr]

Yep, it does. In every single vanilla-based server implementation.
That's really the total extent any of us have bothered to check into this issue.

[BlackBeltPanda]

I think it's also causing this OOM error with Waterfall/Bungee: https://github.com/WaterfallMC/Waterfall/issues/53

[minecrafter]

In all likelihood, this is probably an issue with FeatherBoard.

Although correlation doesn't imply causation, I have found that FeatherBoard usage is consistent with large scoreboard.dat files.

[BlackBeltPanda]

I tried using NBTExplorer to open the scoreboard.dat file and it showed 195,000 teams but attempting to expand them just crashes NBTExplorer. If I can get it open to see the names it might be possible to determine what's creating them.

[BlackBeltPanda]

Finally got it open. It's basically a ton of these entries:
https://i.gyazo.com/4b01bacc7d5cf748337083f85390c28f.png

Each entry seems to be the same except for the "DisplayName" and "Name", which seem to be unique identifiers so I still don't know what's creating them.

[aikar]

You ultimately need to stop using the main scoreboard for things.

If Featherboard is using it, they need to stop.

Using custom scoreboards for temporary activity is a must to prevent it from being saved to the global file.

[Techcable]

Can we just add an option to disable scoreboard persistence?

[Brokkonaut]

We had the same problem on our server (huge scoreboard.dat). the culprit was the Citizens-plugin, that creates thousands of scoreboard teams...

[Techcable]

So crappy plugins save all their scoreboard data to scoreboard.dat like @aikar said.
Minecraft sends the entire scorebard.dat file to the client, assuming we're command block idiots who want their scoreboard to persist.
This retarted behavior causes multiple issues
1. Waterfall/Bungee memory usage goes crazy
- Paper stores the scoreboard.dat data globaly, waterfall scores it player-specificly (since it assumes you're not retarted and its being faked by plugins)
- I will add a workaround for this, but its still stupid
1. The client (and waterfall) take longer to load
- We have to load gigabytes of crap into memory, so this takes a while on anyone who doesn't have a 400 GB/s fiber link to OVH.

Though waterfall can fix this by sharing scoreboard data where possible, and only sending team data if nessicary. It'd be nice if server owners could disable plugins saving to scoreboard.dat, and we publicly warn plugin authors not to save to scoreboard.dat (like aikar said).

[Techcable]

Nevermind, its actually the sheer volume of the scoreboard.dat that is causing waterfall's memory usage to spike.
Waterfall assumes you're using scoreboard.dat sanely, with a correctly coded plugin or command blocks/adventure maps.

[aikar]

Try updating paper to 726+ - I just pushed a fix for this, to stop saving scoreboard teams that do not contain any players.

It appears many plugins have a bug where they do not unregister the team after removing the player, and its saving lots of empty teams.

The good news is, this patch will 'clean' your dat file, so the first boot after updating will have tons of bad teams, you can then restart 1 more time to clean out all the junk.