Sponge: Custom Data location to store

[gabizou]

So, naturally, Entity and TileEntity can store custom DataManipulators perfectly fine, but it might be prudent that custom data from plugins may be loaded/stored outside of the Entity/TileEntity's specific NBTTagCompound to avoid contamination and cases where "server owner wants to just delete the plugin from existence, and all custom data with it." without having the issue of seeing spam errors of deserializing a DataManipulator because the plugin's DataManipulatorBuilder wasn't found.

There were a few options thrown around in the past:
1) Store the custom data as it is currently stored, all on the owner's compound, no validation and throws an error to log for missing DataManipulatorBuilders.
2) Store custom data in a separate file location per plugin where it's something similar to <worldname>/data/<pluginid>/entities.dat etc.
3) ???

So, just collecting some thoughts on how _best_ to make this work, and possibly ensure that there's some sense of performance with whatever situation it is.

[simon816]

3) A way to purge data that no longer has a registered builder. (/sponge purge-data?)

[octylFractal]

@simon816 but also only for a certain plugin, since you might not want to clear all of it in some cases?

[simon816]

Yeah that's possible assuming the data gets tagged with the plugin id or something /sponge purge-data <pluginId>

[TheRaspPie]

@simon816 But then you need Sponge to remove the plugin data. What if you want to move the world to a vanilla server for example and you don't want to start a Sponge server first, but also delete the data?

[JBYoshi]

Then the vanilla server doesn't read it (I assume), so it doesn't get saved.

[Zidane]

As with our multi-world system, we need to store this data separately. Your structure sounds good for number 2 gabizou.

EDIT
We can also add a command so you don't need to go and delete the files so @simon816 is happy and @TheRaspPie is too.

[TheRaspPie]

@Zidane Great.
I also like number 2 best because it separates vanilla data from plugin data.

[me4502]

My main reasons to do the second option are that server owners quite frequently disable plugins temporarily, which would erase all their data. Also they sometimes improperly set them up, or use unstable dev builds which cause errors on startup - causing the same side effect.

[ryantheleach]

@amaranth Had some constructive comments the last I saw him mention it in IRC, Amaranth it would be really useful to see your comments re-iterated here where it can be better preserved.

I think gab had some good input as well.

[amaranth]

My view on this was more or less the second choice with the additional requirement that you always load and attach metadata to entities and block entities even if you don't have a deserializer for it. This ensures the data stays in sync with the entity's world/position and existence.

Edit: And also for performance reasons you wouldn't want per-plugin files for this, as nice as that would be. You'll likely need to do per-region or per-chunk files with the data from every plugin in one file. One file per plugin per world would be too large (see structures) and one file per plugin per entity would be too small (see chunk storage before regions).

[XakepSDK]

Second solution sounds good.

[gabizou]

This has been resolved with the custom data config that allows admins to prune custom data by their string id.

Specifically as such:

sponge {
    data-registration {
        # A configurable list of registration ids that are to be removed 
        # when discovered for deserialization. This can be controlled by 
        # commands in sponge. It is adviseable to refer to the lists made 
        # available through 'failed-data-list', as using any id's from 
        # 'registered-data' will result in custom data being deleted at 
        # every load.
        data-to-purge=[
            "myhomes:home",
            "myhomes:friends"
        ]
        # An auto generated list, by Sponge, to discover and list 
        # all failed custom data deserializations at runtime due 
        # to a lack of the registrations being made by a plugin. 
        # Not to be confused by failed deserialization due to bad data. 
        # Modifying the list will result in no effect as Sponge auto 
        # generates this list. This is merely for user configuration.
        failed-data-list=[]
        # In the cases where there is already previously discovered data 
        # we don't want to spam the log on each discovery in certain 
        # contexts. If it is required, we still can emit the log warning 
        # when necessary.
        print-on-discovery=false
        # An auto generated list, by Sponge, to provide a list of 
        # registered custom data manipulators by plugins. Since 
        # the list is generated AFTER the game starts, modifying 
        # this list will not affect Sponge's system in any way. 
        # However, it is advisable to view what registered datas 
        # exist on a server instance, such that when Sponge completes 
        # startup, it will be verified that all existing registrations 
        # are accounted for. A warning will be emitted for any existing 
        # registrations that were not registered, and moved to the 
        # 'failed-data-list'.
        registered-data=[
            "myhomes:home",
            "myhomes:friends"
        ]
    }
}