Vega-strike-engine-source: Change the Saved Game Format

[BenjamenMeyer]

Presently the game uses a CSV format, and the former maintainers tried to also split the CSV data into multiple CSV files which we've found to be problematic (see https://github.com/vegastrike/Assets-Production/pull/7 for details).

Let's move to a different save game format.

Currently JSON, TOML, and YML are all currently excepted formats. However, only JSON has native support in Python (even Py3) without requiring additional libraries so if we need to open it from within the Python code, then we should limit it to the JSON format. (NOTE: JSON and TOML are both related to YML; I don't know why Python doesn't have a built-in YML parser.)

Tasks:

• [ ] Decide on the format to use
• [ ] Make the saved game file functionality a specific function of the game engine that is used by the assets so the assets don't have to be concerned about the file format, but can instead just do a load/save with the game engine.
• [ ] Make any conversion tools required to upgrade old file formats to the new file format.
• [ ] Provide a saved game editor tool that works with both the old and new formats, perhaps even providing a delta between the two so folks can ensure that the conversion worked properly.

[pyramid3d]

My recommendation:

Serialization Formats

There are many formats available for serialization, the most commonly used being CSV, XML, JSON, YAML.

Our requirement for data serialization is that it would be:

• Editable with a text editor
• Human readable
• Machine readable
• Compact even for large data sets
• Libraries available in popular programming languages

Following the analysis and recommendations in this article, we equally recommend JSON as the select serialization data format for fulfilling all of the above criteria better than other formats.

[1] CSV vs XML vs JSON – Which is the Best Response Data Format?. Digital Hospital. 2016. Retrieved on 2020-02-23.

Source: https://github.com/LibreGamingManifest/libre-gaming-manifest/blob/master/libre-gaming-manifest.md#serialization-formats

[stephengtuggy]

Cool. Sounds like JSON might be the way to go then.

[stephengtuggy]

Question: Are we making the correct distinction between the CSV format of the assets (which don't change) vs. the saved game format?

[nabaco]

I think @BenjamenMeyer meant the CSV assets table, since the savefile doesn't seem to be a CSV file at all.
In that regard, I would think about YAML, since it's more human friendly, especially for editing. And it is a superset of JSON, so moving from one to another should be easy.

I would also add to this discussion the game's config file.

Some more info on that:
https://www.json2yaml.com/yaml-vs-json
http://stackoverflow.com/questions/1726802/ddg#1729545

[BenjamenMeyer]

So this particular issue is the Save Game format itself; however, we might want to expand it towards what all the file formats should be. There is also some benefit to using one format versus a dozen formats as it can simplify any tool chains.

@nabaco my only issue with YAML is that it requires an additional library for Python to load it natively, and for how we use Python it'd be best to keep to just what is built-in if we can. JSON fulfills that requirement. Otherwise I wouldn't have any objection to YAML.