Cataclysm-dda: What to do with obsolete/unsupported mods before stable?

[Coolthulhu]

Stable is getting closer.
We have a bunch of mods that are no longer supported in the mainline.

I see three options:

• Drop them before stable, release stable without unsupported mods.
• Get the newer versions of those, clean them up a bit and bundle with stable. Delete from experimentals after release. Note that newer versions of those mods are not compatible with older saves.
• Clean up the versions bundled with experimentals. Delete them after release.

I'd prefer the first option. Keeping the unsupported mods actually makes it harder for people to play with the updated versions of them due to conflicts during loading requiring manual deletion of files.

[ZhilkinSerg]

First option.

[Zireael07]

First option.

[DangerNoodle]

I would advocate retaining obsolete mods if it is possible to restore them to a functioning state. If I recall, the animatronic and dinosaur mods were obsoleted due to overriding monster groups not working in mods, correct? Does copy-from now permit the ability to insert edits to a monster group? If so, at a bare minimum it might be possible to restore the dinosaur mod.

[kevingranade]

Need a case by case decision, the process of obsoleting these seemed a bit
trigger happy to me, though I could be wrong. I dont have a good picture
of what the obsoletion criteria is. If it really is broken and
unsupported, I agree dropping the code is the best option.
The main guarantee that needs to be met is if you could generate a save in
0.C, we need to be able to load it in 0.D, though the newer migration cide
can get excersized to make that easier.

[DangerNoodle]

Need a case by case decision, the process of obsoleting these seemed a bit trigger happy to me, though I could be wrong.

Agreed. There are some mods that seem to have been too hastily dropped. I am still disappointed in both @chaosvolt and @mugling for their actions leading to the obsolescence of Arcana and Magic Items, but this is a moot point now that the mod author is maintaining their own repository.

[Coolthulhu]

Blazemod and PK mod are two huge mods that required lots of maintenance on our side or relatively frequent updates from the mod makers' side. The updates were rather sporadic, meaning that the core version of the mods often broke, then due to sheer size of the mods the problems remained hidden.
Players who actually cared about those mods kept updating the versions from authors' sources.

Arcana was obsoleted due to being huge and having no maintainers. As far as I recall there was a volunteer, so if it was to be consistently updated, not broken for most of the time and cleaned up for ease of maintenance, it would be the easiest one to re-mainline here.

[DangerNoodle]

Blazemod

I do not recall Vehicle Additions Pack being obsoleted. Remember that Tanks and Other Vehicles is dependent on it.

Arcana was obsoleted due to being huge and having no maintainers.

I was willing to maintain the mod, yes. I continue to find it frustrating that, prior to the mod being declared obsolete, my attempts at maintenance were repeatedly closed.

[keyspace]

Here's a table with mods currently (0.C-20878-g9b22f711c7) in master, with their authors, maintainers and obsoletion status.

---

EDIT: Oops! Had my screwit in that list - striked out.

[keyspace]

Need a case by case decision, the process of obsoleting these seemed a bit trigger happy to me,

Indeed! Procedures both for inclusion and obsoletion are pretty undefined.

TBH, I was amazed that C:DDA ships with mods in the first place, and not just "example" ones. IMO this is best handled by an external "mod manager" application, that checks compatibility, downloads mod updates, and the like.

Then I understood that:

• C:DDA includes next to every possible endworld scenario there could be, so some mods currently in master can be justified by content/balance changes. E.g., disabling "them pesky triffids who piggyback on any cataclysm they get!".

• There are many ways to play the game (stationary/mobile, city scavenger/woods survivalist, etc.), so some mods are certainly warranted to further adjust balance towards a certain play style. More_Survival_Tools is a good example IMO.

• Yet others turn certain mechanics on/off, mostly because they're experimental - e.g. boats, novitamins, filthy_morale, Experimental-Z-Level...

(Are there other categories where players benefit greatly if they're shipped with vanilla?)

Anyway, is this issue a one-off for the current situation, or about estabilishing a procedure for removal/inclusion?

[Coolthulhu]

Some mods need to be shipped in the game even if just an example of the features being used. No vitamins for example - it is implemented in such a way that it can't be easily separated. Boats are essentially mainlined content locked behind a mod due to it being unfinished.

If we need explicit rules:

Ruleset 1 (less restrictive):

• The mod needs an active maintainer OR only implement things that do not need maintenance (mapgen at the moment)
• If the mod has a separate version outside DDA repo, the one in DDA repo should be synced often (~monthly at the very least) and fully, with DDA repo rules (ie. small diffs, works between diffs, new files are linted). Further versions of the mod should be developed from the synced version, not the pre-sync one.

Ruleset 2:

• The mod is maintained in DDA repo. It doesn't have a separate version distributed outside it (except older versions), its development is mostly halted while PRs are waiting for merging.
• The mod has a maintainer or is fully linted and written without hacks.

Basically, that the mod is mainlined content "hidden" in a mod.

Examples of problems that should keep the mods off mainline:

• When PK was mainlined, the changes to it were often 1K line diff PRs that didn't work (load errors). They needed further 1K line diff PRs to start working. It was clear that the mod is being "chunked" just to get around the "no PRing 10K line diffs".
• When the most recent update to blazemod was PR'd, it was a giant dump of jsons that contained outdated hacks, incorrect definitions and was way too huge to actually test it all. We needed a specifically written test just to check if all the turrets in the mod are actually capable of firing. Now, automatic testing in itself isn't a bad idea, but it should test if the old things didn't break, not if the new things actually work.
• Animatronics were added and... that's it. All the changes to them were made only to keep them alive. Last update from the author of the mod was 2 years ago. It's a zombie of a mod that is rather unpopular and could exist outside mainline just fine.

tl;dr version: Mod is in mainline if author works with us to keep it in mainline

Now, the rules aren't written down anywhere "official" or being enforced, it's just how I'd like to see it.

[Coolthulhu]

Reviving the topic because the stable is just around the corner.

At the moment the most likely version is deleting all obsolete mods just after the stable.

[kevingranade]

There's a major point missing here, which is why from a development point
of view mods are bundled with the game in the first place.
That is to avoid having to maintain an api, potentially one that maintains
backwards compatability across changes to the game itself. Changes to the
game that impact json definitions should have those effects applied to the
affected json immediately, regardless of whether the json in question is
'core' or 'mod'.

There's a secondary consideration in that maintaining mods in mainline
encourages moving content into the core game definition. dda is voracious
for content, and the fact that certain content first appears in the main
game or in a mod is frequently arbitrary.

Regardless of whether mods live inside or outside the main repository, I
have no interest in maintaining such an api as it would necessarily divert
effort away from maintenance of the game itself and in many cases hinder
development of new features.

Unless that changes (hint:very unlikely), it behooves us to be permissive
with respect to which mods are kept in mainline.

E.g. @DangerNoodle, if you're interested in maintaining more survival tools
or arcana ( don't remember which one was at issue), feel free to pr it.

[DangerNoodle]

E.g. @DangerNoodle, if you're interested in maintaining more survival tools or arcana ( don't remember which one was at issue), feel free to pr it.

I have been maintaining More Survival Tools, but it is not obsoleted, and I would be rather saddened if anyone entertained such an idea.

Arcana and Magic Items is the one I mentioned, but it presently is being maintained in a separarate repository by Chaosvolt.

[Coolthulhu]

Changes to the game that impact json definitions should have those effects applied to the affected json immediately, regardless of whether the json in question is 'core' or 'mod'.

Regardless of whether mods live inside or outside the main repository, I have no interest in maintaining such an api as it would necessarily divert effort away from maintenance of the game itself and in many cases hinder development of new features.

Unless that changes (hint:very unlikely), it behooves us to be permissive with respect to which mods are kept in mainline.

We can't have all 3 of the above at once:

• Updating the obsoleted mods is a lot of work. Due to low quality of jsons (lack of mandatory linting), it often can't be done with search+replace.
• Due to lack of standards, automatic tools can't warn when mods have invalid definitions. And a lot of verification code is excessively permissive and will let invalid jsons through.
• Manual testing of semi-related changes to mods as big as blazemod is just plain out of the question. If they don't work with standard way of defining things, they're just out of luck. And that's even before considering that the big mods tend to use complex tricks to achieve things we didn't plan for.
• Automatic testing, which is the only option for big mods (unless we consider not testing them at all an option) can give false negatives. Those may often stay under the radar for months, until someone reports them.

Out of the 3 obsoleted-in-mainline mods, ironically the one most likely to stay updated in mainline is Arcana.
PKmod is updated in its own repo which is now pretty giant. Mainlining the updates would take a whole lot of work and patience for everyone involved.
Blazemod looks broken in its own repo and the mainlined version is a giant block of copypasted definitions full of redundant fields. Manual testing or reviewing just won't cut that.