Terasology: gradle reliability roadmap

[keturn]

Reliable builds during development with support for combined gradle & IntelliJ IDEA.

I have a sort of roadmap for this. Maybe it's the sort of thing that should be a Milestone or have a Project board, but I'll start by writing here so it's someplace besides my head:

• [ ] [regression] #4014: transient modules/*.jar
• [x] docs: remove mentions of gradlew idea and iml files from wiki:Developing Modules
• [ ] determine whether gradlew idea and gradlew cleanIdea do harmful things and if we need to disable them
• [ ] gradle terasology-modules plugin. started by #3993.
  
  
  
  • [ ] extract cacheReflections task. work in progress, #4022. also relates to #2445.
  
  
  • [ ] make terasology-module plugin not depend on /config/gradle/publish.gradle. Surprisingly enough, it seems to work, but it is unexpected source code organization and a blocker to the plugin being used outside this repository. https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/buildSrc/src/main/kotlin/terasology-module.gradle.kts#L89
  
  
• [x] #4021: reflections and other work with resources will be easier if engine gets rid of its dev source set, see https://github.com/MovingBlocks/Terasology/issues/3979#issuecomment-633094908
• [ ] make syncAssets task (or copyResourcesToClasses, as it's called in engine's build) gradle-safe. or make unnecessary by getting Gestalt to allow us to register more than one filesystem-directory to a gestalt-module. see #3957 & https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/engine/build.gradle#L277-L278
• [ ] remove outputDir overrides on IDEA config; colliding gradle and IDEA build directories is an anti-goal.
  
  
  
  • [ ] in modules: https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/buildSrc/src/main/kotlin/terasology-module.gradle.kts#L235-L236
  
  
  • [ ] in engine: https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/engine/build.gradle#L301-L302
  
  
  • [ ] in facade:
    
    https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/facades/PC/build.gradle#L426-L427
  
  
• [ ] decouple facade's configuration from engine and module projects. Goal is to improve confidence that engine and all modules will be up-to-date whenever facade's exec targets are launched. Hope is that following multi-project best practices makes this more reliable. See also Decoupled Projects.
• [ ] streamline facade's exec tasks. There is a _lot_ of duplication between them. They can probably also take advantage of the java-application run task.
• [ ] IntelliJ mystery: no longer able to effectively rebuild individual modules? Right click module -> Rebuild (if no previous build directory) may result in nothing building, yet Build -> Build Project builds it fine (item added and observed by Cerv)

Additional Criteria

• Use task configuration avoidance and lazy configuration to keep big multi-project workspaces fast.
  
  
  
  • Retain compatibility with run-time reloading of classes (with IDE) and assets (with gestalt).
  
  

Optional Side Quests

• [ ] separate module test-dependencies from their application dependencies.
• [ ] publish terasology gradle plugin(s) for use from other repositories.
• [ ] extractNatives - is there a better place for this to be so IntelliJ will recognize it as a dependency and always make sure the libs exist before it runs the facade? does anything else need them other than facades? (e.g. tests, compile-time dependencies)

What you were trying to do

just, like, be consistently able to build and run stuff

What actually happened

I learned more about gradle than I ever wanted to know.

[keturn]

make syncAssets task (or copyResourcesToClasses, as it's called in engine's build) gradle-safe. or make unnecessary by getting Gestalt to allow us to register more than one filesystem-directory to a gestalt-module.

I still have the feeling that gestalt might be the better place to do this. But as many times as gradle has yanked the football out from under me, I'm still less afraid of working with gradle than I am of working on the old gestalt branch.

https://gfycat.com/yawningsplendidargentinehornedfrog

[Cervator]

Updated https://github.com/MovingBlocks/Terasology/wiki/Developing-Modules and checked off its box above

[keturn]

It's also worth having another look at Facade's distribution tasks, as came up in #4141. Something about the use of task properties is wonky, but only turned up as broken under Windows?

I think that changing that distribution task is mostly independent from the module build/dependency resolution stuff that makes up the bulk of the work mentioned in this ticket, so I think it'd be possible to do some work on that without being blocked by the other issues.

[keturn]

FYI: Gradle 6.7 has entered release candidate stage, which usually indicates it will be stable within a week or two.

I haven't seen anything between 6.4 – 6.6 that makes upgrading a priority for us, but I will be interested in grabbing 6.7 before resuming work on the terasology-module gradle plugin in /buildSrc/. We don't exactly depend on the features released in 6.7 but they did a significant amount of work under the hood to support the new Java Toolchain stuff, and that will be relevant for the parts of the build that get a little too involved with the java tasks's implementation details, i.e. #4022.

I don't see anything I expect to give us trouble in the upgrade guide.