Micronaut-core: Incremental annotation processor lists many originating elements for subclasses, causing full compilation.

[nhoughto]

Steps to Reproduce

1. Clone https://github.com/nhoughto/incrementalAnnotationTesting
2. Run gradlew compileJava -i
3. Change MySubClass.java
4. Re-run gradlew compileJava -i

Expected Behaviour

Incremental compilation works and MySubClass is recompiled

Actual Behaviour

Gradle full recompilation is triggered as $$MySubClassDefinitionClass has multiple originating classes.

> Task :compileJava
Caching disabled for task ':compileJava' because:
  Build cache is disabled
Task ':compileJava' is not up-to-date because:
  Input property 'stableSources' file /Users/user/Desktop/incrementalAnnotationTesting/src/main/java/test/MySubClass.java has changed.
Created classpath snapshot for incremental compilation in 0.041 secs. 1 duplicate classes found in classpath (see all with --debug).
Class dependency analysis for incremental compilation took 0.013 secs.
Full recompilation is required because the generated type 'test.$MySubClassDefinitionClass' must have exactly one originating element, but had 2. Analysis took 0.062 secs.
Compiling with JDK Java compiler API.
Note: Creating bean classes for 3 type elements
:compileJava (Thread[Execution worker for ':',5,main]) completed. Took 1.409 secs.

It appears that super classes are considered as originating classes if they are handled by micronaut annotation processor, in the given example if you remove the @Inject from MyClass.java then Micronaut doesn't have to create the $$.. classes and incremental compilation works as expected, with just MySubClass.java being built. This extends to 3rd party / upstream Micronaut classes too, like io.micronaut.management.health.indicator.AbstractHealthIndicator. I hope this is not expected behaviour 🤞 as it would make it very difficult to get the benefits of incremental compilation with a project of any real complexity (can't use inheritance!?)

Environment Information

• Operating System: macOS
• Micronaut Version: 2.3.3
• JDK Version: Java 14

Example Application

https://github.com/nhoughto/incrementalAnnotationTesting

[nhoughto]

Any thoughts on how to progress this? Happy to do more investigation and provide more info if required. @graemerocher seems like you did a bunch of work in this area around this time last year?

[nhoughto]

Bit more context about the problem it causes, in our relatively small (6 month old codebase) this causes a 30+ sec build step for every tiny code change, change 1 byte in a test it takes 30 seconds to re-run.. its a constant hit to our productivity. I imagine others must be running into this too 🤔

[asodja]

I think this is a result of: https://github.com/micronaut-projects/micronaut-core/issues/4223. So basically IntelliJ wants to have that info but Gradle doesn't like it. So it might be tricky to deal with it.

[graemerocher]

yes that is the root cause of the issue, the only thing I can think of is adding an annotation processor property that you can set to enable or disable this behaviour. Note that there are cases in Micronaut where super classes contribute to a child classes generated bean definition (such as an @Inject on a parent field or method) so if we were to disable this behaviour it would likely lead to incorrect compilation state.

Really what needs to happen here is an issue needs to be raised on the Gradle side so they can implement support for this like IntelliJ's compiler has done. Having said that in the interim I am not opposed to making it configurable.

The change would have to go into this class https://github.com/micronaut-projects/micronaut-core/blob/a3e3dd1c0f702d9d850cb66c79db0f173c8f908e/inject-java/src/main/java/io/micronaut/annotation/processing/AnnotationProcessingOutputVisitor.java#L84

Where it would report only the first element has the originating element.

[nhoughto]

Interesting, wrt that issue is there a test case showing the problem? Or was it more theoretical?

In the example given I’d expect a change to Test to recompile itself, and a change to BaseTest to recompile itself and any subclasses (depending if the change meant a difference to the ABI or not, as per compiler avoidance rules). I wouldn’t expect a change to Test to trigger a recompile of BaseTest (subclass shouldn’t effect a superclass) and then knock on effects of all the BaseTest subclasses..

Is there a scenario where Test is not being recompiled with a change to BaseTest that would require reporting both as originating sources for Test? Can that change just be reverted? 🤞🏼

[graemerocher]

example showing the issue is described in https://youtrack.jetbrains.com/issue/IDEA-250670

[nhoughto]

Hmm ok so it’s an intelij interaction that doesn’t happen in straight gradle, could it be conditional? Like reporting multiple originating sources is against the gradle rules so is breaking the interface.. but then IntelliJ requires multiple sources and not doing that is breaking their interface.. 😖

so both solutions are mutually exclusive, so either forgo one or do both depending on the environment it’s operating in? 😬

[nhoughto]

Strange that intelij would have a problem with this when it just delegates build and test to gradle in its normal configuration 🤔
The non default config of intellij doing build and test itself would make sense.. but then the environment would be quite different and easy to detect?

[asodja]

Yes, this issue described in https://youtrack.jetbrains.com/issue/IDEA-250670 happened if you used Build and run using: IntelliJ and Run tests using: IntelliJ. It also affected Maven users since IntelliJ handles compilation for Maven projects by default I believe (not sure how many Micronaut users use Maven though).

[graemerocher]

@nhoughto as I said I recommend reporting an issue to the Gradle issue tracker https://github.com/gradle/gradle/issues/

so this problem can be addressed on the Gradle side long term. I don't know why Gradle requires a full recompile but IntelliJ does not, seems a limitation with Gradle.

In the meantime we can maybe make it configurable and disable this behaviour for Gradle builds, however like I said it is likely that the issue described in https://youtrack.jetbrains.com/issue/IDEA-250670 would then manifest in Gradle builds so really the correct path forward is an issue report to Gradle and a fix in Gradle

[nhoughto]

I don't see how this can be a Gradle issue? They are quite clear on the requirements for incremental annotation processing (one originating source), and Gradle itself has no issue (as described in the youtrack issue, works fine with Gradle on CLI and if running IntelliJ in default 'delegate to Gradle' mode). It is entirely an IntelliJ problem when using internal IntelliJ processes (not Gradle processes) to build and run tests, so if anything its an IntelliJ issue not Gradle.

My guess would be that IntelliJ would say that is what they need (multiple sources), and Gradle will say they need one source, so incompatible answers. I'd expect the large majority of Micronaut users to be using Gradle and IntelliJ in its default configuration so that could be the default environment to support (which would mean emitting 1 originating source), but obviously it would be better to give the 'correct' answer to all parties but that doesn't seem possible (unless Gradle change and that doesn't seem likely).

So some configuration to change the behaviour to the desired style is the most obvious workaround, something along the lines of what is already required for incremental annotation processing could be the answer?
'-Amicronaut.processing.incremental=true'
maybe another one
'-Amicronaut.processing.gradle_style_incremental=true'
or make Gradle the default and make IntelliJ opt-in
'-Amicronaut.processing.intellij_style_incremental=true'
🤷‍♂️

Happy to look at a PR for this if open to it? I might look at hack it in rolling back the change mentioned earlier to see if it fixes our test compile time.

[graemerocher]

Another alternative is to do some class loader trick to see if the compiler is running in Gradle or IntelliJ

[nhoughto]

Yep, is it the sort of thing you want to paper over though? Intellij / whoever will never fix it if everyone just solves it for them transparently

[graemerocher]

unfortunately that is life we have to often deal with different systems and sometimes it is easier to place a hack then fight for change in some system beyond your control

[graemerocher]

Submitted a PR for this https://github.com/micronaut-projects/micronaut-core/pull/5060

[nhoughto]

I believe this is causing an effect in micronaut-data annotation processors, after upgrading to 2.4.0 to test this out I see incremental compilation! 🎉 but strange failures generating Repository implementations. Hoping not the case! ill work on a reproducer.

Class dependency analysis for incremental compilation took 0.023 secs.
Compiling with JDK Java compiler API.
error: Unable to implement Repository method: GroupRepository.testLong arg0,Long arg1). No method parameter found for named Query parameter : groupId
error: Unable to implement Repository method: GroupRepository.testLong arg0,Long arg1). No method parameter found for named Query parameter : groupId
error: Unable to implement Repository method: GroupRepository.testLong arg0,Long arg1). No possible implementations found.

[nhoughto]

haven't been able to reduce it down to a sharable reproducer yet, but looks like the incremental processing of classes fail in micronaut-data because it loses the names of arguments, so then can't match the arguments to the query. So its still doing the compile but failing because of missing metadata.. so maybe not related to this after all 🤔

[graemerocher]

@nhoughto if you have a reproducer that would be great

[graemerocher]

note that maybe try upgrade to the Micronaut Gradle plugin 1.4.2

there was a bug where -parameters wasn't included as a flag by default for javac compilation

[nhoughto]

hmm not using mn gradle plugin unfortunately so i guess it can't be that, have to repro the hard way 😒

ill check in the javac logs if -parameter is being passed tho 🤷‍♂️

[nhoughto]

I was infact missing -parameters! although not because of the gradle plugin bug, i never even knew there was a gradle plugin, so might look into moving to that, i just didn't realise -parameters was required, as its not documented in the help docs? https://docs.micronaut.io/snapshot/guide/index.html#incrementalannotationgradle

working now either way 💪