Micronaut-core: Incremental Compilation Does Not Work When Using Source Retention Annotations

[jbushdiecker]

Task List

• [x] Steps to reproduce provided
• [x] Stacktrace (if present) provided
• [x] Example that reproduces the problem uploaded to Github
• [x] Full description of the issue provided (see below)

Steps to Reproduce

1. On a Gradle project with micronaut-inject-java as an annotationProcessor, assemble the project
2. Make a simple change like changing a string inside a method body
3. Use gradle assemble --info to recompile

Expected Behaviour

Gradle should use incremental compilation when rebuilding the project

Expected Gradle Output

Task ':compileJava' is not up-to-date because:
Input property 'stableSources' file C:\Users\jackson\IdeaProjects\annotationTest\src\main\java\MyClass.java has changed.
Created classpath snapshot for incremental compilation in 0.0 secs.
Class dependency analysis for incremental compilation took 0.001 secs.
Compiling with JDK Java compiler API.
Incremental compilation of 1 classes completed in 0.1 secs.
:compileJava (Thread[Execution worker for ':',5,main]) completed. Took 0.119 secs.
:processResources (Thread[Execution worker for ':',5,main]) started.

Actual Behaviour

Gradle does a full recompile, citing that @Override has source retention. If all source retention annotations are removed from user code, the incremental compilation does work. This behavior does not happen if Micronaut is removed from the project, even when using other agregating annotation processors.

Actual Gradle output:

Task ':compileJava' is not up-to-date because:
Input property 'stableSources' file C:\Users\jackson\IdeaProjects\annotationTest\src\main\java\MyClass.java has changed.
Created classpath snapshot for incremental compilation in 0.0 secs.
Class dependency analysis for incremental compilation took 0.0 secs.
Full recompilation is required because '@Override' has source retention. Aggregating annotation processors require class or runtime retention. Analysis took 0.001 secs.
Compiling with JDK Java compiler API.
:compileJava (Thread[Execution worker for ':',5,main]) completed. Took 0.259 secs.
:processResources (Thread[Execution worker for ':',5,main]) started.

Environment Information

• Operating System: Windows 10 66-bit
• Micronaut Version: 1.2.10
• JDK Version: 1.8.0_201

Example Application

https://github.com/jbushdiecker/incrementalAnnotationTesting

[graemerocher]

We will have to explore what can be done here.

@melix @oehme Do you know what could be the cause of this?

[oehme]

That's because micronaut registers itself for *, which captures every annotation, including @Override (which has source retention) and many others you probably don't care about. Can you provide a more specific set of processed annotations?

[graemerocher]

No we can’t because Micronaut has the ability to map existing potentially user defined annotations. However we have a more limited interest in source level annotations. Is there any way around this?

[oehme]

The only efficient idea that comes to my mind is allowing users to pass an explicit processor argument where they list the annotations to include and then returning those from getSupportedAnnotationTypes instead of returning *.

The other alternative would be to change Gradle to recompile the affected source files from scratch in this case instead of trying to only reprocess the class files (which don't have the source retention annotations). But since almost every source file has some annotations, this would come down to mostly the same thing as a full recompile.

[graemerocher]

@oehme let me explore that option, maybe we can make that extensible.

One thing to note is that I don't think that Micronaut falls within the traditional annotation processor models exposed by Gradle in that most annotation processors generate source code.

Micronaut doesn't generate source code it generates additional byte code that sits alongside the compiled classes. So for example for a given class:

package example;
@Singleton
class MyClass {}

You end up with the following classes in build/classes/java:

example/MyClass.class
example/$MyClassDefinition.class
example/$MyClassDefinitionClass.class

So given we generate byte code the only case where maybe a full recompile is needed is if one deletes or renames a class during a refactoring?

Does it make sense for Gradle to expose another annotation processor type that fits Micronaut's model? What do you think?

[oehme]

It doesn't matter whether you generate source code or byte code, both follow the same pattern. The problem is with the annotated source elements, not with what you generate. An aggregating processor merges multiple sources into one output (e.g. creating a registry of sorts). This means that on any change we need to provide it with all annotated elements again, so that it sees the full picture. To make this a little more efficient, we only reprocess the class files instead of recompiling the sources of the unchanged files. But that doesn't work with source retention annotations.

[graemerocher]

Gotcha. So I think it may be doable to use getSupportedAnnotationTypes and allow that to be user extensible. Only issue is it is a breaking change since downstream projects that process annotations will have to be updated so it may be we cannot do this until Micronaut 2.0

[oehme]

It doesn't have to be breaking - You can default to * if the user doesn't specify otherwise.

[graemerocher]

That is true. Actually I think there is a way to do this automatically, we would need to figure out from the annotation processor classpath all the different AnnotationMapper and TypeElementVisitor present and use only the types that those handle in getSupportedAnnotationTypes.

However to be 100% sure this is not a breaking change we would probably need to supply a fall back to handle *. Thanks for the feedback.

[graemerocher]

So I have looked into this further and we will definitely have to default to * and make it a user configurable feature. The issue is that much of the Micronaut API is based on meta-annotations and annotation composition.

So for example to define a new AOP advice you would annotate another annotation with @Around:

@Around
@Type(MyInterceptor.class)
@Retention(RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface SomeAdvice {
    String value();
}

So whilst we can register @Around as an annotation we handle, our processor is not triggered when another annotation (in this case @SomeAdvice) is annotated with @Around.

Meta annotations are used everywhere to define bean scopes, AOP etc. and we cannot know ahead of time what annotations the user is going to come up with.

The best we are going to be able to do is default to * and then provide an option for the user to enable incremental annotation processing and another option to supply additional custom annotation names that are user defined.

[graemerocher]

On slightly brighter news it seems the annotation processor API allows returning patterns. So we could allow something like:

-Dmicronaut.incremental.processing=true -Dmicronaut.incremental.annotations=com.foo.*

Which from a usability perspective is better and we can also default to include io.micronaut.* which covers most of Micronaut subprojects.

[graemerocher]

PR https://github.com/micronaut-projects/micronaut-core/pull/2721 provides a potential fix. Example config:

tasks.withType(JavaCompile) {
    options.compilerArgs = [
        '-Amicronaut.processing.incremental=true',
        '-Amicronaut.processing.annotations=com.foo.*',
    ]
}