Error-prone: Infinite compilation on JDK11

[ben-manes]

When upgrading to JDK 11.01, the compilation never finishes but has high CPU activity. When removing ErrorProne, the compilation completes without a problem. I am unsure how to debug, e.g. to see what the javac process is doing, to provide more details.

Environment:

• JDK 11.01 (upgrading from 10+46)
• Mac OS X Mojave
• Gradle 5.0
• ErrorProne:
  
  
  
  • core: 2.3.2
  
  
  • javac: 9+181-r4173-1
  
  
  • plugin: 0.6
  
  
• Annotation Processors:
  
  
  
  • autoFactory: 1.0-beta6
  
  
  • autoValue: 1.6.3
  
  
  • autoValueBuilder: 2.9.1
  
  

Of course if I disable the annotation processors then it fails due to the missing generated classes, but does terminate. Removing autoFactory leads to the hang, while autoValue is used too frequently to get a hang prior. So there is a chance that it is related to annotation processing (such as due to Gradle's incremental support), but could be something else.

[Stephan202]

@ben-manes does running jstack <pid> give any clue?

[ben-manes]

Unfortunately not much that I can debug from, but maybe others can.

Running with -d --no-daemon I get

14:12:03.701 [INFO] [org.gradle.api.internal.tasks.compile.JdkJavaCompiler] Compiling with JDK Java compiler API.

and the sections that look interesting are,

"C2 CompilerThread0" #5 daemon prio=9 os_prio=31 cpu=29238.27ms elapsed=211.65s tid=0x00007f87b683a800 nid=0x3f03 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   Compiling: 18724       4       com.squareup.javapoet.CodeWriter::emit (781 bytes)

Execution worker for ':'" #37 prio=5 os_prio=31 cpu=169492.65ms elapsed=204.29s tid=0x00007f87b630e800 nid=0x14d03 runnable  [0x000070000b233000]
   java.lang.Thread.State: RUNNABLE
    at java.nio.file.Files.provider([email protected]/Files.java:100)
    at java.nio.file.Files.isDirectory([email protected]/Files.java:2228)
    at com.sun.tools.javac.file.PathFileObject.<init>([email protected]/PathFileObject.java:352)
    at com.sun.tools.javac.file.PathFileObject$DirectoryFileObject.<init>([email protected]/PathFileObject.java:111)
    at com.sun.tools.javac.file.PathFileObject.forDirectoryPath([email protected]/PathFileObject.java:102)
    at com.sun.tools.javac.file.JavacFileManager.getFileForOutput([email protected]/JavacFileManager.java:872)
    at com.sun.tools.javac.file.JavacFileManager.getJavaFileForOutput([email protected]/JavacFileManager.java:816)
    at jdk.internal.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke([email protected]/Method.java:566)
    at org.gradle.api.internal.tasks.compile.reflect.SourcepathIgnoringInvocationHandler.invoke(SourcepathIgnoringInvocationHandler.java:64)
    at com.sun.proxy.$Proxy131.getJavaFileForOutput(Unknown Source)
    at com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager.getJavaFileForOutput([email protected]/ClientCodeWrapper.java:298)
    at com.sun.tools.javac.processing.JavacFiler.createSourceOrClassFile([email protected]/JavacFiler.java:500)
    at com.sun.tools.javac.processing.JavacFiler.createSourceFile([email protected]/JavacFiler.java:426)
    at org.gradle.api.internal.tasks.compile.processing.IncrementalFiler.createSourceFile(IncrementalFiler.java:49)
    at net.autobuilder.core.AutoBuilderProcessor.write(AutoBuilderProcessor.java:176)
    at net.autobuilder.core.AutoBuilderProcessor.processTask(AutoBuilderProcessor.java:159)
    at net.autobuilder.core.AutoBuilderProcessor.doProcess(AutoBuilderProcessor.java:131)
    at net.autobuilder.core.AutoBuilderProcessor.process(AutoBuilderProcessor.java:109)
    at org.gradle.api.internal.tasks.compile.processing.DelegatingProcessor.process(DelegatingProcessor.java:62)
    at org.gradle.api.internal.tasks.compile.processing.AggregatingProcessor.process(AggregatingProcessor.java:50)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor([email protected]/JavacProcessingEnvironment.java:972)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs([email protected]/JavacProcessingEnvironment.java:888)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run([email protected]/JavacProcessingEnvironment.java:1214)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing([email protected]/JavacProcessingEnvironment.java:1326)
    at com.sun.tools.javac.main.JavaCompiler.processAnnotations([email protected]/JavaCompiler.java:1258)
    at com.sun.tools.javac.main.JavaCompiler.compile([email protected]/JavaCompiler.java:936)
    at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0([email protected]/JavacTaskImpl.java:104)
    at com.sun.tools.javac.api.JavacTaskImpl$$Lambda$200/0x0000000800c50c40.call([email protected]/Unknown Source)
    at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions([email protected]/JavacTaskImpl.java:147)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall([email protected]/JavacTaskImpl.java:100)
    at com.sun.tools.javac.api.JavacTaskImpl.call([email protected]/JavacTaskImpl.java:94)
    at org.gradle.api.internal.tasks.compile.AnnotationProcessingCompileTask.call(AnnotationProcessingCompileTask.java:89)
    at org.gradle.api.internal.tasks.compile.ResourceCleaningCompilationTask.call(ResourceCleaningCompilationTask.java:57)
    at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:50)

I know that the javac is doing work as sampling the compilation thread shows updates to the stack, e.g.

"Execution worker for ':'" #37 prio=5 os_prio=31 cpu=549949.22ms elapsed=595.96s tid=0x00007f87b630e800 nid=0x14d03 runnable  [0x000070000b234000]
   java.lang.Thread.State: RUNNABLE
    at java.lang.Iterable.forEach([email protected]/Iterable.java:75)
    at com.sun.tools.javac.code.DeferredCompletionFailureHandler$1.uninstall([email protected]/DeferredCompletionFailureHandler.java:82)
    at com.sun.tools.javac.code.DeferredCompletionFailureHandler.setHandler([email protected]/DeferredCompletionFailureHandler.java:106)
    at com.sun.tools.javac.api.MultiTaskListener.started([email protected]/MultiTaskListener.java:121)
    at com.sun.tools.javac.main.JavaCompiler.enterTrees([email protected]/JavaCompiler.java:1062)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.enterTrees([email protected]/JavacProcessingEnvironment.java:1194)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.<init>([email protected]/JavacProcessingEnvironment.java:1080)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.next([email protected]/JavacProcessingEnvironment.java:1100)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing([email protected]/JavacProcessingEnvironment.java:1337)
    at com.sun.tools.javac.main.JavaCompiler.processAnnotations([email protected]/JavaCompiler.java:1258)
    at com.sun.tools.javac.main.JavaCompiler.compile([email protected]/JavaCompiler.java:936)
    at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0([email protected]/JavacTaskImpl.java:104)
    at com.sun.tools.javac.api.JavacTaskImpl$$Lambda$200/0x0000000800c50c40.call([email protected]/Unknown Source)
    at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions([email protected]/JavacTaskImpl.java:147)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall([email protected]/JavacTaskImpl.java:100)
    at com.sun.tools.javac.api.JavacTaskImpl.call([email protected]/JavacTaskImpl.java:94)
    at org.gradle.api.internal.tasks.compile.AnnotationProcessingCompileTask.call(AnnotationProcessingCompileTask.java:89)
    at org.gradle.api.internal.tasks.compile.ResourceCleaningCompilationTask.call(ResourceCleaningCompilationTask.java:57)
    at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:50)

[Stephan202]

Hmm, I had "hoped" that maybe compilesWithFix would stand out, but that's not the case. (See #1166; the fix for that issue is not yet released.)

Another idea to help narrow down the cause: you could see whether the issue is intrinsically caused by Error Prone, or rather some interaction with one of the checks, by compiling with -XepDisableAllChecks.

[cushon]

at com.sun.tools.javac.code.DeferredCompletionFailureHandler$1.uninstall([email protected]/DeferredCompletionFailureHandler.java:82)

Maybe this is JDK-8209055?

Are you using -XDcompilePolicy=simple? If so, does removing that flag make a difference?

[ben-manes]

hmm, good idea on disabling all checks. I'll try that now.

I am not using anything fancy in this project, except for minor workarounds for module/jigsaw compatibility issues.

tasks.withType(JavaCompile) {
  options.encoding = 'UTF-8'
  options.compilerArgs += [
    '-Xlint:all',
    '-Xlint:-processing',
    '--add-modules', 'java.compiler',
  ]
  options.errorprone {
    disableWarningsInGeneratedCode = true
    excludedPaths = "${buildDir}/generated-sources/.*"
  }

  afterEvaluate {
    configurations.patchJsr305.each {
      options.compilerArgs += [ '--patch-module', "java.xml.ws.annotation=${it}" ]
    }
  }
}

[ben-manes]

No luck with disabling all checks. I also tried updating to 2.3.3-SNAPSHOT which also hangs.

[Stephan202]

Based on the posted config it seems you're not compiling with -XDcompilePolicy=simple. Maybe add that? (I.e. the inverse of @cushon's suggestion; admittedly I don't know whether this suggestion makes sense, but the documentation _does_ recommend using this flag.)

[cushon]

I'm still suspicious of JDK-8209055.

I think -XDcompilePolicy=simple is baked in to the gradle plugin:
https://github.com/tbroyer/gradle-errorprone-plugin/blob/21ac96affce535420cf8266e8095a16047eb3e82/src/main/kotlin/net/ltgt/gradle/errorprone/ErrorPronePlugin.kt#L172

Error Prone is supposed to throw an exception if that flag isn't set: https://github.com/google/error-prone/blob/20bf092117ec70e14dfdd125d49f36c5a3f5bc0c/check_api/src/main/java/com/google/errorprone/BaseErrorProneJavaCompiler.java#L164

[ben-manes]

I did try adding -XDcompilePolicy=simple which hung, though I see now you're right it's probably set implicitly. I tried adding doFirst { println options.compilerArgs } and didn't see it, and would have to dig in further to see if it can be manipulated / exposed somehow

> Task :compileJava
[-Xlint:all, -Xlint:-processing, --add-modules, java.compiler, --patch-module, java.xml.ws.annotation=/Users/ben/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar]

[ben-manes]

I tried disabling error prone and adding -XDcompilePolicy=simple to options.compilerArgs and it did not hang, if that helps any.

[cushon]

I tried disabling error prone and adding -XDcompilePolicy=simple to options.compilerArgs and it did not hang, if that helps any.

That weakens my theory but I don't think it's conclusive, my understanding of the bug is that it might be exacerbated by running Error Prone.

Can you try running Error Prone and modifying the gradle plugin to use -XDcompilePolicy=byfile instead of simple?

Alternately, you could try a JDK 12 early access build, since it includes the fix for that bug.

[ben-manes]

Okay, I'll try JDK 12 since that's faster than making a custom build of the plugin.

P.S. Thanks for the help so far @Stephan202 & @cushon

[ben-manes]

Java 12 worked! 😅

[cushon]

@tbroyer is there a good way to customize -XDcompilePolicy=simple when using Error Prone with gradle? -XDcompilePolicy=byfile is also buggy, but it might be helpful to have a choice for some builds (at least until everyone is using JDK 12 :).

[ben-manes]

For now, I'm going to resolve this by disabling error prone only for JDK11 using:

options.errorprone.enabled = !JavaVersion.current().isJava11()

That way when I upgrade to 12, I don't forget to re-enable 😄

[tbroyer]

@tbroyer is there a good way to customize -XDcompilePolicy=simple when using Error Prone with gradle?

Not what I'd call a "good way", but it should be possible (using the Kotlin DSL, as my Groovy is rather bad):

tasks.withType<JavaCompile>().configureEach {
    // prevent net.ltgt.errorprone plugin from adding the compiler args
    options.errorprone.isEnabled = false
    // add the `-Xplugin:ErrorProne` argument with Error Prone options, but without the -XDcompilePolicy
    options.compilerArgumentProviders.add(MyProvider(options.errorprone))
}

class MyProvider(
    @Nested val errorproneOptions: ErrorProneOptions
) : CommandLineArgumentProvider {
    override fun asArguments()= listOf("-Xplugin:ErrorProne ${errorproneOptions}")
}

(fwiw, @ben-manes, you might want to use compilerArgumentProviders instead of your afterEvaluate for your --patch-module; declaring the configuration as a @Nested property in the provider's class; also btw, use options.allCompilerArgs to get all arguments, including those added by argument providers)

@cushon should I add a configurable property to the plugin to allow setting the -XDcompilePolicy? (or at a minimum toggling whether the plugin adds it, so one could add its own) or it's not necessary as this is fixed already? (and should be backported to javac 11 at some point)

[jbrinegar]

I am hitting this issue as well, on:

• Java 11.0.2
• Linux
• Gradle 5.1.1
• ErrorProne 2.3.2
• Plugin: 0.6.1

Switching to -XDcompilePolicy=byfile did not help (I did so by modifying the gradle plugin & installing locally). Disabling all checks still reproduces the issue.

[cushon]

@jbrinegar can you provide a repro?

[jbrinegar]

@cushon unfortunately, I cannot provide a reproducing test case. I did try a handful of open source projects on 11 + errorprone, but it does look like this is related to annotation processors. For now, I am working around this by just not allowing errorprone to run on the impacted module.

I did confirm that JDK-8209055, when applied to jdk11u, does fix the problem entirely. It should be nominated for backporting.

[shevek]

Related: https://issues.apache.org/jira/browse/NETBEANS-2291 - also presumably putting hooks into JDK. Something's leaking a hueg list of something?

[ben-manes]

Closing since JDK14 is supported and the underlying bug was resolved.