Error-prone: OutOfMemoryError in FragmentNotInstantiable

[regrog]

What version of Error Prone are you using?

2.3.1

Does this issue reproduce with the latest release?

Yes

What did you do?

java 8
gradle 4.9
I was building a large multi-project with gradle 4.9 and this error appeared:

Please report this at https://github.com/google/error-prone/issues/new and include the following:
error-prone version: 2.3.1
     BugPattern: FragmentNotInstantiable
     Stack Trace:
     com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2216)
    at com.google.common.cache.LocalCache.get(LocalCache.java:4147)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)
    at com.google.errorprone.VisitorState.getTypeFromString(VisitorState.java:180)
    at com.google.errorprone.suppliers.Suppliers$5.get(Suppliers.java:118)
    at com.google.errorprone.suppliers.Suppliers$5.get(Suppliers.java:115)
    at com.google.errorprone.matchers.IsSubtypeOf.matches(IsSubtypeOf.java:40)
    at com.google.errorprone.matchers.Matchers$5.matches(Matchers.java:148)
    at com.google.errorprone.matchers.Matchers$4.matches(Matchers.java:129)
    at com.google.errorprone.bugpatterns.android.FragmentNotInstantiable.matchClass(FragmentNotInstantiable.java:97)
    at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:581)
    at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:145)
    at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:808)
    at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
    at com.google.errorprone.scanner.Scanner.scan(Scanner.java:82)
    at com.google.errorprone.scanner.Scanner.scan(Scanner.java:42)
    at com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
    at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
    at com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
    at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:602)
    at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:145)
    at com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:591)
    at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
    at com.google.errorprone.scanner.Scanner.scan(Scanner.java:64)
    at com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
    at com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:152)
    at com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:120)
    at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1404)
    at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1353)
    at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:952)
    at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:100)
    at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:142)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:96)
    at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:90)
    at com.google.errorprone.BaseErrorProneCompiler.run(BaseErrorProneCompiler.java:137)
    at com.google.errorprone.BaseErrorProneCompiler.run(BaseErrorProneCompiler.java:108)
    at com.google.errorprone.ErrorProneCompiler.run(ErrorProneCompiler.java:118)
    at com.google.errorprone.ErrorProneCompiler.compile(ErrorProneCompiler.java:65)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at net.ltgt.gradle.errorprone.ErrorProneCompiler.execute(ErrorProneCompiler.java:73)
    at net.ltgt.gradle.errorprone.ErrorProneCompiler.execute(ErrorProneCompiler.java:25)
    at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.delegateAndHandleErrors(NormalizingJavaCompiler.java:100)
    at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:52)
    at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:38)
    at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:40)
    at org.gradle.api.internal.tasks.compile.incremental.IncrementalCompilerFactory$2.execute(IncrementalCompilerFactory.java:111)
    at org.gradle.api.internal.tasks.compile.incremental.IncrementalCompilerFactory$2.execute(IncrementalCompilerFactory.java:107)
    at org.gradle.api.internal.tasks.compile.incremental.SelectiveCompiler.execute(SelectiveCompiler.java:65)
    at org.gradle.api.internal.tasks.compile.incremental.SelectiveCompiler.execute(SelectiveCompiler.java:35)
    at org.gradle.api.internal.tasks.compile.incremental.IncrementalResultStoringDecorator.execute(IncrementalResultStoringDecorator.java:44)
    at org.gradle.api.internal.tasks.compile.incremental.IncrementalResultStoringDecorator.execute(IncrementalResultStoringDecorator.java:28)
    at org.gradle.api.tasks.compile.JavaCompile.performCompilation(JavaCompile.java:156)
    at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:124)
    at sun.reflect.GeneratedMethodAccessor308.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
    at org.gradle.api.internal.project.taskfactory.IncrementalTaskAction.doExecute(IncrementalTaskAction.java:50)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
    at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
    at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
    at org.gradle.api.internal.tasks.execution.SkipCachedTaskExecuter.execute(SkipCachedTaskExecuter.java:105)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
    at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
    at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:66)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
    at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
    at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
    at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
    at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
    at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
    at org.gradle.execution.taskgraph.LocalTaskInfoExecutor.execute(LocalTaskInfoExecutor.java:42)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:273)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:258)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:135)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.execute(DefaultTaskPlanExecutor.java:200)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.executeWithWork(DefaultTaskPlanExecutor.java:191)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
    at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
    at java.lang.Thread.run(Thread.java:748)
  Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

[cushon]

Does setting a larger heap size allow the compilation to succeed?

(See also: #558)

[regrog]

Yeah, my Gradle heap size is already more than 5g.
Unfortunately, I cant provide you my projects.

I reported this because maybe it could be useful for optimizations.
I have many checks that runs with gradle and they probably contribute in memory usage.

[tbroyer]

You could possibly try using the net.ltgt.errorprone-javacplugin instead (with additional configuration to make it work with JDK 8, see its docs (open the “Running with JDK 8, prepending Error Prone javac to the bootstrap classpath” collapsible panel)), which allows (requires for JDK 8) forking a compiler daemon (https://guides.gradle.org/performance/#compiler_daemon).

Note however that the way you configure the Error Prone options is different (to easily migrate, you can simply add them to options.errorprone.errorproneArgs instead of options.compilerArgs, without necessarily migrate to the full DSL)

[regrog]

I read net.ltgt.errorprone-javacplugin documentation but I can't understand what its purpose.
I already have this configuration:

tasks.withType(JavaCompile) {
    options.compilerArgs = [...]
    options.fork = true
}

What can I achieve using net.ltgt.errorprone-javacplugin? Why it should be "better" than the classic plugin?

[cushon]

Why it should be "better" than the classic plugin?

It uses the official javac plugin mechanism, so it should be easier to maintain and support. There aren't any differences in functionality or (as far as I now) in performance.

Yeah, my Gradle heap size is already more than 5g. Unfortunately, I cant provide you my projects. I reported this because maybe it could be useful for optimizations.

Okay, thanks. There may be stuff to optimize here, but it's hard to say without a repro. Do you have any interest in profiling your build and letting us know if you spot any issues with Error Prone? If not, there's not much we can do to investigate without more information.

[tbroyer]

I already have this configuration:

tasks.withType(JavaCompile) {
    options.compilerArgs = [...]
    options.fork = true
}

And does it really fork a compiler daemon? I don't remember the details but IIRC, by replacing the javaToolchain with its own, the gradle errorprone plugin actually disables forking.

Why it should be "better" than the classic plugin?

It uses the official javac plugin mechanism, so it should be easier to maintain and support.

Also, it uses the official Gradle public APIs (vs. internal APIs), and it will eventually actually become the next version of the net.ltgt.errorprone plugin.

[regrog]

Okay, thanks. There may be stuff to optimize here, but it's hard to say without a repro. Do you have any interest in profiling your build and letting us know if you spot any issues with Error Prone? If not, there's not much we can do to investigate without more information.

I'm sorry, I can't provide you a repro, they are not my property and it's impossible to recreate the same ecosystem.

And does it really fork a compiler daemon? I don't remember the details but IIRC, by replacing the javaToolchain with its own, the gradle errorprone plugin actually disables forking.

Good point! I will test this ASAP and provide you the result.
Thanks

[regrog]

It's damn true, the fork property is not considered at all.
I switched to the javac plugin, let's see what will happen in the coming days.
Now it's forking correctly

Thanks

[tbroyer]

Fwiw, I just published v0.5 of the net.ltgt.errorprone-javacplugin with built-in support for JDK 8 (among other things). This makes the switch from net.ltgt.errorprone v0.0._n_ even easier: https://github.com/tbroyer/gradle-errorprone-javacplugin-plugin/blob/26dcf11669a71c74085c59991b5bfcf8eb8065da/README.md#migration-from-netltgterrorprone

[regrog]

Great, the overall memory usage is decreased, and that's good thing.
I just updated to v0.5.
I only have 1 doubt about JDK 8 usage. Actually my build is running with java 8 and I don't have any JDK 9 installed. I thought that the new plugin were supposed to fail.

[tbroyer]

The new plugin will only fail with JDK 8 if you didn't configure an errorproneJavac dependency to com.google.errorprone:javac (or, with v0.4, if you didn't explicitly configure forking and -Xbootclasspath/p:); otherwise it is expected to fail (generally with a NoClassDefFoundError about JavacMessages, or a related NoSuchMethodError; see https://github.com/google/error-prone/issues/535#issuecomment-411965977)

See https://github.com/gwtproject/gwt-http/commit/c907fe76accf49df0e8919db761c3df072211482 for an example of a migration.

[regrog]

Ok, thanks for the answers.
With the last version, it's easier to configure and to migrate. Also the code is cleaner.
Now everything works fine!