Lombok: Java 9 Annotation Processing Broken

[gudenau]

I am using Lombok 1.16.18 on JDK 9 and IntelliJ Idea 2017.2.4.

When using Lombok's annotation processor it causes the compiler to throw an exception.
Warning:(8, 8) java: lombok.javac.apt.LombokProcessor could not be initialized. Lombok will not run during this compilation: java.lang.IllegalArgumentException: com.sun.tools.javac.api.ClientCodeWrapper$WrappedStandardJavaFileManager extends com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager implements javax.tools.StandardJavaFileManager at lombok.javac.apt.LombokFileObjects.getCompiler(LombokFileObjects.java:130) at lombok.javac.apt.InterceptingJavaFileManager.<init>(InterceptingJavaFileManager.java:40) at lombok.javac.apt.LombokProcessor.placePostCompileAndDontMakeForceRoundDummiesHook(LombokProcessor.java:164) at lombok.javac.apt.LombokProcessor.init(LombokProcessor.java:85) at lombok.core.AnnotationProcessor$JavacDescriptor.want(AnnotationProcessor.java:87) at lombok.core.AnnotationProcessor.init(AnnotationProcessor.java:140) at lombok.launch.AnnotationProcessorHider$AnnotationProcessor.init(AnnotationProcessor.java:69) at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment$ProcessorState.<init>(JavacProcessingEnvironment.java:675) at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment$DiscoveredProcessors$ProcessorStateIterator.next(JavacProcessingEnvironment.java:774) at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:869) at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.access$2200(JavacProcessingEnvironment.java:108) at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1206) at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1315) at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1246) at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:922) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:100) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:142) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:96) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:90) at org.jetbrains.jps.javac.JavacMain.compile(JavacMain.java:190) at org.jetbrains.jps.incremental.java.JavaBuilder.compileJava(JavaBuilder.java:473) at org.jetbrains.jps.incremental.java.JavaBuilder.compile(JavaBuilder.java:328) at org.jetbrains.jps.incremental.java.JavaBuilder.doBuild(JavaBuilder.java:255) at org.jetbrains.jps.incremental.java.JavaBuilder.build(JavaBuilder.java:208) at org.jetbrains.jps.incremental.IncProjectBuilder.runModuleLevelBuilders(IncProjectBuilder.java:1260) at org.jetbrains.jps.incremental.IncProjectBuilder.runBuildersForChunk(IncProjectBuilder.java:937) at org.jetbrains.jps.incremental.IncProjectBuilder.buildTargetsChunk(IncProjectBuilder.java:1009) at org.jetbrains.jps.incremental.IncProjectBuilder.buildChunkIfAffected(IncProjectBuilder.java:900) at org.jetbrains.jps.incremental.IncProjectBuilder.buildChunks(IncProjectBuilder.java:733) at org.jetbrains.jps.incremental.IncProjectBuilder.runBuild(IncProjectBuilder.java:385) at org.jetbrains.jps.incremental.IncProjectBuilder.build(IncProjectBuilder.java:192) at org.jetbrains.jps.cmdline.BuildRunner.runBuild(BuildRunner.java:138) at org.jetbrains.jps.cmdline.BuildSession.runBuild(BuildSession.java:295) at org.jetbrains.jps.cmdline.BuildSession.run(BuildSession.java:125) at org.jetbrains.jps.cmdline.BuildMain$MyMessageHandler.lambda$channelRead0$0(BuildMain.java:236) at org.jetbrains.jps.service.impl.SharedThreadPoolImpl.lambda$executeOnPooledThread$0(SharedThreadPoolImpl.java:42) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base/java.lang.Thread.run(Thread.java:844)

[gudenau]

Side note, I never realized how much I used Lombok until it broke. Most of my classes need changing to compile correctly without it!

[Mart-Bogdan]

Sadly it's using internal api, hope it would be fixed in few monthes, thankfully Java8 is still ok :-)

[wrprice]

I'm also getting what appears to be the same error, but using the command line and _not_ IntelliJ.

• Lombok edge 1.16.19 (2017-07-17 20:26:39 UTC)
• JDK 9 GA Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
• Gradle 4.2 2017-09-20 14:48:23 UTC
• Win10 x64

I've opened the relevant modules/packages, and even tried --illegal-access=permit

warning: lombok.javac.apt.LombokProcessor could not be initialized. Lombok will not run during this compilation: java.lang.IllegalArgumentException: com.sun.tools.javac.api.ClientCodeWrapper$WrappedStandardJavaFileManager extends com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager implements javax.tools.StandardJavaFileManager
        at lombok.javac.apt.LombokFileObjects.getCompiler(LombokFileObjects.java:143)
        at lombok.javac.apt.InterceptingJavaFileManager.<init>(InterceptingJavaFileManager.java:40)
        at lombok.javac.apt.LombokProcessor.placePostCompileAndDontMakeForceRoundDummiesHook(LombokProcessor.java:164)
        at lombok.javac.apt.LombokProcessor.init(LombokProcessor.java:85)

I pulled down the current HEAD of the jdk9 branch and did some println debugging. In LombokFileObjects.java:115 there's an if branch checking class names for "c.s.t.j.api.ClientCodeWrapper$WrappedStandardJavaFileManager". We go into that branch, and the code expects the wrappedManager that it accessed via reflection to be an instance of com.sun.tools.javac.file.BaseFileManager.

On my machine, it turns out to be something different. Here's my debug output:

Wanted com.sun.tools.javac.file.BaseFileManager, but is: com.sun.proxy.$Proxy7
        super: java.lang.reflect.Proxy
        implements: [interface javax.tools.StandardJavaFileManager]

[wrprice]

I've got a bit of a hack workaround locally. Working on something I could submit as a starting point. (Need to go through a contribution process w/ my employer since it's on work time.)

This should be referenced from https://github.com/rzwitserloot/lombok/issues/985

[tmurakam]

I've created a patch to fix this issue.
The wrappedManager seems to be created with wrapWithCallDispatcher() of https://github.com/JetBrains/intellij-community/blob/master/jps/jps-builders-6/src/org/jetbrains/jps/javac/JavacMain.java
and it does not extends BaseFileManager. So it should be wrapped with another class extends BaseFileManager.

[gudenau]

Any news on when this will be on Maven?

[rspilker]

We've created an edge release. Can someone verify that this version fixes the problem?

[rspilker]

@gudenau We will make a new release after we get feedback on the edge release.

[krzyk]

Does it exist in any maven repo, or should I install it manually?

[krzyk]

@rspilker I can confirm that with edge release in IntelliJ IDEA 2013.3 EAP (Build #IU-173.3302.5, built on October 17, 2017) lombok annotations work fine.

[rspilker]

Thanks

[thetric]

The edge release unfortunately doesn't fix the Gradle build issue. The build still fails with the original stacktrace. (Built with Gradle 4.2.1, Java 9.0.1)

Edit: Seems to be my fault. In a fresh project the edge release works with Gradle + Jdk 9

[lpandzic]

@rspilker when can we expect a release? Do you need more verification?

[wrprice]

+1 the new edge release works with Gradle 4.2.1

[gudenau]

Sorry, I had other things to do.

How does one access the edge release?

[krzyk]

There is a link on lombok download page: https://projectlombok.org/download-edge

[maystrovyy]

@krzyk supra, thank you! What about maven repo?

[gudenau]

Interesting, I am getting an error regarding a javax error from modules not being visible.

Error:(17, 1) java: package javax.annotation is not visible
  (package javax.annotation is declared in module java.xml.ws.annotation, which is not in the module graph)

And the line with the error is

@ToString

on a class definition.

[thetric]

To fix this add this (assuming you're using gradle)

compileOnly 'javax.annotation:javax.annotation-api:1.3.1'

Or if you're using no build system: download this jar and add it to the buildpath

[gudenau]

I am using IntelliJ Idea, without a build system.

[rspilker]

To get rid of the javax.annotation problem, you could also create a lombok.config file, or download this one.

[gudenau]

So the latest edge build does work for me.

[rspilker]

@gudenau Can you tell me what version of IntelliJ you've used, so I can update the compatibility notes?

[lpandzic]

I've tested it and it works with IntelliJ 2017.2.

[gudenau]

IntelliJ Ultimate 2017.2.5.

[kamilgregorczyk]

When can we expect it to be in maven?

[wrprice]

@rspilker - My organization is eagerly waiting for 1.16.20 since we need the fixes in the current edge release but our policy prohibits depending upon what is effectively a snapshot. Since .18 is now four months old, is there anything else we can do to help unblock the release of .20?

[sskorol]

@wrprice could you please share your build.gradle configuration, which works for you with edge version?

[wrprice]

@sskorol - I can't share my entire project config, but these are the things I changed when I was initially trying to get it working:

dependencies {
    compileOnly files("path/to/lombok-edge.jar")
}

tasks.withType(JavaCompile) { // see note below
    options.fork = true
    options.forkOptions.jvmArgs += [
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED',
          '--add-opens', 'jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED'
        ]
}

As a test, I just commented out the options.fork and options.forkOptions.jvmArgs and my build is still working on JDK 9 GA and Gradle 4.2.1 using the newer java-library plugin instead of the older java plugin. (shrug) So, with that combination of software, it seems to be simply a matter of applying the new Lombok jar.

[krzyk]

@rspilker Are more tests needed to confirm the fix for IntelliJ? Or there are other issues that are targeted for .20 release? We are eagerly awaiting new release so we can use Java 9 in projects with lombok.

[rspilker]

We're still finalizing jdk9 support.

[slnowak]

@rspilker can we expect it this year?

[aalmazzo]

+1 please support java9

[jayantbit]

+1
Lombok edge jar is working perfectly with gradle 4.2.1.
Please release it to maven central.

[rspilker]

1.16.20 has just been uploaded to our website and should be available on maven central soon.

[gudenau]

Cheers!

[lpandzic]

@rspilker, why was out of the box support for ConstructorProperties removed? It was working fine on edge release.

[krzyk]

I think lack of ConstructorProperties breaks behavior with deserialization with jackson. Is there a switch to bring it back?

[lpandzic]

Yes, this is exactly the issue I encountered, single parameter constructor doesn't work without an annotation.

[krzyk]

@lpandzic I found in the changelog that now we need to use lombok.anyConstructor.addConstructorProperties = true to have it work (and probably add java.desktop module)

[rspilker]

The reason it was removed by default is that it would break jdk9 compilation if you didn't specify the dependency on the extra module.

[randakar]

Damn. Would be nice to have that documented somewhere. I want to upgrade our projects to JDK 9 but gotcha's like that (we use Jackson, ofc) would make it more difficult. A small migration guide would help.

[krzyk]

@randakar https://projectlombok.org/changelog is not enough?

[lpandzic]

Well, for jackson, this was a hack anyway, we need real support like I've requested in this issue - https://github.com/FasterXML/jackson-databind/issues/1498.
Thanks @rspilker .

[randakar]

@krzyk I guess you do have a point there. I was thinking along the lines of documenting what happens if you upgrade an existing JDK 8 / jackson project, but if it's just this one single thing I guess the changelog will have to do.

[Arpit0492]

This worked for me https://github.com/mplushnikov/lombok-intellij-plugin/issues/415

[eximius313]

This comment could be related with this bug