Graal: okhttp parse error with GraalVM CE 19.0.0

Created on 23 Jul 2019  路  10Comments  路  Source: oracle/graal

This is a different okhttp issue from #1294. The native-image compiler fails to parse the OkHttpClient.Builder(). A sample that recreates the issue is available here. Here is the gist:

import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.ResponseBody
import java.io.IOException
import java.util.concurrent.TimeUnit

object Test {

    private fun call(request: Request): ResponseBody {
        val caller = OkHttpClient.Builder()
                .readTimeout(60, TimeUnit.SECONDS)
                .build()
        val response = caller.newCall(request).execute()
        if (!response.isSuccessful) {
            throw IOException("Unexpected Code: $response")
        }
        return response.body!!
    }

    private fun get(url: String): ResponseBody {
        val request = Request.Builder()
                .url(url)
                .build()
        return call(request)
    }

    @JvmStatic
    fun main(args: Array<String>) {
        println(get("https://icanhazip.com/").string())
    }
}

The sample will compile with the following JRE

java -version
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-20190420112649.buildslave.jdk8u-src-tar--b03)
OpenJDK GraalVM CE 19.0.0 (build 25.212-b03-jvmci-19-b01, mixed mode)

Running with java works:

java -jar ./target/okhttp-test-0.0.1-jar-with-dependencies.jar                                                        
xxx.xxx.xxx.xxx

Compiling with native-image fails:

native-image -cp ${PWD}/target/okhttp-test-0.0.1-jar-with-dependencies.jar\
    -H:Name=okhttp\
    -H:Class=com.delphix.okhttp.Test\
    -H:+ReportUnsupportedElementsAtRuntime\
    -H:ReflectionConfigurationFiles=${PWD}/config/reflect-config.json\
    -H:ResourceConfigurationFiles=${PWD}/config/resource-config.json\
    -H:+ReportExceptionStackTraces\
    --allow-incomplete-classpath\
    --enable-url-protocols=http\
    --initialize-at-build-time\
    --no-server

Stack trace output:

[okhttp:31741]    classlist:   4,585.04 ms
[okhttp:31741]        (cap):  10,440.34 ms
[okhttp:31741]        setup:  11,648.34 ms
[okhttp:31741]     analysis:   1,039.11 ms
Error: Error encountered while parsing okhttp3.OkHttpClient$Builder.<init>() 
Parsing context:
        parsing com.delphix.okhttp.Test.call(Test.kt:12)
        parsing com.delphix.okhttp.Test.get(Test.kt:26)
        parsing com.delphix.okhttp.Test.main(Test.kt:31)
        parsing com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:153)
        parsing com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)

com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing okhttp3.OkHttpClient$Builder.<init>() 
Parsing context:
        parsing com.delphix.okhttp.Test.call(Test.kt:12)
        parsing com.delphix.okhttp.Test.get(Test.kt:26)
        parsing com.delphix.okhttp.Test.main(Test.kt:31)
        parsing com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:153)
        parsing com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)

        at com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:138)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:323)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
        at com.oracle.graal.pointsto.flow.SpecialInvokeTypeFlow.onObservedUpdate(InvokeTypeFlow.java:421)
        at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:352)
        at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:394)
        at com.oracle.graal.pointsto.flow.SourceTypeFlowBase.update(SourceTypeFlowBase.java:121)
        at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:509)
        at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:171)
        at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: java.lang.NoClassDefFoundError: okhttp3/Authenticator$DefaultImpls
        at parsing okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.kt:435)
        at org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2621)
        at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:85)
        at org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3421)
        at org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3223)
        at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:944)
        at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:838)
        at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84)
        at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
        at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
        at org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
        at org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:214)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:333)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
        ... 13 more
Caused by: java.lang.NoClassDefFoundError: okhttp3/Authenticator$DefaultImpls
        at java.lang.Class.getEnclosingMethod0(Native Method)
        at java.lang.Class.getEnclosingMethodInfo(Class.java:1072)
        at java.lang.Class.isLocalOrAnonymousClass(Class.java:1462)
        at java.lang.Class.isLocalClass(Class.java:1422)
        at jdk.vm.ci.hotspot.HotSpotJDKReflection.isLocalClass(HotSpotJDKReflection.java:94)
        at jdk.vm.ci.hotspot.HotSpotResolvedObjectTypeImpl.isLocal(HotSpotResolvedObjectTypeImpl.java:923)
        at com.oracle.graal.pointsto.meta.AnalysisType.isLocal(AnalysisType.java:925)
        at com.oracle.svm.hosted.SVMHost.createHub(SVMHost.java:250)
        at com.oracle.svm.hosted.SVMHost.registerType(SVMHost.java:181)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.createType(AnalysisUniverse.java:263)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:204)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:181)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:75)
        at com.oracle.graal.pointsto.infrastructure.UniverseMetaAccess.lookupJavaType0(UniverseMetaAccess.java:91)
        at com.oracle.graal.pointsto.infrastructure.UniverseMetaAccess.lookupJavaType(UniverseMetaAccess.java:82)
        at com.oracle.graal.pointsto.meta.AnalysisMetaAccess.lookupJavaType(AnalysisMetaAccess.java:47)
        at com.oracle.graal.pointsto.meta.AnalysisMetaAccess.lookupJavaType(AnalysisMetaAccess.java:39)
        at com.oracle.graal.pointsto.infrastructure.UniverseMetaAccess.lookupJavaType(UniverseMetaAccess.java:71)
        at com.oracle.graal.pointsto.meta.AnalysisMetaAccess.lookupJavaType(AnalysisMetaAccess.java:61)
        at com.oracle.graal.pointsto.meta.AnalysisMetaAccess.lookupJavaType(AnalysisMetaAccess.java:39)
        at org.graalvm.compiler.core.common.type.StampFactory.forConstant(StampFactory.java:224)
        at org.graalvm.compiler.nodes.ConstantNode.forConstant(ConstantNode.java:181)
        at org.graalvm.compiler.nodes.ConstantNode.forConstant(ConstantNode.java:189)
        at org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.foldConstant(ConstantFoldUtil.java:62)
        at org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.foldConstant(ConstantFoldUtil.java:47)
        at org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:86)
        at com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:72)
        at org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:47)
        at com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.tryConstantFold(ConstantFoldLoadFieldPlugin.java:64)
        at com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.handleLoadStaticField(ConstantFoldLoadFieldPlugin.java:60)
        at org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4809)
        at org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4776)
        at org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5251)
        at org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3416)
        ... 24 more
Error: Image build request failed with exit status 1

It doesn't look to be a reflection issue. Does anyone have a clue why this is happening?

native-image
Source
picture mcred

Most helpful comment

Kotlin issue ticket https://youtrack.jetbrains.com/issue/KT-33097

picture yschimke on 1 Aug 2019
👍3

All 10 comments

I have a branch I'm poking at for okhttp, using the trivial built in curl command. Mostly working but relies on SunEC native lib. It also works because it hacks the OkHttp interface definitions to include default methods.

https://github.com/square/okhttp/pull/5294

May not help with this issue, but could help in other ways.

~Hmmm.... I'm trying to remember why I needed to add some methods to Dns and Authenticator get them to pass. Might be related...~

So on the PR, I force the DefaultImpls class to exist by implementing dummy methods on the interfaces. But this is possible when you are directly changing the okhttp codebase.

yschimke picture yschimke on 23 Jul 2019

Also if this is really blocking you, falling back to okhttp 3.14.2 should work, but you will need to revert some nice kotlin source changes. OkHttp 3.14 is java implemented, while OkHttp 4.0 is kotlin based.

yschimke picture yschimke on 24 Jul 2019

FWIW I also get the following warnings in output, specifically related to the classes that trigger DefaultImpls errors 

warning: unknown locality of class Lokhttp3/Dns$Companion$SYSTEM$1;, assuming class is not local. To remove the warning report an issue to the library or language author. The issue is caused by Lokhttp3/Dns$Companion$SYSTEM$1; which is not following the naming convention.
yschimke picture yschimke on 24 Jul 2019

I got OkHttp working for me end to end by

  • Using a overlay jar with hacked versions of the 4 kotlin implemented interfaces
  • Cutting out all ECC ciphers (falling back to much weaker ciphers) for now. This avoids SunEC or bouncycastle setup drama
yschimke picture yschimke on 24 Jul 2019

Thanks for all the input @yschimke! I was able to solve the issue by downgrading to 3.14.2. It only required some very minor changes. Pretty much just missing the companion objects. I will keep an eye on the 4.0 branch and if a fix gets pushed, I'll upgrade again. For future reference, what is causing this incompatibility with graal?

mcred picture mcred on 24 Jul 2019

Not sure. I assume kotlin and how it creates interfaces. When you decompile kotlin there is a lot of syntactic sugar to make modern features work down to JDK 1.6. So I think edge cases really.

yschimke picture yschimke on 24 Jul 2019

@mcred based on the stack trace I think it's issues related to "unknown locality of class" with Kotlin.

yschimke picture yschimke on 26 Jul 2019

Kotlin issue ticket https://youtrack.jetbrains.com/issue/KT-33097

yschimke picture yschimke on 1 Aug 2019
👍3

Fixed on okhttp using a workaround described on this ticket

mageddo picture mageddo on 12 Dec 2019
👍2

Thank you, @mageddo!

cstancu picture cstancu on 12 Dec 2019
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

ghost picture ghost  路  3Comments
gersonimo picture gersonimo  路  3Comments
igor-ramazanov picture igor-ramazanov  路  3Comments
ilopmar picture ilopmar  路  3Comments
helloguo picture helloguo  路  3Comments