Graal: CL.exe compilation errors not returned

[osmanrafiqss]

When native-image compilation fails on Windows the error included is not very informative.

Consider the following example:

[getting-started-1.0-SNAPSHOT-runner:12224] classlist: 8,740.59 ms [getting-started-1.0-SNAPSHOT-runner:12224] (cap): 1,634.56 ms [getting-started-1.0-SNAPSHOT-runner:12224] setup: 2,111.65 ms Error: Error compiling query code (in C:\tmp\SVM-2783671696419126403\NativeInfoDirectives.cpp). Compiler command CL C:\tmp\SVM-2783671696419126403\NativeInfoDirectives.cpp /FeC:\tmp\SVM-2783671696419126403\NativeInfoDirectives.exe output included error: [Microsoft (R) C/C++ Optimizing Compiler Version 16.00.40219.01 for x64, Copyright (C) Microsoft Corporation. All rights reserved., ] com.oracle.svm.core.util.UserError$UserException: Error compiling query code (in C:\tmp\SVM-2783671696419126403\NativeInfoDirectives.cpp). Compiler command CL C:\tmp\SVM-2783671696419126403\NativeInfoDirectives.cpp /FeC:\tmp\SVM-2783671696419126403\NativeInfoDirectives.exe output included error: [Microsoft (R) C/C++ Optimizing Compiler Version 16.00.40219.01 for x64, Copyright (C) Microsoft Corporation. All rights reserved., ] at com.oracle.svm.core.util.UserError.abort(UserError.java:114)

I am by no means a Microsoft Compiler expert - but it seems that CL.exe writes compilation errors to stdout and compiler errors to stderr.

If I update substratevm to capture stdout when CL.exe exits with a non-zero error code - I get the following more informative error:

[getting-started-1.0-SNAPSHOT-runner:8848] classlist: 9,326.16 ms [getting-started-1.0-SNAPSHOT-runner:8848] (cap): 2,198.22 ms [getting-started-1.0-SNAPSHOT-runner:8848] setup: 2,802.02 ms Error: Error compiling query code (in C:\tmp\SVM-1270989077554696147\NativeInfoDirectives.cpp). Compiler command CL C:\tmp\SVM-1270989077554696147\NativeInfoDirectives.cpp /FeC:\tmp\SVM-1270989077554696147\NativeInfoDirectives.exe output included error: [NativeInfoDirectives.cpp, C:\tmp\SVM-1270989077554696147\NativeInfoDirectives.cpp(32) : fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory] com.oracle.svm.core.util.UserError$UserException: Error compiling query code (in C:\tmp\SVM-1270989077554696147\NativeInfoDirectives.cpp). Compiler command CL C:\tmp\SVM-1270989077554696147\NativeInfoDirectives.cpp /FeC:\tmp\SVM-1270989077554696147\NativeInfoDirectives.exe output included error: [NativeInfoDirectives.cpp, C:\tmp\SVM-1270989077554696147\NativeInfoDirectives.cpp(32) : fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory]

I have spent a few days figuring out why native-image compilation when using Quarkus was failing on my setup - having a more informative error message would really be helpful.

[olpaw]

@dougxc added some improvements in error handling of native-compilation issues. Can you still reproduce this problem?

[famod]

Seeing the same not very helpful error message with graalvm-ce-java11-19.3.1:

[modmono-quarkus-1.0-SNAPSHOT-runner:17584]    classlist:   3,943.13 ms
[modmono-quarkus-1.0-SNAPSHOT-runner:17584]        (cap):     798.64 ms
[modmono-quarkus-1.0-SNAPSHOT-runner:17584]        setup:   1,134.50 ms
Error: Error compiling query code (in C:\Develop\Temp\SVM-5228049311242870963\NativeInfoDirectives.cpp). Compiler command  CL C:\Develop\Temp\SVM-5228049311242870963\NativeInfoDirectives.cpp /FeC:\Develop\Temp\SVM-5228049311242870963\NativeInfoDirectives.exe output included error: [Microsoft (R) C/C++-Optimierungscompiler Version 19.16.27035 f?r x64, Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten., ]
com.oracle.svm.core.util.UserError$UserException: Error compiling query code (in C:\Develop\Temp\SVM-5228049311242870963\NativeInfoDirectives.cpp). Compiler command  CL C:\Develop\Temp\SVM-5228049311242870963\NativeInfoDirectives.cpp /FeC:\Develop\Temp\SVM-5228049311242870963\NativeInfoDirectives.exe output included error: [Microsoft (R) C/C++-Optimierungscompiler Version 19.16.27035 f?r x64, Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten., ]
        at com.oracle.svm.core.util.UserError.abort(UserError.java:114)
        at com.oracle.svm.hosted.c.NativeLibraries.reportErrors(NativeLibraries.java:218)
        at com.oracle.svm.hosted.NativeImageGenerator.processNativeLibraryImports(NativeImageGenerator.java:1518)
        at com.oracle.svm.hosted.NativeImageGenerator.setupNativeLibraries(NativeImageGenerator.java:1006)
        at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:835)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:528)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:445)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Error: Image build request failed with exit status 1

Invocation:

C:\Develop\_tools\graal-default\bin\native-image.cmd "-J-Dsun.nio.ch.maxUpdateArraySize=100" "-J-Djava.util.logging.manager=org.jboss.logmanager.LogManager" "-J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory" "-J-Dvertx.disableDnsResolver=true" "-J-Dio.netty.leakDetection.level=DISABLED" "-J-Dio.netty.allocator.maxOrder=1" "--initialize-at-build-time=" "-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime" -jar modmono-quarkus-1.0-SNAPSHOT-runner.jar "-H:FallbackThreshold=0" -H:+ReportExceptionStackTraces -H:-AddAllCharsets "-H:EnableURLProtocols=http" -H:+JNI -H:-UseServiceLoaderFeature -H:+StackTrace modmono-quarkus-1.0-SNAPSHOT-runner

[olpaw]

Seeing the same not very helpful error message with graalvm-ce-java11-19.3.1

Hi @famod I recently came across the query code compilation bits and fixed that on master (d537daf8633d5353e71fc0c76e9aebd07bed7a3a).
This change will also be in the 20.0.0 release.

[famod]

Hi @olpaw, thanks for letting me know.
I suppose I don't have any chance to get more info with 19.3.1?

[olpaw]

any chance to get more info with 19.3.1?

It should also be possible by using -H:TempDirectory=myknownpath. This will prevent temporary files from getting deleted. That way you are able to replay the linker command on the Windows command prompt. (Copy the linker command from the error message (Compiler command CL .....) and paste it into the windows command prompt window)

[famod]

Thanks @olpaw! This revealed that I am having the same problem as @osmanrafiqss (unistd.h is missing).