Graal: `mx build` for graal/compiler fails on Windows

Created on 26 Nov 2019  ·  25Comments  ·  Source: oracle/graal
  1. Windows SDK 7.1 Command Prompt
  2. c:\Progra~1\Git\git-bash.exe
  3. export MKS_HOME=/c/Progra~1/Git/usr/bin
  4. export PATH=$MKS_HOME:${PATH}
  5. mkdir graal
  6. cd graal
  7. git clone https://github.com/graalvm/mx.git
  8. export PATH=$PWD/mx:$PATH
  9. export JAVA_HOME=/c/openjdk1.8.0_232-jvmci-19.3-b05
  10. export EXTRA_JAVA_HOMES=/c/Progra~1/RedHat/java-11-openjdk-11.0.3-1
  11. git clone https://github.com/graalvm/graal.git
  12. cd graal/compiler
  13. mx build
    ...Error Output...
    output.txt
    :

Compiling com.oracle.truffle.api.instrumentation.test with javac-daemon(JDK 1.8)... [dependency TRUFFLE_DSL_PROCESSOR update d] Archiving TRUFFLE_INSTRUMENT_TEST... [dependency com.oracle.truffle.api.instrumentation.test updated] Traceback (most recent call last): File "C:/src/graal/mx/mx.py", line 19292, in main() File "C:/src/graal/mx/mx.py", line 19273, in main retcode = c(command_args) File "C:\src\graal\mx\mx_commands.py", line 147, in __call__ return self.command_function(*args, **kwargs) File "C:/src/graal/mx/mx.py", line 13565, in build t.execute() File "C:/src/graal/mx/mx.py", line 4660, in execute _built = self.build() File "C:/src/graal/mx/mx.py", line 5022, in build self.subject.make_archive(getattr(self, 'javac_daemon', None)) File "C:\src\graal\mx\mx_jardistribution.py", line 533, in make_archive addSrcFromDir(srcDir, archivePrefix, arcnameCheck=overlay_check) File "C:\src\graal\mx\mx_jardistribution.py", line 414, in addSrcFromDir contents = fp.read() File "C:\Program Files\Python37\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 33280: character maps to <undefined>

picture veganaize

All 25 comments

I might be mistaken, but afair it was mentioned on another issue that the JDK 11 build of Graal requires a Windows 10 SDK.

nightscape picture nightscape on 26 Nov 2019
👍1

I was able to get past the hurdle above by installing Python 2.7 and making it first in the PATH.

(Also of note is that I didn't include Java11 in EXTRA_JAVA_HOMES this time, either)

Now it makes it up until a _bad command to the linker_:

[8/8] LINK trufflenfi.dll
FAILED: trufflenfi.dll
link -nologo -dll -out:trufflenfi.dll src\api.obj src\closure.obj src\intrinsics.obj src\jni.obj src\lookup.obj src\lookup_win32.obj src\signature.obj C:\src\graalvm\graal\truffle\mxbuild\windows-amd64\src\libffi\amd64\ffi.lib
link: unknown option -- n
Try 'link --help' for more information.
ninja: build stopped: subcommand failed.

Full output:
output2.txt

veganaize picture veganaize on 26 Nov 2019

Success!

$ where link
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64\link.exe
C:\Program Files\Git\usr\bin\link.exe

_^ Those were the other way around so I stuffed the VC linker in the front of PATH._

Successful mx build output:
output3.txt

Successful mx vm output:
output4.txt

• I did _not_ need the Windows SDK 10. I _do_ have the Windows SDK 8.1and Windows SDK 7.1 installed.

I'm going to do it all over again with Java11 as the primary JDK. I will verify and document each step and post them here, below.

veganaize picture veganaize on 26 Nov 2019

New Build Config

Requires:
• Git for Windows
https://gitforwindows.org/

• Windows SDK v7.1 ( or build-tools w/ SDK 8.1 or 10 ? )
(web installer)
https://www.microsoft.com/en-us/download/details.aspx?id=8279
(dvd iso)
https://www.microsoft.com/en-us/download/details.aspx?id=8442

• Python 2.7
https://www.python.org/downloads/

• OpenJDK 8 (JVMCI)
https://github.com/graalvm/openjdk8-jvmci-builder/releases/download/jvmci-19.3-b05/openjdk-8u232-jvmci-19.3-b05-windows-amd64.tar.gz

• OpenJDK 11 (Red Hat)
https://developers.redhat.com/products/openjdk/download
(portable zip)
https://developers.redhat.com/download-manager/file/java-11-openjdk-11.0.5.10-2.windows.redhat.x86_64.zip
(msi installer)
https://developers.redhat.com/download-manager/file/java-11-openjdk-11.0.5.10-2.windows.redhat.x86_64.msi

_--> All Programs --> Microsoft Windows SDK v7.1 --> Windows SDK 7.1 Command Prompt_

> setenv /?
> setenv /Release /x64 /win7
> \progra~1\git\git-bash.exe

$ cd
$ pwd -W
$ mkdir graalvm
$ cd graalvm

$ export MKS_HOME=/c/Progra~1/Git/usr/bin
$ export JAVA_HOME=/c/Progra~1/RedHat/java-11-openjdk-11.0.5-2
$ export EXTRA_JAVA_HOMES=/c/openjdk1.8.0_232-jvmci-19.3-b05

$ export PATH=$MKS_HOME:${PATH}
$ export PATH=/c/Progra~2/Micros~4.0/VC/bin/amd64:${PATH}
$ export PATH=/c/Python27:${PATH}
$ export PATH=$JAVA_HOME/bin:${PATH}
$ export PATH=$PWD/mx:${PATH}

$ git clone https://github.com/graalvm/mx.git
$ cd mx
$ git tag -l
$ git checkout tags/5.247.4
$ cd ..

$ git clone -b release/graal-vm/19.3 https://github.com/graalvm/graal.git

$ cd graal/compiler
$ mx clean
$ mx build

And... bummer, it doesn't compile. ☺

in short:
cl : Command line error D8021 : invalid numeric argument '/FS'

mx build output:
output5.txt

veganaize picture veganaize on 26 Nov 2019
🎉1

Success! [ OpenJDK 11 (primary) + OpenJDK 8 ]

• Git for Windows
https://gitforwindows.org/

• MSVC Build Tools
https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools

• Windows 10 SDK
https://developer.microsoft.com/en-US/windows/downloads/windows-10-sdk

• Python 2.7
https://www.python.org/downloads/

• OpenJDK 8 (JVMCI)
https://github.com/graalvm/openjdk8-jvmci-builder/releases/download/jvmci-19.3-b05/openjdk-8u232-jvmci-19.3-b05-windows-amd64.tar.gz

• OpenJDK 11 (Red Hat)
https://developers.redhat.com/products/openjdk/download

_--> Visual C++ Build Tools --> Windows Desktop Command Prompts --> Visual C++ 2015 x64 Native Build Tools Command Prompt_
or
_--> Visual Studio 2015 --> Visual Studio Tools --> Windows Desktop Command Prompts --> VS2015 x64 Native Tools Command Prompt_

> cl -Bv
> \progra~1\git\git-bash.exe

$ cd
$ pwd -W
$ mkdir graalvm
$ cd graalvm



md5-38b0a4071d0ec16dc2744b3d4526f2c1



$ export MKS_HOME=/c/Progra~1/Git/usr/bin
$ export JAVA_HOME=/c/Progra~1/RedHat/java-11-openjdk-11.0.5-2
$ export EXTRA_JAVA_HOMES=/c/openjdk1.8.0_232-jvmci-19.3-b05



md5-38b0a4071d0ec16dc2744b3d4526f2c1



$ export PATH=$MKS_HOME:${PATH}
$ export PATH=/c/Progra~2/Micros~1.0/VC/bin/amd64:${PATH}
$ export PATH=/c/Python27:${PATH}
$ export PATH=$JAVA_HOME/bin:${PATH}
$ export PATH=$PWD/mx:${PATH}



md5-38b0a4071d0ec16dc2744b3d4526f2c1



$ git clone https://github.com/graalvm/mx.git
$ cd mx
$ git tag -l
$ git checkout tags/5.247.4
$ cd ..



md5-38b0a4071d0ec16dc2744b3d4526f2c1



$ git clone -b release/graal-vm/19.3 https://github.com/graalvm/graal.git



md5-38b0a4071d0ec16dc2744b3d4526f2c1



$ cd graal/compiler
$ mx clean
$ mx build
$ mx vm
veganaize picture veganaize on 27 Nov 2019

Well done @veganaize ! I don't suppose you'd like to try submit a PR to update https://github.com/oracle/graal/blob/master/.travis.yml that adds a Windows Travis gate to capture this recipe and do some basic testing on the resulting build? /cc @christhalinger

dougxc picture dougxc on 27 Nov 2019

Thank you @dougxc, it would be my pleasure.

veganaize picture veganaize on 27 Nov 2019
❤1

@veganaize, this is a great job! If I had seen this before...
I also spent time building Graal (with "VM Suite") based on JDK 11 for Windows and one problem I ran into was the right choice of the base JDK.
I tried AdoptOpenJDK, Graal 19.3.0 itself and then I found in Graal CI so-called "labs" JDK.
And only the last one works.
Just to clarify and maybe for others who ran into similar problems, here are the errors.
AdoptOpenJDK (jdk-11.0.5+10):

D:\dev\projects\graalvm\graal\vm>set JAVA_HOME=D:\dev\sdk\jdk-11.0.5+10

D:\dev\projects\graalvm\graal\vm>mx --env ce-win build
build: Checking SubstrateVM requirements for building ...
Could not find any unresolved JVM_* symbols in static JDK libraries

Graal 19.3.0 (graalvm-ce-java11-windows-amd64-19.3.0):

D:\dev\projects\graalvm\graal\vm>set JAVA_HOME=D:\dev\sdk\graalvm-ce-java11-19.3.0

D:\dev\projects\graalvm\graal\vm>mx --env ce-win clean
Cleaning ...

D:\dev\projects\graalvm\graal\vm>mx --env ce-win build
build: Checking SubstrateVM requirements for building ...
Updated D:\dev\projects\graalvm\graal\substratevm\src\com.oracle.svm.native.jvm.windows\src\src_gen\JvmFuncsFallbacks.c
JAVA_HOME: D:\dev\sdk\graalvm-ce-java11-19.3.0
EXTRA_JAVA_HOMES:
Dependencies removed from build:
 project org.graalvm.libgraal.jdk13 was removed as JDK 13 is not available
 project org.graalvm.libgraal.jdk8 was removed as JDK 1.8 is not available
 project org.graalvm.compiler.serviceprovider.jdk13 was removed as JDK 13 is not available
 project org.graalvm.compiler.serviceprovider.jdk8 was removed as JDK 1.8 is not available
 project org.graalvm.compiler.hotspot.jdk12 was removed as JDK 12 is not available
 project org.graalvm.compiler.hotspot.jdk13 was removed as JDK 13 is not available
 project org.graalvm.compiler.hotspot.jdk14 was removed as JDK 14 is not available
 project org.graalvm.compiler.hotspot.jdk8 was removed as JDK 1.8 is not available
 project com.oracle.truffle.api.jdk8 was removed as JDK 1.8 is not available
 project org.graalvm.compiler.truffle.runtime.serviceprovider.jdk8 was removed as JDK 1.8 is not available
 project org.graalvm.compiler.truffle.runtime.hotspot.jdk8+13 was removed as JDK 1.8 is not available
 optional library LLVM_PLATFORM_SPECIFIC_SHADOWED was removed as it is not available
 removed org.graalvm.compiler.core.llvm because LLVM_PLATFORM_SPECIFIC_SHADOWED was removed
 project com.oracle.svm.truffle.tck was removed as JDK 1.8 is not available
 project org.graalvm.compiler.replacements.jdk12.test was removed as JDK 12 is not available
 project com.oracle.svm.hosted.jdk14 was removed as JDK 14 is not available
 project com.oracle.svm.core.jdk8 was removed as JDK 1.8 is not available
 com.oracle.svm.native.jvm.posix removed: only darwin and linux are supported
 project com.oracle.graalvm.locator.jdk8 was removed as JDK 1.8 is not available
 removed org.graalvm.compiler.replacements.llvm because org.graalvm.compiler.core.llvm was removed
 project org.graalvm.compiler.hotspot.management.libgraal was removed as JDK 1.8 is not available
 distribution GRAAL_LLVM was removed as all its dependencies were removed
 removed com.oracle.svm.core.graal.llvm because GRAAL_LLVM was removed
 optional library LLVM_ORG was removed as it is not available
 distribution GRAAL_MANAGEMENT_LIBGRAAL was removed as all its dependencies were removed
 distribution SVM_TRUFFLE_TCK was removed as all its dependencies were removed
 distribution SVM_LLVM was removed as all its dependencies were removed
Non-default dependencies removed from build (use mx build --all to build them):
 JACOCOREPORT_0.8.4
Compiling com.oracle.mxtool.compilerserver with javac(JDK 11)... [D:\dev\env\mx\mxbuild\java\com.oracle.mxtool.compilerserver\bin\com\oracle\mxtool\compilerserver\CompilerDaemon.class does not exist]
Compiling org.graalvm.compiler.truffle.runtime.serviceprovider with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\compiler\mxbuild\src\org.graalvm.compiler.truffle.runtime.serviceprovider\bin\org\graalvm\compiler\truffle\runtime\serviceprovider\TruffleRuntimeServices.class does not exist]
Compiling org.graalvm.compiler.truffle.runtime.serviceprovider.jdk11 with javac-daemon(JDK 11)... [dependency org.graalvm.compiler.truffle.runtime.serviceprovider updated]
Compiling org.graalvm.graphio with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\compiler\mxbuild\src\org.graalvm.graphio\bin\org\graalvm\graphio\DefaultGraphBlocks.class does not exist]
Compiling org.graalvm.compiler.api.directives with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\compiler\mxbuild\src\org.graalvm.compiler.api.directives\bin\org\graalvm\compiler\api\directives\GraalDirectives.class does not exist]
Compiling org.graalvm.compiler.api.runtime with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\compiler\mxbuild\src\org.graalvm.compiler.api.runtime\bin\org\graalvm\compiler\api\runtime\GraalJVMCICompiler.class does not exist]
Compiling org.graalvm.compiler.api.test with javac-daemon(JDK 11)... [dependency org.graalvm.compiler.api.runtime updated]
Compiling org.graalvm.compiler.processor with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\compiler\mxbuild\src\org.graalvm.compiler.processor\bin\org\graalvm\compiler\processor\AbstractProcessor.class does not exist]
Compiling org.graalvm.compiler.serviceprovider.processor with javac-daemon(JDK 11)... [dependency org.graalvm.compiler.processor updated]
Archiving GRAAL_PROCESSOR_COMMON... [dependency org.graalvm.compiler.processor updated]
Archiving GRAAL_SERVICEPROVIDER_PROCESSOR... [dependency org.graalvm.compiler.serviceprovider.processor updated]
Compiling org.graalvm.compiler.options.processor with javac-daemon(JDK 11)... [dependency org.graalvm.compiler.processor updated]
Archiving GRAAL_OPTIONS_PROCESSOR... [dependency org.graalvm.compiler.options.processor updated]
Compiling org.graalvm.compiler.nodeinfo.processor with javac-daemon(JDK 11)... [dependency org.graalvm.compiler.processor updated]
Archiving GRAAL_NODEINFO_PROCESSOR... [dependency org.graalvm.compiler.nodeinfo.processor updated]
Compiling org.graalvm.compiler.nodeinfo with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\compiler\mxbuild\src\org.graalvm.compiler.nodeinfo\bin\org\graalvm\compiler\nodeinfo\InputType.class does not exist]
Compiling org.graalvm.collections with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\sdk\mxbuild\src\org.graalvm.collections\bin\org\graalvm\collections\EconomicMap.class does not exist]
Compiling org.graalvm.word with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\sdk\mxbuild\src\org.graalvm.word\bin\org\graalvm\word\ComparableWord.class does not exist]
Compiling org.graalvm.options with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\sdk\mxbuild\src\org.graalvm.options\bin\org\graalvm\options\OptionCategory.class does not exist]
Compiling org.graalvm.nativeimage with javac-daemon(JDK 11)... [dependency org.graalvm.word updated]
Compiling org.graalvm.home with javac-daemon(JDK 11)... [dependency org.graalvm.nativeimage updated]
Compiling org.graalvm.polyglot with javac-daemon(JDK 11)... [dependency org.graalvm.collections updated]
Archiving GRAAL_SDK... [dependency org.graalvm.polyglot updated]
Building Java module org.graalvm.sdk (graal-sdk.jar) from GRAAL_SDK
Compiling org.graalvm.util with javac-daemon(JDK 11)... [dependency GRAAL_SDK updated]
Compiling org.graalvm.compiler.serviceprovider with javac-daemon(JDK 11)... [D:\dev\projects\graalvm\graal\compiler\mxbuild\src\org.graalvm.compiler.serviceprovider\bin\org\graalvm\compiler\serviceprovider\BufferUtil.class does not exist]
Compiling org.graalvm.compiler.options with javac-daemon(JDK 11)... [dependency GRAAL_SDK updated]
Traceback (most recent call last):
  File "D:\dev\env\mx\mx.py", line 19360, in <module>
    main()
  File "D:\dev\env\mx\mx.py", line 19341, in main
    retcode = c(command_args)
  File "D:\dev\env\mx\mx_commands.py", line 147, in __call__
    return self.command_function(*args, **kwargs)
  File "D:\dev\projects\graalvm\graal\substratevm\mx.substratevm\mx_substratevm.py", line 1298, in build
    orig_command_build(args, vm)
  File "D:\dev\env\mx\mx_commands.py", line 147, in __call__
    return self.command_function(*args, **kwargs)
  File "D:\dev\env\mx\mx.py", line 13632, in build
    t.prepare(daemons)
  File "D:\dev\env\mx\mx.py", line 6967, in prepare
    self.compiler.prepare_daemon(daemons, self.compileArgs)
  File "D:\dev\env\mx\mx.py", line 7310, in prepare_daemon
    self.daemon = JavacDaemon(self.jdk, jvmArgs)
  File "D:\dev\env\mx\mx.py", line 7423, in __init__
    CompilerDaemon.__init__(self, jdk, jvmArgs, 'com.oracle.mxtool.compilerserver.JavacDaemon', jdk.toolsjar, ['--force-javac'])
  File "D:\dev\env\mx\mx.py", line 7359, in __init__
    raise RuntimeError('Error starting ' + self.name() + ': returncode=' + str(returncode) + '\n' + ''.join(pout))
RuntimeError: Error starting javac-daemon: returncode=-1
Exception during JVMCI compiler initialization:

java.net.SocketException: Connection reset
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
        at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
        at java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
        at java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
        at com.oracle.mxtool.compilerserver.CompilerDaemon$Connection.run(CompilerDaemon.java:137)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

Labs JDK (labsjdk-ce-11.0.5+10-jvmci-19.3-b06-windows-amd64):

D:\dev\projects\graalvm\graal\vm>set JAVA_HOME=D:\dev\sdk\labsjdk-ce-11.0.5-jvmci-19.3-b06

D:\dev\projects\graalvm\graal\vm>mx --env ce-win clean
Cleaning ...

D:\dev\projects\graalvm\graal\vm>mx --env ce-win build
build: Checking SubstrateVM requirements for building ...
Updated D:\dev\projects\graalvm\graal\substratevm\src\com.oracle.svm.native.jvm.windows\src\src_gen\JvmFuncsFallbacks.c
JAVA_HOME: D:\dev\sdk\labsjdk-ce-11.0.5-jvmci-19.3-b06
EXTRA_JAVA_HOMES:
...
Everything is fine!

D:\dev\projects\graalvm\graal\vm>mx --env ce-win graalvm-home
D:\dev\projects\graalvm\graal\sdk\mxbuild\windows-amd64\GRAALVM_CE_JAVA11\graalvm-ce-java11-20.0.0-dev

So, in the end, the choice of base JDK itself is important too.
I will try to build Graal with Red Hat JDK now! Thank you!

jbreathe picture jbreathe on 24 Dec 2019
👍1

Okay. So I've been attempting to transfer this knowledge into a .travis.yml
configuration and I just can't seem to get around a _maven certificate issue_...
https://travis-ci.com/veganaize/graal/builds/142655804#L530

I don't get this issue on my local machine. It only appears on the Travis CI Windows Server instance. I've tried opening up the firewall, various flavors of OpenJDK 11, manually installing packages, auto installing packages, ... you name it.

Here is the raw log file:
log[1].txt

... I suppose I should file an issue with them for leaving all of those ANSI escape characters in there ☺

Show stopper:
Error reading from https://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>

Any pointing in any direction would be much appreciated!

veganaize picture veganaize on 30 Dec 2019

Maybe there's some help here: https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error

dougxc picture dougxc on 6 Jan 2020
👍1

Maybe try using python3 instead of python2?

dougxc picture dougxc on 6 Jan 2020
👍1

I was getting build errors on my local machine with Python 3.x... I will try it on the server though; that's a good idea.

Also, while it's on my mind, I wanted to mention that I was having issues on my local machine when opening Git's BASH first (how Travis CI does it), instead of opening the MSVC++ CMD prompt and then launching Git's BASH from within that.

I've also tried running cmd /c based commands from Git's BASH, on the CI server instance, as well, but that doesn't seem to work well (if at all). So another option might be to stuff each CI build phase into a batch file -- maybe that would be both effective & easy... But I'd like to avoid adding any files to the source tree -- another option is to echo ... >>file.bat the file contents via the .travis.yml file, but I have tried that and there were issues. [ Try Powershell ? ]
-end notes to self-

veganaize picture veganaize on 6 Jan 2020

If it helps, we can add a top level .ci directory under which there would be a travis directory. At some point we might add an oracle sibling subdirectory for scripts used by ci.hocon.

dougxc picture dougxc on 6 Jan 2020
👍1

Okay. I was able to _conquer the certificate error_ by adding the following to the (global) environment:

SSL_CERT_FILE=$HOME/cacert.pem

and downloading the cert to the home folder:

cd $HOME
wget --quiet https://curl.haxx.se/ca/cacert.pem

...But now running into other issues... O, the joy!

Now the error is:

C:\Users\travis\build\veganaize\graal\truffle\mxbuild\windows-amd64\src\libffi\libffi-3.2.1\include\ffi.h(75): fatal error C1083: Cannot open include file: 'stddef.h': No such file or directory

⛏ Which I have faced and overcome before! ⛏

[ source: https://access.redhat.com/articles/2039753 (towards bottom of page) ]
[ Thanks @dougxc !! That [link ](https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error#comment70323432_27835619) helped !! ]

veganaize picture veganaize on 10 Jan 2020

@veganaize the required file can be found a few times in the Travis VM:

/c/Program Files/LLVM/lib/clang/9.0.0/include/stddef.h
/c/Program Files (x86)/Android/android-sdk/build-tools/25.0.3/renderscript/clang-include/stddef.h
/c/Program Files (x86)/Windows Kits/10/Include/10.0.10240.0/ucrt/stddef.h
/c/Program Files (x86)/Windows Kits/10/Include/10.0.10586.0/ucrt/stddef.h
/c/Program Files (x86)/Windows Kits/10/Include/10.0.15063.0/ucrt/stddef.h
/c/Program Files (x86)/Windows Kits/10/Include/10.0.16299.0/ucrt/stddef.h
/c/Program Files (x86)/Windows Kits/10/Include/10.0.17134.0/ucrt/stddef.h
/c/Program Files (x86)/Windows Kits/10/Include/10.0.17763.0/ucrt/stddef.h
/c/Program Files (x86)/Windows Kits/10/Include/10.0.18362.0/ucrt/stddef.h
/c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib/gcc/x86_64-w64-mingw32/8.1.0/include/stddef.h
/c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/x86_64-w64-mingw32/include/stddef.h

Do you happen to know which one is the correct one?
Also, the 1.0 in this path looks rather strange, even for a Windows short name:
/c/Progra~2/Micros~1.0/VC/bin/amd64

nightscape picture nightscape on 14 Jan 2020

@nightscape Thank you for the info. I'd imagine that any of the Windows 10 SDK ucrt folders should do. I'm personally more than a little disgusted that Microsoft would hide something so standard so far away ( stddef.h isn't even included with their standalone compiler / build tools ). It's probably just a PATH (ordering) issue.

The Micros~1.0 is a result of it being the first short name that begins with "Micros~" and the .0 is literally the extension of the (longer) name: Microsoft Visual Studio 14.0 (...see the dot zero at the end?) Try making a really long folder name that ends with a dot zero and see what you get ☺

[ P.S. I just finally got what you meant about the short name! It's likely different. Thanks! ]

veganaize picture veganaize on 14 Jan 2020

Apparently the biggest issue (after adding the Windows 10 SDK paths) was the Windows Kits folder name. Specifically: the space [in the long name] was an issue, as well as the fact that many sub-folders within the %ProgramFiles(x86)% folder have a tendency to start with Windows so the short name is very variable. I was able to obtain the Travis CI instance's specific name with:

$ cd /c/Progra~2
$ cmd //c "dir /x"

...Which was WINDOW~3
Then I added the following to the _global environment variables_...

        - INCLUDE=/c/Progra~2/Micros~1.0/VC/INCLUDE:/c/Progra~2/WINDOW~3/10/include/10.0.18362.0/ucrt:/c/Progra~2/WINDOW~3/10/include/10.0.18362.0/shared:/c/Progra~2/WINDOW~3/10/include/10.0.18362.0/um:/c/Progra~2/WINDOW~3/10/include/10.0.18362.0/winrt
        - LIB=/c/Progra~2/Micros~1.0/VC/LIB/amd64:/c/Progra~2/WINDOW~3/10/lib/10.0.18362.0/ucrt/x64:/c/Progra~2/WINDOW~3/10/lib/10.0.18362.0/um/x64
        - VCINSTALLDIR=/c/Progra~2/Micros~1.0/VC/
        - VS140COMNTOOLS=/c/Progra~2/Micros~1.0/Common7/Tools/

Got it to build against the 19.3 release...
https://travis-ci.com/veganaize/graal/builds/144365619#L848

...With this YAML file...
https://github.com/veganaize/graal/blob/release/graal-vm/19.3/.travis.yml

So this gets us as far as: The command "mx build" exited with 0.
(Which I had to rub my eyes when I saw)

TO BE CONTINUED.

veganaize picture veganaize on 14 Jan 2020
🚀1

The command "mx -V vm" failed and exited with 1 during .

Traceback (most recent call last):
  File "C:/Users/travis/mx/mx.py", line 19362, in <module>
    main()
  File "C:/Users/travis/mx/mx.py", line 19343, in main
    retcode = c(command_args)
  File "C:/Users/travis/mx/mx_commands.py", line 147, in __call__
    return self.command_function(*args, **kwargs)
  File "C:\Users\travis\build\veganaize\graal\compiler\mx.compiler/mx_compiler.py", line 988, in run_vm
    return run_java(args, nonZeroIsFatal=nonZeroIsFatal, out=out, err=err, cwd=cwd, timeout=timeout)
  File "C:\Users\travis\build\veganaize\graal\compiler\mx.compiler/mx_compiler.py", line 957, in run_java
    return mx.run(cmd, nonZeroIsFatal=nonZeroIsFatal, out=u.out, err=u.err, cwd=cwd, env=env)
  File "C:/Users/travis/mx/mx.py", line 12496, in run
    raise subprocess.CalledProcessError(retcode, ' '.join(args))
subprocess.CalledProcessError: Command 'C : \ U s e r s \ t r a v i s \ b u i l d \ v e g a n a i z e \ g r a a l \ c o m p i l e r \ m x b u i l d \ w i n d o w s - a m d 6 4 \ g r a a l j d k s \ j d k 1 1 - c m p \ b i n \ j a v a . e x e   - s e r v e r   @ C : \ U s e r s \ t r a v i s \ b u i l d \ v e g a n a i z e \ g r a a l \ c o m p i l e r \ m x b u i l d \ w i n d o w s - a m d 6 4 \ g r a a l j d k s \ j d k 1 1 - c m p \ . a d d _ e x p o r t s   - X X : + U n l o c k E x p e r i m e n t a l V M O p t i o n s   - X X : + E n a b l e J V M C I   " - D g r a a l . C o m p i l a t i o n F a i l u r e A c t i o n = D i a g n o s e "   " - D g r a a l . P r i n t G r a p h = N e t w o r k "' returned non-zero exit status 1

https://travis-ci.com/veganaize/graal/builds/144432361#L3589

veganaize picture veganaize on 15 Jan 2020

That's the expected exit code when you run java (which is effectively what mx vm does) without any arguments.

dougxc picture dougxc on 15 Jan 2020

@dougxc on my personal machine (if I'm remembering correctly) I get the java command output after running mx build (successfully), but mx vm should exit cleanly (ie. without a traceback). It looks like the problem is the apparent spaces added in between each character in the path name string. I will try Python 3 again.

veganaize picture veganaize on 15 Jan 2020

mx vm will exit with a status of 1 and no traceback. However, mx -v vm will have the same exit code but include a traceback (a traceback is always shown for mx subprocess executions that exit with non-zero exit code if -v or -V is supplied).

dougxc picture dougxc on 15 Jan 2020

That seems counterintuitive & unconventional. It also causes the Travis CI instance to abort in error. Which means that mx unittest never gets executed...
https://travis-ci.com/veganaize/graal/builds/144526008#L822

veganaize picture veganaize on 15 Jan 2020

Not exactly sure what you are stating is counterintuitive & unconventional but the java launcher has always had that behavior. The question is why mx -V vm is being run in the Travis gate.

dougxc picture dougxc on 15 Jan 2020
👍1

Because the relevant documentation makes it seem as though that's an essential part of the build process...
https://github.com/oracle/graal/tree/master/compiler

I see what you mean. It could be clarified that running mx vm step simply executes the default vm.

Thank you @dougxc. I will eliminate that step and go straight to mx unittest.

veganaize picture veganaize on 15 Jan 2020

I'm closing this issue because the mx build command works under Windows given the correct tools & environment, as detailed above, and the remaining issues are with mx unittest and mx gate.

Continued here...
mx gate for graal/compiler on Windows

veganaize picture veganaize on 16 Jan 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

gersonimo picture gersonimo  ·  3Comments
helloguo picture helloguo  ·  3Comments
ilopmar picture ilopmar  ·  3Comments
janostgren picture janostgren  ·  3Comments
borkdude picture borkdude  ·  3Comments