Graal: How to include msvcr100.dll in Windows native image

[remkop]

Related to #1407: I'm trying to create an instruction manual for creating native image command line applications using the picocli library.

On Windows 10, the generated native image cannot be executed on a clean machine; it fails with an error like the following:

Note that this problem does not manifest on the machine where the native image was built. The DLL will be present on machines that have the toolchain set up to build the native image (Microsoft Windows SDK for Windows 7 and .NET Framework 4 and the C compilers from KB2519277). The problem only manifests when you try to execute the generated native image on a "clean" Windows 10 machine. (I used a VirtualBox VM with a 64-bit Windows 10 guest OS to reproduce the issue.)

It turns out that msvcr100.dll from VS C++ Redistributable 2010 must be present or the application will fail to start. I would like to include msvcr100.dll in the native image so my application can be distributed as a single executable.

Will this DLL be included automatically in the native image in future releases? If not, is there a way for me ensure it is included?

[remkop]

Steps to reproduce:

C:\Temp\issue1762>echo public class HelloWorld { public static void main(String[] args) { System.out.println("Hello World"); } } > HelloWorld.java

C:\Temp\issue1762>javac HelloWorld.java

C:\Temp\issue1762>C:\apps\graalvm-ce-19.2.1\bin\native-image -cp . HelloWorld

C:\Temp\issue1762>dir
...
22/10/2019  01:16               421 HelloWorld.class
22/10/2019  01:20         8,235,520 helloworld.exe
22/10/2019  01:20           263,425 helloworld.exp
22/10/2019  01:16               108 HelloWorld.java
22/10/2019  01:20           439,264 helloworld.lib
22/10/2019  01:20         1,338,368 helloworld.pdb

C:\Temp\issue1762>helloworld.exe
Hello World

Test on clean machine:

• install Oracle VM VirtualBox
• create new virtual machine with guest OS 64-bit Windows 10
• copy helloworld.exe to this VM (see shared folders)
• open DOS console and execute helloworld.exe
• this will show the above System Error - The program can't start because MSVCR100.dll is missing from your computer... error dialog

[webfolderio]

GraalVM must use /MT compiler flag to static CRT link. But it use /MD flag.

[remkop]

@webfolderio Thanks for the link!

I understand now that a module compiled with MT will have the runtime "inside it", while a module compiled with MD will link with a DLL at the moment of execution. When compiled with MD the DLL must exist on the target machine, which is the problem because MSVCR100.dll is often missing.

The comment in the source says:

Must use /MD in order to link with JDK native libraries built that way

I found that MD compiled DLLs should not be mixed with MT compiled DLLs. This is probably what the comment is referring to.

So, in order to fix this, we need to get MT compiled versions of some JDK native libraries? Does anyone know which JDK native libraries the comment refers to?

[kangzq0530]

As most of jdk native libraries are provided as static lib and already included in compiled exe, I think we can use /MT mode at most time. But if --enable-all-security-service is enabled and so sunec.dll must be present, /MT maybe cause crash.Maybe you can use some third party crypto libraries instead of jce in this case.

[remkop]

@bobvandette Would it be possible to use /MT by default, and only switch to /MD when necessary, e.g., when we need to link with a JDK native library built that way? (Which JDK native libraries are built with MD?)

[bobvandette]

The JDK static libraries for Windows are all built with /MD. I would have to experiment with building them with /MT to see if this could work. I would expect your generated native images to increase in size if we did this.

[remkop]

That makes sense. The msvcr100.dll file is ~810 KB but I suppose other libraries may be pulled in too. It would be good to better understand the trade-offs.

For many CLI applications, the ability to distribute apps as a single executable is a big part of the value of native images. It's a pity that we cannot rely on msvcr100.dll always being present.

Thank you for spending time on this!

[remkop]

@bobvandette Have you had a chance to try this? Is it feasible to remove the dependency on an external msvcr100.dll?

[remkop]

Any chance of getting this in GraalVM 19.3?

[bobvandette]

I did look into this and came to the conclusion that this wasn't something we should do since you cannot mix executables and libraries that were built with /MD and /MT. There are use cases where we generate DLL's that are used with the JDK and other cases where .DLL's from the JDK are used to augment a generated native-image.

The only way around this would be to build your own JDK and native-image binaries with some minor modifications to the sources. Sorry.

[remkop]

@bobvandette I am aware that /MD and /MT cannot be mixed, and I understand there are use cases where /MD is unavoidable.

What I was hoping was that the native-image generator tool could be enhanced to offer an option on Windows to build with /MT, where the tool would exit abnormally when it detects that /MT cannot be used because of the use cases you mention (required dependency on other .DLL's from the JDK, for example).

[remkop]

@shelajev I finally had a chance to try this with GraalVM 19.3 for Java 11. The issue remains with a different error (because native-image with Java 11 uses a different toolchain). The Japanese error message reads:

System Error
The program can't start because VCRUNTIME140.dll is missing from your computer. You may be able to resolve this by reinstalling the program.

@bobvandette raises a good point that executables and libraries that were built with /MD and /MT cannot be mixed, so this needs to be done with care.

However, I expect a similar request will be raised to allow statically linking libstdc++ on the linux/maxos platforms, because since 19.3 the sunec lib gets statically linked into the image when needed, which means that on Linux and macOS the image will depend on libstdc++.

I propose adding a new option to native-image, say, -H:+StaticallyLinkAll. Using this option would result in all dependencies being statically linked (with /MT on windows). There may be restrictions on the resulting images (for example, generate .exe only, no .dll's; cannot be used with dynamically linked libs, etc.), but this option would allow developers to distribute their application as a single executable file.

[sogaiu]

Minor point -- I think the message actually refers to VCRUNTIME140.dll.

[remkop]

Minor point -- I think the message actually refers to VCRUNTIME140.dll.

@sogaiu Thank you! I fixed the typo in my previous comment.

[sogaiu]

allow developers to distribute their application as a single executable file

This sounds nice if doable!

[bobvandette]

The suggestion of implementing -H:+StaticallyLinkAll sounds useful but has a few challenges associated with its implementation.

GraalVM 19.3 switched to using JDK native code instead of the manual substitutions that were used for earlier versions of GraalVM. The JDK native code is delivered in GraalVM in the form of static libraries. These static libraries (zip.a, net.a, java.a, nio.a, etc) are built on Windows using /MD. The reason for this is that it allows the use of other JDK DLLs or third party DLL with native-image generated .exe's. If -H+StaticallyLinkAll were implemented, we would have to provide another set of these static libraries that are compiled using /MT and find a way of prohibiting the use of other DLLs or static libraries that were not properly compiled.

The mac/linux example described above is not the same problem since Linux and Mac do not have a problem mixing shared libraries and static libraries in a single executable. The solution to the libstdc++ problem is to optionally add the static version of that library on the link line in order to eliminate the runtime dependency on the shared library. This of course assumes that this static version is available in the toolchain. I don't see this library in my Xcode installation.

[remkop]

@bobvandette Is this related? RFR: 8239563 - Reduce public exports in dynamic libraries built from JDK static libraries

[bobvandette]

@bobvandette Is this related? RFR: 8239563 - Reduce public exports in dynamic libraries built from JDK static libraries

No, sorry. That fix is about avoiding adding export symbols from the JDK static libraries to native-image generated DLLs.

[micaelgallego]

Is there any plans to solve this issue?

It prevents providing an autocontained binary for windows.