Ompi: Open MPI Fails to Build Under OS X v10.11.6 'El Capitan' Due to the Absence of `javah` in Java 10

[RandomDSdevel]

Background information

What version of Open MPI are you using? (e.g., v1.10.3, v2.1.0, git branch name and hash, etc.)

     The version of Open MPI that I was attempting to build from source and install was v3.0.1 (I currently have v3.0.0 installed.)

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

     I initially installed Open MPI via the Homebrew package manager for OS X/macOS using the following command:

HOMEBREW_BUILD_FROM_SOURCE=1 brew install -vd --build-from-source open-mpi --with-java --with-mpi-thread-multiple

I keep Open MPI up to date by running the following command when upgrading only it:

HOMEBREW_BUILD_FROM_SOURCE=1 brew upgrade -vd --build-from-source open-mpi --with-java --with-mpi-thread-multiple

(Please note, however, that setting HOMEBREW_BUILD_FROM_SOURCE=1 is not standard practice for users of Homebrew, so individuals attempting to reproduce this issue should omit it if they either already have all Homebrew dependencies for Open MPI installed from source or binary or don't mind having Homebrew install 'bottles' pre-built using the project's CI.)

Please describe the system on which you are running

• Operating system/version: Apple OS X v10.11.6 'El Capitan,' build 15G20015
• Computer hardware: Apple Mid-2007 Aluminum iMac (iMac7,1) (More details available on request.)
• Network type: Cable broadband over local wireless (More details available on request.)

---

Details of the problem

     Open MPI fails to build its Java bindings from source under Homebrew on systems which only have Java 10 installed due to how its build system looks for and subsequently cannot find javah, which was deprecated in Java 9 in removed entirely in Java 10. I also reported this problem downstream to Homebrew as Homebrew/homebrew-core#26009 before opening this issue here.

[RandomDSdevel]

     Per and following up on investigations I've reported in more detail downstream, It appears to me that Open MPI's build system is picking up shims to Java commands that exist on OS X systems under /usr/bin/ (as symbolic links to the real shims hosted under /System/Library/Frameworks/JavaVM.frameworks/Versions/Current/Commands/.) Open MPI's build scripts pick up, among other shims, one for javah that Apple has not removed even though Oracle no longer supplies javah with JDKs.

[RandomDSdevel]

     What beyond the mentions of javah that searching for it via GitHub's search bar in this repository turns up and lines 96–108 of /ompi/config/opal_setup_java.m4 might I have to change to refer to javac -h instead of javah (and, on OS X/macOS systems, look in a subdirectory of /Library/Java/JavaVirtualMachines instead of where it looks now for JDK executables so your build system doesn't find Apple's shims?)

[RandomDSdevel]

     How exactly should I implement how Open MPI's Java binding's build infrastructure handles determining how it should generate the relevant C header files? As I've mentioned before, Java 9 introduced javac's -h option and deprecated javah, and Java 10 obsoleted and removed javah, but I don't know what the preference would be here regarding how this part of the project's build process should function with respect to this. I could change the makefiles to either:

1. Use your installed JDK's version to determine whether to look for javah (in which case they could, when used on a system that has Java 9 installed, either:
   
   
   
   1. Look for javah even though Java 9 deprecated the tool's use, or
   
   
   2. Skip looking for javah _because_ Java 9 deprecated it even though it's still there, electing to run with using javac -h instead.)
   
   
2. Use the existence of javah to determine whether to try falling back to see if javac has a -h switch available. (This would obviously build the binding headers using javah automatically on systems where Java 9 is available.)

If, after either path is taken forward, the build system is written to use javah even on systems which have Java 9 (and thus a deprecated javah) installed, then its invocations of that tool would involve it spewing deprecation warnings, just like it did while Open MPI users only had JDK 9 as the highest version of the toolset they could install.
     If a path-based approach were to be favored, then I'd definitely have to fix OS X/macOS JDK tool path detection first. (I thought I might be able to tackle the build system's javah handling first and the former later, and was inclined to do so since the latter's make script looks simpler to fiddle with. I can do things in the opposite order if anybody wants me to, though.)

[RandomDSdevel]

     For future reference, #5001 has now solved half of this issue with #5029 having backported the commit associated with the first of those two PRs back to Open MPI v3.0.x. #5015 will resolve the other half of this issue and should follow shortly.

[jsquyres]

Merged into all release branches. Closing this issue.

[RandomDSdevel]

@jsquyres: I know @ggouaillardet's #5001 was, but what about my #5015?

[RandomDSdevel]

     Nearly forgot to mention that #5015 has now been merged at least to master (and probably to other branches, too, but I haven't checked on that.)

Update: I don't see this on the v3.0.x branch; investigating…

[RandomDSdevel]

     I should note here for posterity that #5119, which backports #5015 to the v3.0.x branch, has now been merged, wrapping things up here. Now to downstream all of this to Homebrew…

[RandomDSdevel]

     For future reference, I will also note this addendum to my last comment: #5160, a correction to #5015, has now been merged to master and backported on top of #5119 to the v3.0.x branch by #5161. _Now_ I finally think I'm done here; _phew…_

[jsquyres]