Onnxruntime: What package should java install from maven

[Jie-Yuan]

What package should java install from maven

[pranavsharma]

The java bindings are not available on maven yet. We'll work on it.

[pranavsharma]

cc @Craigacp

[Craigacp]

If you've built it yourself you can publish it to your local maven repo by adding id 'maven-publish' after line 2 of the build.gradle and then running gradle publishToMavenLocal from the java directory.

Edit: actually that silently fails. I'll figure out another approach.

[Craigacp]

This commit (https://github.com/Craigacp/onnxruntime/commit/7be7b5702a29f76938946bcd33b1d77a700e12f2) will allow you to push to your local maven repository which you can then use in a downstream project. Or you can push the jars to an artifactory or similar private maven repository and access them from there (that works without the commit).

[Craigacp]

You'll need to run the gradle command inside the java folder as follows gradle -DcmakeBuildDir=../build/Linux/<build-type> publishToMavenLocal. As Pranav said, we're still working on proper maven builds.

[faxu]

Dupe of #2675

[hjander]

Will there a java version of the runtime beside bytedeco one ? Will there the be versions for dnnl and mkl ?

[Craigacp]

I believe the current plan is for a multiplatform CPU build, and possibly a Windows and Linux CUDA build.

Note the bytedeco version uses different library loading and is a modified version of the code supplied in this repository.

[saudet]

Note the bytedeco version uses different library loading and is a modified version of the code supplied in this repository.

Yes, modified, because it fixes the issues mentioned at https://github.com/microsoft/onnxruntime/pull/2215#issuecomment-545243304. It wouldn't need to be modified if the maintainers here accepted to apply these fixes to this repository.

[Craigacp]

Most of those are no longer an issue with the latest release. There are os and architecture specific folders and it loads the appropriate one for the platform. If you don't want to load the binary from the jar there is a flag which controls that. There are alternate paths for loading libraries on Android. I think the only thing left from your complaint list is packaging CUDA or MKL with the jars, and that's not going to happen because it causes huge download sizes amongst other problems.

[saudet]

Downloads for CUDA and MKL can be made optional. It works just fine and many people are happy about them:
https://github.com/bytedeco/javacpp-presets/tree/master/onnxruntime#sample-usage

[saudet]

In any case, I see you're well underway in recreating JavaCPP just for ONNX Runtime, congratulations! But JavaCPP works with other libraries than ONNX Runtime. You're also neglecting the most important issue, support for Java containers:

As discussed in pull #1723 your loader doesn't support using the same library from multiple class loaders, which are heavily used by containers like Flink, Tomcat, WebLogic, etc, see issue tensorflow/tensorflow#6926, or build systems like sbt, see issue tensorflow/tensorflow#19298. Similar issues occur with shading, see issue netty/netty#7272.

[Craigacp]

As I said in the previous issue, and the other times you've brought it up, it's a security thing to prevent JNI libraries from passing classes between different class loaders. There is progress in the JDK towards removing this limitation by adding a new library loading path - https://bugs.openjdk.java.net/browse/JDK-8228336, but that's part of Panama, and only present in JDK 15.

[saudet]

As I said in the previous issue, and the other times you've brought it up, it's a security thing to prevent JNI libraries from passing classes between different class loaders. There is progress in the JDK towards removing this limitation by adding a new library loading path - https://bugs.openjdk.java.net/browse/JDK-8228336, but that's part of Panama, and only present in JDK 15.

That won't work with JNI and I'm pretty sure that the FFI part of Panama isn't going to be in JDK 15 or any version of JDK before 20 or so, but OpenJDK is not releasing any roadmaps to the public, so no one really knows. Besides, you are writing the wrappers for ONNX Runtime with JNI, not Panama. Are you planning on rewriting everything with Panama?

In any case, we are stuck with JNI and its limitations for a while still. I understand you don't personally like how everyone is doing it, but the community has come to accept the risks because it's a necessary evil, as with anything else related to native code. Users do not have a choice when it comes to adopting such workarounds to get things working with Java, and besides it's been working perfectly fine. I'm not aware of anyone reporting any security issues. Since you're representing Oracle here though, you could actually do something more about it. Just preventing users from loading libraries like ONNX Runtime with their frameworks isn't going to help Oracle earn any brownie points. If you need to "get it right", then please go and get it done, but if you're not in a position to fix this, I would encourage you, as they say, to not let perfect get in the way of good.

[Craigacp]

Binaries are now available on maven central for CPU and GPU with the v1.3.1 release.

[Craigacp]

@faxu this can be closed now.