Conda-forge.github.io: Packaging of Java components

[ctrueden]

At the suggestion of @jakirkham, I am opening this issue here to start a conversation about how conda might manage the packaging for Java components in a uniform way.

Quick introduction: I am maintainer of the ImageJ and Fiji projects for N-dimensional scientific image processing and analysis. A major interest of mine is making ImageJ accessible from other ecosystems such as PyData and Node.js. We now have an imagej module on PyPI allowing to access an ImageJ installation as Python code in-process or interprocess. But it was suggested that we also create a conda package that ships all of ImageJ. @jakirkham indicated the most proper way to do this would be publish each Java component (i.e. JAR file) as its own conda package, with a dependency hierarchy matching that declared in the Maven POMs.

So my first question is: do others concur that the ideal thing would be a one-for-one correspondence between Java components and Conda packages?

If so, my second question then is: what would be the best way forward to achieve that?

@jakirkham suggested we could invent some machinery to auto-update conda packages of Java components based on their Maven releases. I countered with the idea that it would be more elegant to augment conda itself to understand Maven repositories directly, so that conda install java:org.scijava:scijava-common or some such would install the latest version of org.scijava:scijava directly from Maven Central.

A third related question is: would it be OK to simply ship binaries, rather than rebuilding from source? I am ignorant of the conda-forge (and conda in general) policy on building from source... but one nice thing about Java artifacts is that once a release is built, it is completely portable and there is no need to rebuild it—and in fact, rebuilding it can be problematic, since JAR files are typically signed by the author to ensure fidelity. (Of course, artifacts _can_ be rebuilt from source by third parties, but the resulting JAR itself will not be identical, even if the bytecode of all the .class files inside is the same.) So it would be ideal if conda could support _distributing the already-built artifacts_ rather than mandating rebuilds from source.

There are many issues here, and I expect the conversation will be quite complex—but I am willing to invest some energy if people in the community are earnestly interested in making Java components available via conda. See also imagej/imagej#162 which is about packaging Java components for Gentoo but has probably a high degree of overlap with many of the concerns here.

Thoughts? Questions? Ideas?

[jakirkham]

cc @conda-forge/core

[scopatz]

Hi @ctrueden,

do others concur that the ideal thing would be a one-for-one correspondence between Java components and Conda packages?

Yes.

what would be the best way forward to achieve that?

I agree with @jakirkham here that the best way forward is to update the conda-forge bot (see https://github.com/regro/cf-scripts) to understand maven.

Also, I think modifying conda itself to pull from maven central is the wrong way to go. This is for a couple of reasons:

1. conda is and should be language agnostic, it should not give special treatment to any particular language.
2. Components on Maven Central do not have the meta data that conda expects. Integrating with Maven Central would require pulling down and construting a dependency graph which conda could use to solve which packages should be installed. If conda itself did this each time a conda install was called, the run time of installing package would balloon worse than it already is.
3. There are other places that Java components could choose to be hosted other than Maven central. So doing something special for Maven Central covers most, but not all, of the use cases.
4. The bot is where we put external package hosting (pypi, libraries.io, etc) code already. It makes sense to put Maven Central stuff here, rather than shoehorn it in elsewhere.

would it be OK to simply ship binaries, rather than rebuilding from source?

Yes, as long as they don't do anthing with the JNI or other external shared objects, this should be totally fine, and is in fact desirable. There should probably be an equivalent noarch: java to denote that a java package is platform independent.

[CJ-Wright]

Attn: @justcalamari

[ctrueden]

Thanks for your comments, @scopatz!

conda is and should be language agnostic, it should not give special treatment to any particular language.

As an architect I deeply sympathize with that sentiment from a "clean design" point of view. But pragmatically, I would push back on this: conda should support all the major ecosystems in a way that is both correct and convenient for end users. If that means special casing support for the major ecosystems, and that special casing avoids creating a lot of work and/or noise for humans, then I am all for it. To me this is no different than, say, git's various git-remote backend plugins for systems like SVN and mercurial. The benefits of leveraging existing mature and popular distribution platforms outweighs the intangible benefits of "agnosticism" here. And the system would still be "agnostic" in the sense that downstream code leveraging conda would not need to care which "backends" are being utilized for any packages being reasoned about.

If conda itself did this each time a conda install was called, the run time of installing package would balloon worse than it already is.

I agree that leaning directly on Maven remote repositories, with no caching mechanism, would be suboptimal performance-wise. But the metadata could be derived and cached in conda's repository after the first query. This is how Maven repository proxying works, and it is very powerful, allowing for aggregation of multiple repositories into a single unified facade, without undue burden on the servers upstream—in fact, it relieves their burden because every such proxy cache becomes a new source of the same artifacts, i.e. an additional mirror.

There are other places that Java components could choose to be hosted other than Maven central. So doing something special for Maven Central covers most, but not all, of the use cases.

Absolutely. Actually, my proposal would be to support retrieving Java artifacts from arbitrarily specified Maven repositories. For my projects I'd definitely need that, since many/most ImageJ and Fiji artifacts are currently hosted at https://maven.imagej.net/, not Maven Central (yet). And there are also jitpack.io and Bintray's JCenter, which are increasingly popular alternatives to Maven Central. It would be magical if conda simply supported all of these out of the box.

The bot is where we put external package hosting (pypi, libraries.io, etc) code already.

I understand. And I figured that people would be against ideas that obviate the need for such a bot. Of course, package systems want to be fully in control of all their data, and leaning on external systems is potentially error-prone. So if what must happen is that a bot scrapes Maven repositories and translates the information there into conda-friendly structures, I can live with that. But I think that requiring every Java project to explicitly "opt in" to being available on conda would drastically reduce the utility and value of conda's Java support—especially if maintainers must manually trigger a corresponding conda package update after doing each Java release. Conversely, if conda transparently supported all Java components published to Maven repositories, it would be a huge coup for conda for many reasons; e.g., it would make multi-language projects really easy to distribute with conda. And it would avoid a classic and painful source of programming errors: duplicate (but all too often slightly out of sync) data structures.

as long as they don't do anthing with the JNI or other external shared objects, this should be totally fine, and is in fact desirable. There should probably be an equivalent noarch: java to denote that a java package is platform independent.

Right. So we need some place to stash additional metadata about projects beyond what is found in the Maven POM. We could certainly detect if a JAR file contains this additional metadata file. Then individual projects could self-declare. But for the majority of Java projects that will never cut a new release with such metadata incorporated, there must be another option.

• One idea would be to allow Java projects to declare such missing metadata about not only themselves, but also their dependencies. So e.g. SciJava Common depends on eventbus, an old library that is no longer developed. It could declare any needed metadata about eventbus as well, such that conda can recursively package eventbus if has not already been done. One downside is possible clashes if multiple packages declare mismatched dependency metadata; another is potential security concerns if a package misrepresents a dependency somehow?
• Alternately, there could be meta-project(s) that are specifically designed just to greenlight a bunch of Java projects for use as dependencies. But someone in the central community would probably have to maintain that more actively.

In summary: I would strongly favor a system that does not require specifying additional metadata beyond what is already in the POM, so that you can get the vast majority of already-published Java projects for free. Any approach that requires white-listing or processing Java projects will substantially diminish the utility here. But if others disagree or see insurmountable technical obstacles to what I'm proposing, then I'd rather have a small subset of Java components in conda than no Java in conda at all.

[CJ-Wright]

But I think that requiring every Java project to explicitly "opt in" to being available on conda would drastically reduce the utility and value of conda's Java support—especially if maintainers must manually trigger a corresponding conda package update after doing each Java release.

I think that this "opt in" is an important part of the conda-forge maintenance, since it makes certain that there are people responsible for the maintenance of the package and that the package is needed. I think this is similar as to why we haven't tried to package everything on PyPI. If we were to package everything on Mavin who would be responsible for making sure the conda package worked? Who would be the point of contact? There has been some discussion on automatic updates of versions: https://github.com/regro/cf-scripts/issues/87. However, I think the current consensus is that at the current level of automation direct pushing would result in more broken packages than it would help. Having the maintainers in the loop help to identify and fix problems that otherwise would have made it into production.

And I figured that people would be against ideas that obviate the need for such a bot.

I think that the bot currently provides a lot of utility to the maintainers by proposing version updates to recipes. I would have no problem eliminating the bot if it was no longer needed, provided that its replacement provided the same level of control and quality.

[ctrueden]

If we were to package everything on Mavin who would be responsible for making sure the conda package worked? Who would be the point of contact?

That is a very good point, @CJ-Wright. The only reasonable answer I can think of is: the maintainers of conda's integration mechanism itself would do their best to ensure it covers the widest swath of use cases. And in edge cases where that is too difficult or time consuming, the maintainer response would have to be two-fold: A) that package does not conform to the requirements of conda's integration mechanism; and B) you can eliminate this edge case by special-casing the project via some explicit special-casing mechanism that overrides the default behavior somehow. But of course it would not be the core maintainers' job to take care of doing this every time an edge case arises. The Pareto principle is highly applicable here.

In regro/cf-scripts#87 @msarahan wrote:

humans should always be in the loop for stuff that gets deployed to our primary channel

I think users would understand that java/maven packages are a different (but hopefully still well supported) beast than "native" packages in the primary channel itself. I urge against reflexively nixing fully automatic packaging approaches due to "quality" concerns. There are an astounding number of packages across many ecosystems. Requiring manual review of each and every package creates a commensurately astounding—and ultimately untenable—amount of work for the conda community. For conda to truly conquer cross-ecosystem packaging effectively, humans simply _cannot_ be involved with every new+existing package in the wild.

[djsutherland]

Updating conda to understand maven might be nice, but that's not a conda-forge thing, that's a conda thing. The only thing conda-forge itself can do is make it easier to create and maintain packages from Maven, through the bot and a templating script like we have for R packages.

(In any case, it would be nicer to have all of Maven available for free than having to opt-in – we have the same problem with PyPI and CRAN – but previous attempts at making conda automatically load things from pypi didn't work out, because as @scopatz pointed out not all of the metadata that conda needs was available there.)

I know very little about Java, but we could probably use noarch: generic for Java packages, if they're literally just a JAR file that needs to go in some system-agnostic location.

[ctrueden]

Updating conda to understand maven might be nice, but that's not a conda-forge thing, that's a conda thing.

Yeah. But I was not sure of a better place to start a more general conversation about that. @jakirkham told me that some Anaconda folks pay attention to this issue tracker, so I thought it could make sense to discuss here first, and potentially move upstream somewhere if there's critical mass of support in that direction.

I know very little about Java, but we could probably use noarch: generic for Java packages, if they're literally just a JAR file that needs to go in some system-agnostic location.

In the 80% case, I think it might be!

A Maven-based Java package is:

1. A JAR file containing the binaries.
2. A Maven POM with metadata—although this is often, but not always, also embedded in the JAR file as well.
3. Optionally/often, a corresponding -javadoc classifier JAR containing generated javadocs.
4. Optionally/often, a corresponding -sources classifier JAR containing the source code.
5. MD5 and SHA1 checksum files for each of the above, to validate integrity.

Here is an example: org.scijava:scijava-common:2.73.1.

From a conda perspective, it is likely that only the JAR file need be shipped to target systems. The Maven POM is mainly useful at packaging time to extract needed metadata. (I say in ignorance of conda's internals.)

As we move into the 20%, there are more possible things:

1. A -tests classifier JAR containing the compiled unit tests (not normally part of the main artifact). The only reason to publish this is if you want to have downstream projects depend on the _test_ classes of a dependency for some reason. This is rare.
2. One or more -natives or -native-<platform> classifier JARs containing native libraries upon which the Java code depends. The details of how these native libraries get built and packaged at compile time, as well as unpacked and loaded at runtime, vary a lot by project, unfortunately.
3. Uber-JARs with all dependencies embedded. This is an evil thing to do from a component-oriented packaging perspective, but a convenient thing for simple application deployment.
4. Open-ended other classifier JARs according to the needs of the project. For example, the com.miglayout:miglayout project publishes several classifiers to offer different versions of the library for Swing vs. Eclipse SWT. This is not necessarily best practice though, and quite rare.

[jakirkham]

cc @kalefranz @msarahan @mingwandroid

[jakirkham]

For context the closest I think we got with PyPI previously was conda pipbuild, which traced through all dependencies of a specific package and converted them to conda packages in a local cache, which could then be explicitly installed. This wound up being deprecated and removed due to some issues (my memory is hazy here). Still this sort of thing seems really desirable. Not sure if there are new efforts to tackle this again.

[kalefranz]

There should probably be an equivalent noarch: java to denote that a java package is platform independent.

If java actually needs special things done at install time, that's fine. For example, noach: python (1) creates .pyc files, (2) does a translation to the correct lib/pythonX.Y/site-packages directory, and (3) creates entry points. If nothing like that is required, we can just use noarch: generic. If there are special things for java though, I'm very happy to add them to conda as a noarch: java package type.

[ctrueden]

If nothing like that is required, we can just use noarch: generic. If there are special things for java though, I'm very happy to add them to conda as a noarch: java package type.

@kalefranz I can't think of anything that would be necessary for Java components. The only thing that occurs to me that _might_ be useful is creating a launcher in the case of an "executable" JAR file—i.e., a JAR file with a declared Main-Class attribute, which can be run via java -jar myJar.jar invocation. I actually wrote a simple Java code launcher called jrun to facilitate running Java code from Maven repositories; something like that could be built in to conda, perhaps. But it is hardly a necessary feature.

[kalefranz]

Conda's original language ecosystem target was the python world. And I'm calling that out because we're just now in the last several months starting to work out if/how we want to give conda the ability to directly install packages from PyPI. It's not an easy problem to solve, even though we eventually want to get there someday. Ultimately, if we succeed, it's not going to be something that'll be complete in the next month or two. It could be a year or more before we're able to directly install even a subset of packages (e.g. pure-python wheels) directly from a PyPI-like repository. This, after a half-decade of this conda community working most intensely around python.

I love this topic, and the discussion about more richly supporting the java community. I'm personally 👍 👍 . However, we need to start with conda packages. We need to build out a substantial base of java as conda packages, just as we have for python and r, so that we can understand the design, architecture, and usage patterns involved. So that we can gain enough expertise as a community for us to know what something much more advanced like having the conda solver interact with a maven repository, and having conda's unlink-link transactions directly install artifacts from such repositories, might ultimately look like.

[kalefranz]

I also appreciate the listing of package types from https://github.com/conda-forge/conda-forge.github.io/issues/590#issuecomment-391095456. For a while now I've been a proponent of having separate package types for source, test, docs, and build logs, in addition to the primary binary artifact.

[jakirkham]

The only thing that occurs to me that _might_ be useful is creating a launcher in the case of an "executable" JAR file—i.e., a JAR file with a declared Main-Class attribute, which can be run via java -jar myJar.jar invocation. I actually wrote a simple Java code launcher called jrun to facilitate running Java code from Maven repositories; something like that could be built in to conda, perhaps.

This seems worth noting in a conda/conda-build issue to start. Should be useful regardless how the rest of this discussion shakes out.

[sodre]

This discussion could not be more timely to the package I am reviewing now.

__tabula-py_ #5930 is a python wrapper around a uber-jar CLI. The wheel in pypi includes the .jar just like we might include a .so for native bindings.__

It is not clear to me how packages like that should be handled within conda-forge. The answer can go from accept it just like it is in pypi, down to requiring everything to be built from source. I think the answer has to lie somewhere in-between and I am leaning towards building the uber-jar from source(but not its dependencies) and patching the python module to look for the library at $PREFIX/lib.

One word of caution on splitting open uber jars is that Java programs often don't agree on versions. For example, look how the Apache NiFi project had to create their own archive format ("nar") to keep plugins from interfering with each other. The closest equivalent in conda-world would be to use conda environments for each plugin.

I agree with @kalefranz that we need to start with conda packages, and I add that they would ideally be hosted at anaconda.org without need to proxy-out to other services like Maven Central.

[sodre]

Maybe a perfectly actionable starting point is for conda-forge to agree on a naming convention for java (go, etc..) packages before we start getting into more package name conflicts.

[jakirkham]

As to naming, would propose java-* to match with existing names of the same form.

[djsutherland]

Agreed with java-*. Though there will continue to be annoyances related to that, and something like the namespaces proposal would probably be nice....

[ctrueden]

I am leaning towards building the uber-jar from source

@sodre For reasons I stated above, I recommend against rebuilding it. Java central design goal ("Write once, run anywhere") is to obviate the need to do that.

look how the Apache NiFi project had to create their own archive format ("nar")

Whoa, I guess that is a separate thing even from the NAR Maven plugin.

Java programs often don't agree on versions.

Could you elaborate on what you mean by that? Are you talking about how only one version of each class should be on the system classpath? And systems like OSGi that provide class loader hierarchies to achieve multiple simultaneous versions at runtime? Or do you mean something else? Any links for further reading?

I agree with @kalefranz that we need to start with conda packages

I understand the rationale. What I'm afraid of is the code paralysis effect where once there are a bunch of Java packages on conda, backwards compatibility becomes paramount, and the mechanism behind how Java fundamentally works on conda becomes extremely painful to change. We see this all the time in tech: phase 1 = "I know I'm reinventing a wheel, but this is just an exploratory prototype" followed by phase 2 = "I added a bunch of features my growing user base wants since I control all the code" and finally phase 3 = "we can't reconcile with other projects because it would break backwards compatibility for my thousands of users." I'm not saying this would definitely happen here—the old approach of wrapping Java packages explicitly could be deprecated but continue to work, for example—but I hear alarm bells in my head. In many ways, now is a perfect time to attempt a true cross-repository integration.

I guess I should put my money where my mouth is and try forking conda itself and adding some automagical Java support. Then we'd have a starting point for further discussion. Unfortunately, the earliest I could work on something like that would be July. And if you all think I'm totally crazy here then I won't pursue it.

[jakirkham]

It may be crazy or it may be genius. It's really hard to say until someone tries it and discovers what dragons may lay hidden. So 👍 for trying and seeing what happens.

FWIW it's only after repeated efforts trying this with Python packages on PyPI that we are anywhere close to seeing this happen correctly in conda. Expect that Java has fewer problems than the Python ecosystem does in this regard. Though I say this mostly being an outsider of the Java world these days.

As to the creating conda packages in the interim, I'm far less pessimistic. Especially if we pick a uniform way of doing these packages. This approach was applied when porting R to conda-forge and not only did it make it much easier to review and add conda packages, but it has been incredibly useful in shaping discussions about how these packages might be changed and updated.

Some thoughts on how to proceed. Adding a skeleton for Java packages to conda-build would be a good first step for this and likely easier to start with. Also taking a look at conda pipbuild in all of its former glory. Should provide some inspiration for something similar on the Java front. Maybe @kalefranz has some more recent scripts along these lines that could be helpful.

[sodre]

@sodre For reasons I stated above, I recommend against rebuilding it. Java central design goal ("Write once, run anywhere") is to obviate the need to do that.

I agree that at the steady state we may not need uber-jars. In the meantime I think we are better off by allowing applications to be selfcontained than to risk two applications conflicting with each other over dependencies, e.g. how to install two packages that depend on conflicting versions of third one.

This uber-jar/fat binary is how all the go applications that I have packaged in conda function. We can argue the merits of that either way, but this is a "modern language" that chose to statically compile all its dependencies, and it can cross compile to all target architecture/os conda supports.

Whoa, I guess that is a separate thing even from the NAR Maven plugin.

I see you wrote/contribute to the plugin, which is awesome! Here is the link to Apache NiFi page describing their NAR files: https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions. Are they the same?

Could you elaborate on what you mean by [Java programs don't often agree on versions]? Are you talking about how only one version of each class should be on the system classpath? And systems like OSGi that provide class loader hierarchies to achieve multiple simultaneous versions at runtime? Or do you mean something else? Any links for further reading?

Yes, I meant systems like OSGi that realized different products/plugins depend on different versions of commonly available libraries.

I guess I should put my money where my mouth is and try forking conda itself and adding some automagical Java support. Then we'd have a starting point for further discussion. Unfortunately, the earliest I could work on something like that would be July. And if you all think I'm totally crazy here then I won't pursue it.

I agree with @jakirkham, it could go either way 👍. With that in mind, consider the need to reproduce/install packages and their dependencies in more restricted environments. I think a solution should continue to leverage Anaconda.org as the place to download conda packages.

[kmader]

@ctrueden this is slightly off topic here, but for the 'simple' use-case it would be nice to import and use batteries-included fiji functions as if they were normal python packages without having to worry about versions, dependencies and maven. So a monthy date-stamped conda package for fiji would make it easy to use the distance map or thickness map functionality within a python script or notebook and barely realize that java is behind it. For specific packages and dependencies probably a more complicated approach makes sense.

[kalefranz]

Also slightly off topic... For the Java people here new-ish to conda, how do the conda open-jdk packages work for you? I believe we have environment variables set during activate, so the best experience probably comes from properly activating and deactivating environments. Mostly just interested in thoughts, feedback, and any suggestions on improving the foundational infrastructure and tooling.

[jakirkham]

Returning to the topic at hand. Normally when thinking of a problem (particularly in conda-forge) we try to tackle it with the understanding that there are different needs and time scales at play.

1. What do we do about in the near term (e.g. days)?
2. How do we standardize and disseminate this in the intermediate term (e.g. weeks, months)?
3. What is our vision for the long term (e.g. months, years)?

In the near term, we have users that want to package things using Java. Case in point PR ( https://github.com/conda-forge/staged-recipes/pull/5930 ). Certainly ImageJ and Fiji are another case. What can we offer them that can be codified into a standard? How can this be designed to be consistent with our views on future standards and future tooling?

Of course once we have a few Java packages available, other needs will arise regarding ways we might combine them in new downstream dependencies. If there is a need to port more Java packages, how should those people be advised? How can we ensure this can be standardized not only to ease integration, but to allow for a smooth transition into our long term solution.

Finally what is our grand vision of Java packaging? This is mostly what we have discussed at this point.

IMHO having solutions for these phases is how we avoid problems like this one...

...the code paralysis effect where once there are a bunch of Java packages on conda, backwards compatibility becomes paramount, and the mechanism behind how Java fundamentally works on conda becomes extremely painful to change.

If we have already done a bit of designing on a smaller scale by creating best practices, recipe templates, naming schemes, maybe even a conda skeleton mvn option, we can guide the process with an eye towards standardization that doesn't break everything when our grand vision can be more easily materialized (e.g. conda mvnbuild ;).

[sodre]

I did a quick review of the conda-forge packages that have openjdk as a dependency. Here is what I found in alphabetical order:

• conda-forge/antlr-feedstock:

  • lib/antlr.jar
  • Uber-jar built from source using maven

• conda-forge/beakerx:

  • site-packages/beakerx/kernel/{sql,java,sparkex,base,scala,groovy,clojure,kotlin}/lib
  • Versioned-jars that are prebuilt before the package it uploaded to pythonhosted.

• conda-forge/bfg-feedstock:

  • lib/bfg.jar,
  • Uber-jar copied from maven.org

• conda-forge/ixmp-feedstock:

  • lib/python3.6/site-packages/ixmp.jar
  • Jar file is committed to upstream github repository
  • Also includes other .jar dependencies with shared-libraries committed to the repo.

• conda-forge/knit-feedstock:

  • /lib/python3.6/site-packages/knit/java_libs/knit-1.0-SNAPSHOT.jar
  • Uber-jar built during setup.py using maven.
  • Knit is currently of Anaconda's Dask project.

• conda-forge/maven-feedstock:

  • opt/maven/lib, and opt/maven/lib/ext
  • Prebuilt versioned-jars, e.g. /opt/maven/lib/commons-cli-1.4.jar

• conda-forge/orekit-feedstock:

  • /lib/python3.6/site-packages/orekit/
  • Prebuilt versioned-jars.

• conda-forge/pyferret-feedstock:

  • lib/{threddsBrowser,toolsUI}.jar
  • I installed pyferret and could not find any jars under the environment, so maybe it is just a mistake.

• conda-forge/scijava-jupyter-kernel-feedstock:

  • opt/scijava-jupyter-kernel/
  • Versioned-jars built from source using maven, e.g. /opt/scijava-jupyter-kernel/commons-cli-1.4.jar

---

I may edit this post to add other packages or add relevant facts.

[hanslovsky]

FWIW, I am currently working on porting jrun to Python. Hopefully that can be used independent of operating system (I only have a Linux machine). My hope is that this will make distributing Java applications via conda very easily. Effectively, developers would only need to create a conda package that wraps a call to jrun. I will post updates of my progress.

[hanslovsky]

FYI, I created a PR with my Python port of jrun:
https://github.com/ctrueden/jrun/pull/12
Also, I created a preliminary jrun conda package on my conda channel as well as an example of how I think Java main class entry points can be distributed:

conda install -c hanslovsky jrun paintera-conversion-helper
paintera-conversion-helper -h

If you do not have maven and/or openjdk on your system, you can get them from conda-forge (I chose not to add them as a dependency to jrun).

[hanslovsky]

As far as I can tell, there is no JavaFX package in default or conda-forge. Without a JavaFX package in conda-forge, users will have to use their system java/javafx:

1. Distribution less simple: especially if users are not familiar with installing Java on their system (instead of a single conda install).
2. Users of relevant packages (e.g. https://anaconda.org/hanslovsky/paintera) must not install the openjdk package from conda-forge.

These are good reasons to introduce a JavaFX package in conda-forge, which will require some effort. If people have interest and resources, please step up on this. For starters, I tried to follow the OpenJDK build instructions but I failed miserably. I do not have any of the OSes that they have instructions for, though.

[hadim]

FYI: it might be possible that in the future JavaFX is distributed on Maven (instead of bundled with a Java distribution): http://mail.openjdk.java.net/pipermail/openjfx-dev/2018-July/022088.html

That being said, it's not gonna happen in the short term.

[hanslovsky]

Good point @hadim I wish that had happened already, then this would be a non-issue.

[hadim]

As conda-forge uses Docker and Centos 6 to compile things I would suggest you to try building JavaFX directly in a Docker instance. (but then what about OSX and Windows?).

[hanslovsky]

FYI I am not going to put more time into this at the moment. I will just add a note to affected packages that users should use system java (jrun conda package does not have openjdk as dependency).

[kalefranz]

FYI ecosystem namespaces are happening in conda! PR is nearing completion. Will land in either 4.6 or 4.7. Nothing you need to do to change packaging at the moment. Probably just keep prepending java- to the package name and make sure open-jdk is listed as a dependency.

https://gist.github.com/mcg1969/da5aec380d2ed083b79ddcf151ca16f1

https://github.com/conda/conda/pull/7446

[scopatz]

Hi @kalefranz - this is pretty exciting! However, it might have interesting implications for the bot and creating and storing historical versions of the graph. Will the rendered meta.yamls contain the fully expanded / qualified package names?

[ctrueden]

An update on the jrun front: it was renamed to jgo, to avoid a clash with an existing PyPI project of that name (which executes Jupyter notebooks from the CLI). @hanslovsky and I have released version 0.1.0 of jgo to PyPI. We also released a package called scyjava which is a union of jgo + pyjnius allowing easy import and usage of Java code from remote Maven repositories in Python. The plan is to submit jgo and scyjava also to conda-forge in future; in the meantime, from our perspective, packaging of Java components with Python code is now "solved", without requiring Java components to be repackaged as conda packages. Here is an example from the scyjava documentation which invokes ImageJ from Python:

>>> import scyjava_config
>>> scyjava_config.add_repositories({'imagej.public': 'https://maven.imagej.net/content/groups/public'})
>>> scyjava_config.add_endpoints('net.imagej:imagej:2.0.0-rc-65')
>>> import scyjava, jnius
>>> System = jnius.autoclass('java.lang.System')
>>> System.setProperty('java.awt.headless', 'true')
>>> ImageJ = jnius.autoclass('net.imagej.ImageJ')
>>> ij = ImageJ()
>>> formula = "10 * (Math.cos(0.3*p[0]) + Math.sin(0.3*p[1]))"
>>> blank = ij.op().create().img([64, 16])
>>> sinusoid = ij.op().image().equation(blank, formula)
>>> print(ij.op().image().ascii(sinusoid))
,,,--+oo******oo+--,,,,,--+oo******o++--,,,,,--+oo******o++--,,,
...,--+ooo**oo++--,....,,--+ooo**oo++-,,....,,--+ooo**oo++-,,...
 ...,--++oooo++--,... ...,--++oooo++--,... ...,--++oooo++-,,...
   ..,--++++++--,..     ..,--++o+++--,..     .,,--++o+++--,..
   ..,,-++++++-,,.      ..,,-++++++-,,.      ..,--++++++-,,.
    .,,--++++--,,.       .,,--++++--,,.       .,,--++++--,..
    .,,--++++--,,.       .,,-+++++--,,.       .,,-+++++--,,.
   ..,--++++++--,..     ..,--++++++--,..     ..,--++++++-,,..
  ..,,-++oooo++-,,..   ..,,-++oooo++-,,..   ..,,-++ooo+++-,,..
...,,-++oooooo++-,,.....,,-++oooooo++-,,.....,,-++oooooo+--,,...
.,,,-++oo****oo++-,,,.,,,-++oo****oo+--,,,.,,,-++oo****oo+--,,,.
,,--++o***OO**oo++-,,,,--++o***OO**oo+--,,,,--++o***OO**oo+--,,,
---++o**OOOOOO**o++-----++o**OOOOOO*oo++-----++o**OOOOOO*oo++---
--++oo*OO####OO*oo++---++oo*OO####OO*oo++---++o**OO####OO*oo++--
+++oo*OO######O**oo+++++oo*OO######O**oo+++++oo*OO######O**oo+++
+++oo*OO######OO*oo+++++oo*OO######OO*oo+++++oo*OO######OO*oo+++

Of course, this does not solve the non-Python-centric "install Java packages via conda" question. But it bridges the two worlds in a fairly convenient and extensible way.

[coderextreme]

Looking to install jar http://www.web3d.org/specifications/java/jars/X3DJSAIL.3.3.full.jar in a package x3dpsail. Have no familiarity with creating conda packages. Link to best page for instructions on how to achieve this are welcome! I will look in staged-recipes now that i see it

[ctrueden]

@coderextreme My advice would be: A) get x3djsail deployed to a Maven repository as an artifact; and then B) access its API from Python via the scyjava approach.

Edit: I see you accomplished part (A) already 😄

[coderextreme]

No, we've never deployed that jar. We have a fat jar created through ant which contains all java dependencies. Old habits and build scripts die hard. That is, the primary maintainer has been doing ant scripts for a very long time. Thus we have another package here

[beenje]

Hello! I'm looking into packaging some java applications with conda. I was wondering if some best practices have emerged since this issue was opened?

The app has a few jar files as dependencies. I'm planning to make one recipe per jar.
Is it ok to download directly an existing jar file or should we try to compile it in the recipe?