Sdkman-cli: Improve handling of Java candidate

[vpavic]

As discussed on Gitter in sdkman/user-issues channel, it would be nice if SDKMAN could provide some special handling for Java candidates in order to address some of the complexity that comes from presence of different JDK flavors, as well as different kinds of _current_ releases (LTS vs non-LTS).

For instance, I have a preference for Zulu JDKs and generally don't need/want JDKs from other vendors. SDKMAN allows to install individual Zulu JDKs and use default command to set the local default, however when using upgrade command SDKMAN will offer to install candidate that is marked as default on the remote - so even though I have the same or even newer release from a different vendor (note that at the time of writing this, Java candidate does not seem to have any candidate marked as default on the remote).

One possible solution would be to have different candidates for each JDK vendor (e.g. java-open, java-zulu, java-adpt-hs, java-adpt-j9, etc.). This would perhaps be more natural as for other candidates (to my knowledge) there is a 1:1 mapping between the candidate and the vendor, whereas for Java it is 1:n due to multiple vendors contributing to Java candidate. This would however add complexity in other places, as it's unclear how JAVA_HOME would behave when there are multiple Java candidates installed.

[vpavic]

The expectations could be expressed as:

• given I have a JDK from vendor X installed, when upgrade command is used SDKMAN should offer to install a new release from vendor X (if exists)
• given I have a JDK from vendor X installed, when upgrade command is used SDKMAN shouldn't offer to install a release from vendor Y (or any other vendor that isn't X)

[marc0der]

I think all the above observations are very valid, but are limited to a single facet (upgrading) of the scope of the improvements I was intending to make for the Java candidate. My thoughts are that in the case of Java, _some animals are more equal than others_ on sdkman :smile:. With that in mind, I would like to let JDKs work a bit differently than other SDKs, granting some concessions to how sdkman handles JDKs.

Here are some of the ideas I had:

• provide a custom list view for java versions on sdk ls java, perhaps having it's own template rendered on the serverside with neat columns and perhaps sections per vendor.
• introduce a JDKs page to the website (will have details about each vendor, current LTS version etc)
• extend the base commands to include a vendor qualifier, so allowing something like sdk install java [vendor] [version]. it would also work for the set and default commands. we might need to think this through a bit more.
• as mentioned above, we could introduce a notion of _pinning_ a JDK vendor, so that upgrades would be applied for a given vendor only. This could be done by using the default command or even introducing a brand new top-level command called pin that would only work for Java.

Regarding introducing candidates for each java/vendor combination would break a lot of things and I think pollute our domain somewhat. I would prefer keeping _candidate_ pure and introducing a new domain concept for _vendor_ instead. We already pushed the limit by overloading _version_ to include the _vendor_, but shifting it to _candidate_ would be equally wrong in my opinion. If we gave vendor it's own domain concept, it would also make the above functionality a lot easier to implement. wdyt?

[kevin-stuart]

Just a thought: Maybe just adding vendor could not be enough for the future and this is an opportunity to make things more future-proof. You now only add one new dimension "vendor". But maybe in the future, there are more fine-grained splits like multiple GraalVM vendors.

I would suggest thinking about something hierarchical like a file system, which has served Unix very well. For example, you could an "oracle" flavor and add have OpenJDK and GraalVM as possible children. The grouping could also be different to have the type like OpenJDK first and then the vendor. Or you could support both. It could also be used to group variants of SDKs better, for example for Zulu you also have a version with JavaFX or you have versions with different GCs like j9 or Hotspot you might want to use.

I know this sounds complicated and this definitely needs more thought, but I just wanted to throw out this idea and start some brainstorming. If I say sdk install java/openjdk/zulu/fx version and would like to pin the variant java/openjdk/zulu/fx I would only get new releases for Zulu with JavaFX. And I might want to have a parallel install of GraalVM from Oracle, which I also want to keep up to date.

I think it might be important to think about GraalVM now if it exists in parallel with OpenJDK now, the other option could be to create a new variant graalvm instead of java, but the downside is that GraalVM clashes with java in PATH, so the current approach makes sense.

[marc0der]

Improve handling of Java candidate

[marc0der]

I've fixed some bugs in the Java list view around displaying of _Unknown_ vendors (locally installed versions that don't belong to a specific JDK vendor).

[vpavic]

I was about to ping you @marc0der about displaying local installs, so thanks for taking a look at that. However, latest changes have broken the listing for me:

$ sdk ls java


<!DOCTYPE html>
<html lang="en">
    <head>
        <title>Error</title>
        <link rel="shortcut icon" href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAlFJREFUeNqUU8tOFEEUPVVdNV3dPe8xYRBnjGhmBgKjKzCIiQvBoIaNbly5Z+PSv3Aj7DSiP2B0rwkLGVdGgxITSCRIJGSMEQWZR3eVt5sEFBgTb/dN1yvnnHtPNTPG4PqdHgCMXnPRSZrpSuH8vUJu4DE4rYHDGAZDX62BZttHqTiIayM3gGiXQsgYLEvATaqxU+dy1U13YXapXptpNHY8iwn8KyIAzm1KBdtRZWErpI5lEWTXp5Z/vHpZ3/wyKKwYGGOdAYwR0EZwoezTYApBEIObyELl/aE1/83cp40Pt5mxqCKrE4Ck+mVWKKcI5tA8BLEhRBKJLjez6a7MLq7XZtp+yyOawwCBtkiBVZDKzRk4NN7NQBMYPHiZDFhXY+p9ff7F961vVcnl4R5I2ykJ5XFN7Ab7Gc61VoipNBKF+PDyztu5lfrSLT/wIwCxq0CAGtXHZTzqR2jtwQiXONma6hHpj9sLT7YaPxfTXuZdBGA02Wi7FS48YiTfj+i2NhqtdhP5RC8mh2/Op7y0v6eAcWVLFT8D7kWX5S9mepp+C450MV6aWL1cGnvkxbwHtLW2B9AOkLeUd9KEDuh9fl/7CEj7YH5g+3r/lWfF9In7tPz6T4IIwBJOr1SJyIGQMZQbsh5P9uBq5VJtqHh2mo49pdw5WFoEwKWqWHacaWOjQXWGcifKo6vj5RGS6zykI587XeUIQDqJSmAp+lE4qt19W5P9o8+Lma5DcjsC8JiT607lMVkdqQ0Vyh3lHhmh52tfNy78ajXv0rgYzv8nfwswANuk+7sD/Q0aAAAAAElFTkSuQmCC">
        <style>
            html, body, pre {
                margin: 0;
                padding: 0;
                font-family: Monaco, 'Lucida Console', monospace;
                background: #ECECEC;
            }
            h1 {
                margin: 0;
                background: #A31012;
                padding: 20px 45px;
                color: #fff;
                text-shadow: 1px 1px 1px rgba(0,0,0,.3);
                border-bottom: 1px solid #690000;
                font-size: 28px;
            }
            p#detail {
                margin: 0;
                padding: 15px 45px;
                background: #F5A0A0;
                border-top: 4px solid #D36D6D;
                color: #730000;
                text-shadow: 1px 1px 1px rgba(255,255,255,.3);
                font-size: 14px;
                border-bottom: 1px solid #BA7A7A;
            }
        </style>
    </head>
    <body>
        <h1>Oops, an error occurred</h1>

        <p id="detail">
            This exception has been logged with id <strong>7ccin52ja</strong>.
        </p>

    </body>
</html>

Here are my Java candidates:

$ tree -L 1 .sdkman/candidates/java/
.sdkman/candidates/java/
├── 11.0.3.hs-adpt
├── 11.0.3.j9-adpt
├── 11.0.3-zulu -> java-11-hotspot
├── 12.0.1.hs-adpt
├── 12.0.1.j9-adpt
├── 19.0.2-grl
├── 7.0.222-zulu
├── 8.0.212.hs-adpt
├── 8.0.212.j9-adpt
├── current -> /home/vpavic/.sdkman/candidates/java/11.0.3.hs-adpt
├── java-11-hotspot -> 11.0.3.hs-adpt
├── java-11-openj9 -> 11.0.3.j9-adpt
├── java-12-hotspot -> 12.0.1.hs-adpt
├── java-12-openj9 -> 12.0.1.j9-adpt
├── java-7-hotspot -> 7.0.222-zulu
├── java-8-hotspot -> 8.0.212.hs-adpt
└── java-8-openj9 -> 8.0.212.j9-adpt

[darek1024]

@vpavic the same happens for me, I want to install Java 12 but even don't know which versions are available 😔

[marc0der]

Just for the record, this was due to some faulty data in the database that caused the rendering of the Java list view to misbehave. I've also put some other safeguards in place. @vpavic can you confirm that everything is working for you again?

[vpavic]

Yes @marc0der, I can confirm it's all good now. :+1:

[tsabirgaliev]

This would also let me register jdk home symlinks in my IDE and update process wouldn't break my IDE workflow

[rgoldberg]

NB: the syntaxes / terms suggested herein can obviously be changed if people prefer other options.

<vendor> should be renamed <dist> (short for distribution).

<dist> support should be general (i.e. not tied to java), so that it can be used when necessary for other candidates.

if <dist> is not included in the command line for a candidate that supports dists, then sdkman can use some multi-distribution upgrade order similar to what it does now for java (though the current ordering for java doesn't make sense, so it should be improved).

sdk dist should list all the candidates that support dists.

sdk dist <candidate> should list all the dists supported by <candidate>

If you don't want to redo command line parsing by supporting an extra <dist> argument, you could support dists by appending - then <dist> to <candidate>, e.g.:

sdk <command> [<candidate>[-<dist>]]

Instead of having <dist> as a separate argument, e.g.:

sdk <command> [<candidate> [<dist>]]

I like the idea of multi-axis distribution qualifiers, though maybe they should be unordered labels rather than hierarchical. If hierarchical, you'd have to remember their ordering, plus you couldn't have cross-cutting concerns. e.g., say multiple distributions use the same jvm, and you just care about the latest version of that jvm, not which vendor has packaged it up (I don't know if j9 is available outside of AdoptOpenJDK, but if it is, j9 & hs could both be used by multiple vendors).

Given that labels aren't hierarchical, a syntax like a,b,c is probably preferable to a/b/c.

I'd assign each label to an axis represented by a field, e.g. j9 & hs would both be values for the jvm field for the java candidate. I'd also allow multiple values for the same field to be synonymous (e.g., hs & hotspot) so people can use either more descriptive values or terser ones (plus you can generate auto-documentation by showing that hs = hotspot). Every value should be unique across all values for all fields for the same candidate. e.g., hs couldn't be a value for 2 different fields for the same candidate. This will allow specifying labels without their field name, though a field name should be optional, like [<field>=]value, so jvm=hs & hs would be equivalent.

A label is the application of a field value to a distribution.

* would be a special non-unique value that could only be used after the field name, e.g.: fx=*, which would indicate that the fx label must be present for the dist, but the exact value doesn't matter (which specifies in this case that the java candidate must support JavaFX, but the type of the JavaFX support doesn't matter).

Boolean fields would have the field name be the same as the only value for that field, e.g., long-term support could be requested via lts=lts or just lts. (EDIT: one issue with lts is that it's on a per version basis, not on a per dist basis, so maybe that & other per-version labels should be applied to versions instead of to dists).

Example fields would be: vendor (e.g., oracle, azul, amazon), jvm (e.g., hs, j9, graalvm), fx (e.g., openjfx), lts (lts), etc. name would be a field where each dist is required to have a unique value (whereas more than one dist can be from the same vendor, or use the same jvm, or supply JavaFX, etc.).

So, e.g., if you just want long-term-support versions from openjdk, you could use sdk i java open,lts.

If no dist matches all labels, or if an optional version was specified, but that version doesn't match the specified labels (e.g., sdk i java open,lts 13.0.1), then sdkman should fail with an error.

Allowing specifying on the command line only one value per field would allow sdkman to report a more descriptive error for a situation like a,b,c,j9,d,e,f,hs,g,h,i, because it could say that you must specify at most one of j9 & hs, instead of just saying that no distribution matches a,b,c,j9,d,e,f,hs,g,h,i.

sdk fields <candidate> should list all the fields for <candidate> (and indicate whether they are dist- or version-specific).

sdk values <candidate> <field> should list all the values for <field> for <candidate>.

sdk labels <candidate> <dist> [<version>] should list all the labels (<field>=<value> pairs) applied to dist for <candidate> (if optional <version> is specified, then would list dist-specific & version-specific labels, indicating which labels are dist / which are version).

sdk show <candidate> <dist> will list all dists for <candidate> that match the labels specified in <dist>.

As an addendum, I think that you should also support locking version prefixes so that sdk upgrade only offers updates with the same prefix. e.g., If version 1.2.3 of a candidate is installed, if I just want upgrades for 1.x (not 2.x, 3.x, etc.), then lock @ 1; so, 1.2.3.1, 1.2.4 & 1.3 would all be allowed as upgrades, but 2.0 wouldn't. If I want upgrades for 1.2.x (but not 1.3.x, etc.), then lock @ 1.2; so, 1.2.3.1 & 1.2.4 would both be allowed as upgrades, but 1.3 & 2.0 wouldn't. You should be able to specify multiple locks for a candidate, so, e.g., if I have 11.0.5, 12.0.2 & 13.0.1 , I could lock at 11, 12.0 & 13.0.1 to vary the prefixes per version. The syntax for locking could be:

sdk lock <candidate> [dist] <comma-separated-version-prefixes>

e.g.,:

sdk lock java open 11,12.0,13.0.1

I can open a new issue for version prefix locking, but I figure if you implement dists, you might be refactoring a lot, and knowing about version prefix locking beforehand might allow you to implement dists in a way that makes it easier to add version prefix locking later.

[ksilz]

I have JDK 11 in the OpenJ9 flavor installed. SDKMAN wants to upgrade this to the Hotspot version. I consider this a bug. Does this issue cover "my bug"?

[tsabirgaliev]

FWIW, I switched to Macports OpenJDK builds. Pros: simultaneous up-to-date OpenJDK 8/11/13 discoverable on Macos by /usr/libexec/java_home. Cons: only OpenJDK.

[marc0der]

This isn't really a bug, since sdkman only holds a the notion of a default version. In other words, if you aren't on the default version it will always suggest that one as the preferred version.

The reason is that we shoehorned java (with multiple versions of the same candidate) into how all other candidates currently work.

Once I've reworked how we handle java, these issues should go away.

[rgoldberg]

@marc0der Thanks for looking into this. Any thoughts about my proposal above? I know it was very long. Are there any points I could clarify? Are there any parts that seem good, bad, unnecessary, etc.? Thanks again.

[marc0der]

@rgoldberg Sorry for not getting back to you before regarding your proposal. I'll be very honest to say that I simply haven't managed to work through your entire proposal yet.

My suggestion would be to break the proposal down into smaller, more digestible bits and then raise PRs where you see fit. I'll be honest to say that I barely have time to implement existing feature requests (all done in my spare time), let alone to refactor the entire codebase.

This project was built from the start through a series of small incremental changes, and has so evolved into what it is today. It's not perfect, but it is solid and makes many people's lives easier. It isn't merely a bash script, but it spans a datastore, many microservices and an extensive bash client. Making such huge changes in the underlying domain will take up much time, time which I simply don't have available.

All that said, if you feel strongly enough about your proposal and are willing to put some action behind your words (and of course if I and the others agree with what is proposed), then I would be happy to merge your PRs. I'm also always available on slack, feel free to drop me a note in the cli-development channel if you would like to chat about individual proposal points prior to doing the work.

Hope that helps!

[rgoldberg]

@marc0der Sorry for late reply.

I'm not a bash expert, but I might be able to look into this at some point.

I'm happy to converse via Slack.

I agree that work should be broken down into manageable chunks, but I think that it would be beneficial to have a completely envisioned UX & code refactoring outcome before working on this. I think it would make sense to discuss any UX revamps holistically before looking at the code, then figure out the requisite code refactoring, to determine the effort required for each UX change as well as their interdependencies.

[spencergibb]

Not sure if this is the right place to comment or if I should open a new issue. I'm looking for something similar for the sdk env usage with .sdkmanrc. I'd like to say I want java=8,hs-adpt (ie java 8 from adopt openjdk, or 11 or 13 or whatever major) without worrying about the minor versions. Then I can check that into source control for a project

[DennisBecker]

I also have problems with the upgrade handling. ALso, I wanted to upgrade the java 8 version of Zulu to a newer java 8 version of Zulu, not any version 11, as I have version 11 and 8 installed from Zulu.

[Karasiq]

Same. I need a pattern like "8-zulu-fx", because with a strict minor version specified, i often get errors like "Stop! java 8.0.262.fx-zulu is not available" in my builds because java was updated and the old version removed.

[woj-tek]

I was meaning to open new FR but luckily found this one - I think it would be sensible to split both vendor and handling of the major version (or maybe even some sort of notion of "LTS", but that would complicate things to much I think). IMHO quite often people would like to install some major version of JDK from particular vendor, so something along the lines of @marc0der suggested here:

extend the base commands to include a vendor qualifier, so allowing something like sdk install java [vendor] [version]. it would also work for the set and default commands. we might need to think this through a bit more.

but with the consideration that [version] would be either major version (so sdk install java adpt 11 and it would automatically upgrade all minor versions) or particular, specific version (i.e. sdk install java adpt 11.0.8.hs)