Triplea: Bundle JRE

[RoiEXLab]

I got reminded of this when seeing #4621 recently.
Starting with Java 9, as you all know the JRE got modular. Since then there's a tool called jlink that basically allows you to merge your JAR file and the installed JRE into a single native executable.
What makes this whole process interesting is that with the modularity the packed JRE image can be shrunk down to a minimum, meaning that parts that are never used by the JAR file don't need to be included in the binary. This can lead to super small JREs (up to factor 10 size reduction) which we could bundle statically with the installer.
I ran the jdeps tool agains the latest headed jar and this revealed that TripleA uses very few JRE modules for its size:

   java.base
   java.compiler
   java.desktop
   java.management
   java.naming
   java.prefs
   java.rmi
   java.security.jgss
   java.security.sasl
   java.sql
   jdk.unsupported

Many of the dependencies are coming from guava or other big libraries TripleA depends on so the list will probably improve over time. (The details are way too complex to list here, I had to increase my terminal limit to be able to see all of them)
For testing purposes I created an image to get a raw file-size estimate for the zipped JRE TripleA ever uses: 46.9MB (Note that zipped only applies to the class files, zipping the rest also reduces file size even further).
But almost 50 MB is still a lot you might say and this is true, so what are the benefits to gain?
The probably biggest benefit is that we could ship a fixed java version. For the users this means less problems with certificates (although the AV problem will probably exist nevertheless), and for us this means we could use newer language features without having to constantly be worried that users didn't upgrade their java. Also using newer APIs will allow us to remove a lot of "compatibility code" we built to make legacy APIs work with Java 8+ objects. Those are the kind of issues that constantly remind me that using newer java versions is a desirable thing.
There are some pre-conditions though: The most notable one is that we'd have to convert TripleA to an explicit module, so for each gradle-subproject we need to analyze what parts of the JRE are required and import them explicitly using module-info.java. After that we could give the whole system a shot. Note that we need to import stuff that is required by our dependencies as well, because those components don't use explicit modules yet.

But for now I'd just like to hear your opinion on bundling the JRE directly. Apart from filesize, is there anything else to consider?

[ron-murhammer]

I think its a good direction in general as we've always had a lot of issues with users having old/buggy/multiple JREs. I think getting to a point where the download contains everything would be great. As long as the size of the download doesn't increase too much, I'm all for this.

[panther2]

@RoiEXLab
Personally I would always prefer to install unbundled versions to avoid redundancies on my system. But I see the benefits of this approach for the gaming community.
Just one question: Based on the experience that the update-"willingness" and frequency of users is very, very volatile - how would such an approach affect compatibility between different versions over time?

[RoiEXLab]

@panther2 Interesting question, I haven't actually considered compatibility.
When I think about it, this will work just like engine compatibility. If a java upgrade would introduce an incompatibility (Although unlikely) we'd have to stay with a certain version until we make a breaking release. But in this case we have 100% control over the used version so users can't accidentally use an incompatible version themselves, which is a pro argument.
For power-users we could of course offer a jre-free version, but if the download size is reasonable, there's no need for it IMO: As of java 9 we can create a minimal image that just brings the features we need without any extras, as already stated above.

[ssoloff]

Agreed. I kind of get the impression this is what we should be doing anyway. Even if, from Java 11 and onward, the JDK is no longer a behemoth, it just feels dirty asking users to download a JDK to run the application.

For power-users we could of course offer a jre-free version

I was going to suggest this, as well. As has been discussed in #4358, there are several use cases for it. Although, I agree that if the bundled JRE size is relatively small, it won't be necessary. Even if we do offer JRE-free versions, I'd suggest restricting it to just the portable installers.

[RoiEXLab]

@ssoloff Thanks for your answer.
Any chance somebody has #4621 on their tracklist? Splitting JavaFX will reduce the current artifact size by a reasonable amount already.
I would do it, but I don't have much time this week to look into this (rather large for me at least) task. Creating a new subproject is one thing, but configuring the build pipeline to publish the new artifact as well is another thing to consider. I haven't done any of both tasks, so it will take me a little bit longer (too long for this week, maybe even too long for the next month) to do that myself.

[panther2]

@ssoloff

Even if we do offer JRE-free versions, I'd suggest restricting it to just the portable installers.

I would agree, if you please reminded me on how to use those on Windows. I remember years ago (when I actually had been using those versions) there was a *.bat for Windows to launch TripleA. I am missing any Windows related files in the zip-releases nowadays. I am aware that I could simply doublecklick the *jar inside the bin-directory but that would not start TripleA with the recommended parameters of the .vmoptions-file.

[ssoloff]

if you please reminded me on how to use those on Windows

@panther2 Here's all you need to do:

1. Unzip the portable installer someplace.
2. Open a Command Prompt in that directory.
3. Run java -jar bin\triplea-game-headed-<version>-all.jar, where _\_ is replaced with the version of TripleA you downloaded (e.g. 1.10.14109; given in the zip file name).

You can make that last step easier by using tab completion in your command prompt. Simply type java -jar bin\ and then press the Tab key. It will fill in the rest of the jar file name for you.

[panther2]

@ssoloff Thank you. But isn't this exactly the same as doubleclicking the *.jar-file?
How does it load the vmoptions, then?

[RoiEXLab]

@panther It's almost the same as double clicking, the only difference is the folder the command is being executed in (I don't think this changes any behaviour, but it makes a difference for the code).
The a vmoptions file doesn't exist for the portable installer: You'll have to specify the arguments directly before -jar i.e. java -Xmx2G -Xms2G -jar bin/something to apply them

[ssoloff]

@panther2 What @RoiEXLab said. :smile:

Regarding the current working directory difference between the command I listed above and double-clicking the jar... That may no longer be a problem with the introduction of the _.triplea-root_ file and associated changes. I would suggest trying it out in your regular gameplay for a while and see if you run into any issues. It may be that double-clicking the jar of the portable installer is now a viable launch option.

[panther2]

Thank you @ssoloff and @RoiEXLab !

[DanVanAtta]

For power-users we could of course offer a jre-free version

Is it actually power-users that have java already installed? My impression, just my impression, is that most people have java installed already for one reason or another.

I've been wondering if it makes sense to have the game engine become smarter about java version and upgrading. We download maps, I was thinking we could also start to download updated game engine versions (possibly in background) and use the process runner util to execute it once downloaded.

As part of this, we could have the game check java version and download a bundled or non-bundled version as appropriate. To have users download the right version in the first place, I think that is probably solved at the download screen, perhaps we can err to use the bundled version for download and then the upgrade path could perhaps use a non-bundled.

[RoiEXLab]

@DanVanAtta I think your proposal is either going to far, or not far enough:
If we want a mechanism to manage JREs based on the users system I think we should go one step further and create a standalone launcher software.
This way we can eliminate the (kinda broken) update mechanism from the engine itself and ensure users are always using up-to-date versions.
I haven't thought this idea entirely through, but it's a start.

[DanVanAtta]

I was wondering/considering to have a launcher or installer app that would be first downloaded and that would be first launched. That app would install the needed version of the game.

These two items made me not suggest it:
A) the game already does the update check, enhancing that to be able to download is essentially already being suggested.

B) The installer app presumably would be written in java, which would also need a bundled JRE. The game we've worked down to under 20MB, the JRE is potentially going to be the dominant part of the download. My first impression is it makes sense to just download 50MB of content rather than downloading 30MB to then then download either 20MB or 30MB more.

In other words, I'm proposing that we bundle the JRE for first time downloads by default and have the game do a little bit more when it comes to the update check and actually download the next version.

[DanVanAtta]

@RoiEXLab to your original research, this is an excellent topic. I think we will have a use-case for both static and non-static bundles. Anything that shrinks a download file size is a good thing. It's an open question whether the download of choice would be a bundled or non-bundled version, but that question is a bit irrelevant to this topic. We will have use for a smaller bundled JRE.

[RoiEXLab]

@DanVanAtta I'm fine with a lot of options as long as we can at some point soon start using java 11 or later for TripleA ^^
Recently I noticed a lot of use cases where newer apis allow us to drop a lot of code.

[DanVanAtta]

JRE is now bundled!
https://github.com/triplea-game/triplea/pull/4876

Marked as "not shared" and static, so install4j will always hopefully install the bundled JRE each time and for each install of each version.