Openrefine: Provide portable package for Windows

[wetneb]

Is your feature request related to a problem or area of OpenRefine? Please describe.
It would be nice to ship a "portable" version of OpenRefine, which would ship with a JRE in it, so it can be run easily without installing anything as an administrator.

Describe the solution you'd like
Launch4j makes it possible to embed a JRE, so we could add another execution in the Maven profile to generate this.

Additional context
Suggested on the mailing list:
https://groups.google.com/forum/#!msg/openrefine/ruVv7z4SThA/qs8qidYNBQAJ

[stragu]

I see this as a major improvement that would vastly facilitate adoption of OpenRefine. Modifying a JRE environment variable seems to be an arcane / scary process for many Windows users.

[wetneb]

https://twitter.com/psychemedia/status/1230237313445310464

[thadguidry]

I think this package enhancement should be completely self-contained by default. In other words, use the JRE supplied with it by default, but not exclusively. If a user wants to use a JRE they have installed, then they make changes to the config somewhere easily to point to their own JRE they want to try to use.

[Shubhangi2901]

Hi @wetneb and @thadguidry I would like to take this up!

[thadguidry]

@Shubhangi2901 Sure, but we need to wait for @wetneb to respond and confirm if using Launch4j is the only sensible way to do this (since in the past we talked about possibly removing Launch4j, so unsure of other OS impacts, like MacOS. He had investigated some of that at one point)

[wetneb]

As far as I am aware, Launch4j is still a good pick for this. JavaPackager (#2107) was only included in Java 8 so migrating to it is probably not very future-proof. If anyone is aware of better packaging tools for Windows, let us know here :)

[thadguidry]

@wetneb So I dug around and asked some previous co-workers how we packaged for customer installations on Windows. They said Inno Setup. I found a Medium article (have a subscription) that I've zipped up here, but I think this is overkill honestly:
java-packager-with-jdk11.zip

So looked around further, and it seems that OpenJDK community has invested time in this now with an updated jpackage tool (non-GUI, only CLI, but I think that's ideal, no muss, no fuss) and it is now officially integrated into JDK 14

So I think looking at jpackage is the right move forward in 2020.
And we can still give users choices and options during installation, just with text CLI based (like the good old days, simple, portable).

[wetneb]

The problem with jpackage (beyond the fact that it's only available in JDK 14) is that it does not support cross-compilation… Citing the JEP:

There will be no support for cross compilation. For example, in order to create Windows packages one must run the tool on Windows. The packaging tool will depend upon platform-specific tools.

That's pretty sad, at the moment we can do almost all of our packaging cross-platform, so migrating to that would come at a big cost (for instance to create nightlies).

[thadguidry]

@wetneb GitHub Actions virtual envs for Linux, Windows, Mac :-)
https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners
Where jpackage would have to be available, indeed, like so:

with:
    java-version: '>=14-ea' # The JDK version to make available on the path.
    java-package: jdk # (jre, jdk, or jdk+fx) - defaults to jdk
    architecture: x64 # (x64 or x86) - defaults to x64

Then the Maven pom.xml specifying the compilation to maintain the backwards compatibility.
https://help.github.com/en/actions/language-and-framework-guides/building-and-testing-java-with-maven
https://github.com/actions/setup-java

So yes, the cost is a bit of work put into improving our GitHub workflows.

(cross compiling is getting less and less attention these days since virtual envs and CI/CD toolchains have good support... just compile on the native environment! Oh you don't have one? Spin one up on us!)

[Shubhangi2901]

Based on the discussion i am thinking of using jpackage on windows for my installers and will create a pr for the same. Let me know is this is the correct way to move forward and i will replicate the steps on Mac and Linux virtual environments.

[thadguidry]

@Shubhangi2901 I think that would be beneficial to us. Any extra help you can provide would be most welcome even with our generation of nightly builds and GitHub Actions automation. Thanks!

[Shubhangi2901]

Hey, I had a doubt while creating the jar for my project. Is it advisable to include test files in my jar?

[thadguidry]

@Shubhangi2901 As a user of OpenRefine do you care that a developer tested OpenRefine functionality? Yes. Do you care to see those tests? Maybe. Can you see those tests by checking out the source yourself? Yes. Do you need them as part of your installation package to use OpenRefine? :)

[wetneb]

@Shubhangi2901 We do not ship test classes in releases, no.

[Shubhangi2901]

Hi @thadguidry and @wetneb I have successfully used Jpackage to create self contained java installer for open refine. Since it is a windows command I have added it in a separate file and included in the project. Let me know the best way to commit the command to the code base.

[thadguidry]

@Shubhangi2901 I don't believe it! That's awesome! You can submit a PR. If this is your first time, follow our guide here: https://github.com/OpenRefine/OpenRefine/wiki/Your-first-pull-request

[Shubhangi2901]

@thadguidry @wetneb Looks like I was a little early to declare it done. Let me walk you through what I have done.

I created an executable jar initially using eclipse, but it was not working since all dependencies weren't getting copied. Then I created a jar using maven, which works and launches openRefine. Then I used jpackage to create the self contained installer, which I managed to do successfully. However, post-installation, the application doesn't launch for some reason when I open the exe.

Any pointers as to what may be wrong here? Am I supposed to pass some parameters to the executable or something?

[wetneb]

One thing that is quite important for us is that the packaging process can be done independently of any particular IDE. That makes it possible to publish nightly builds automatically, for instance.

So I would encourage you to look at the existing Maven configuration we are using: this uses Launch4j to create a Windows executable. Launch4j has the option to embed a JRE, so it should not be too much work to add that there. To create an installer, you could use generic installer generators (not java specific ones) on top of the archive produced by our current workflows, or find a jpackage integration with Maven.

[Shubhangi2901]

@wetneb @thadguidry Thanks for the suggestions. I managed to get the self-contained package working, and it looks at the JRE which is self-contained. I tested the package on a system with no JRE installed, and its working. I have created a pull request for the issue, which is below. Can you please take a look?

https://github.com/OpenRefine/OpenRefine/pull/2486

[wetneb]

Closed by #2486.

[thadguidry]

@wetneb So... I'm getting around to testing this out now with our released 3.4 beta, and I don't see it respecting my JAVA_HOME and PATH defined on Windows 10 and using that as a priority VM to init, instead, it's ALWAYS using the embedded Java 1.8 (version 25.191-b12) no matter what, including if I go back and also specify the JAVA_HOME=E:\Java\openjdk-13.0.1_windows-x64_bin\jdk-13.0.1 in the refine.ini

I am perfectly OK with that, since this is a completely portable package, but I thought we said we would be able to allow users to still change the JRE even with the embedded version? or did I understand incorrectly?

[thadguidry]

Hmm, thinking about this... maybe that's a good thing... we LOCK the Java used on the portable package to the one we provide. If folks want to control or test with another Java version, then they should not use the portable package but our regular release?

[wetneb]

I am perfectly OK with that, since this is a completely portable package, but I thought we said we would be able to allow users to still change the JRE even with the embedded version? or did I understand incorrectly?

It's not clear to me that this is possible with Appbundler at the moment. We do set <bundledJreAsFallback>true</bundledJreAsFallback> in the config but I am not sure this really works.

[thadguidry]

@wetneb yeah, that's what I figured. So... hmm, shall we just call the portable package as LOCKED to embedded JRE and call it good enough for now? and if users what more configuration choices, then they can use the regular version? Sound good?

[wetneb]

Yes, it does not seem critical to me.

By the way, what we have now isn't really "portable" since it still saves user data on the local machine. A fully portable distribution would be something you can put on a USB key, would run out of the box and also save user data in the same directory (such that you can keep it across machines). It would not be very hard to do (just configure the workspace directory in a local subdir) and would probably help a lot of people I think.

[thadguidry]

Sure, and we didn't say it was portable... we only have said it was "embedded Java", so we have the right context now on our website for Downloads.