Elasticsearch: Debian package needs java dependency

[stumyp]

Elasticsearch version: 6.3.1

Plugins installed: []

JVM version: any

OS version Debian or Ubuntu

Description of the problem including expected versus actual behavior:

elasticsearch debian package must list dependency on java runtime.
Both Debian and Ubuntu have the same names for default java runtimes.
current Depends field looks like this:

Depends: bash, libc6, adduser, coreutils

while it should look like something like this:

Depends: bash, libc6, adduser, coreutils, openjdk-10-jre-headless | openjdk-8-jre-headless | java8-runtime-headless | java8-runtime | java10-runtime-headless | java10-runtime

If people want to use different version of java, they can always use update-alternatives command to update default version.

Steps to reproduce:

let's say you have apt sources configured.
simple install never worked, but now it also fails before it even tries to unpack:

root@69db9738979c:/# apt install elasticsearch                                 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  elasticsearch
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 91.4 MB of archives.
After this operation, 143 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/6.x/apt stable/main amd64 elasticsearch all 6.3.1 [91.4 MB]
Fetched 91.4 MB in 4s (22.3 MB/s)        
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package elasticsearch.
(Reading database ... 7273 files and directories currently installed.)
Preparing to unpack .../elasticsearch_6.3.1_all.deb ...
dpkg: error processing archive /var/cache/apt/archives/elasticsearch_6.3.1_all.deb (--unpack):
 subprocess new pre-installation script returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/elasticsearch_6.3.1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

OK, since that never worked, I install desired openjdk version together with elasticsearch, and it used to work before:

root@69db9738979c:/# apt install openjdk-8-jre-headless elasticsearch
<skip long installation log>
dpkg: error processing archive /tmp/apt-dpkg-install-9DAxH4/33-elasticsearch_6.3.1_all.deb (--unpack):
 subprocess new pre-installation script returned error exit status 1
Errors were encountered while processing:
 /tmp/apt-dpkg-install-9DAxH4/33-elasticsearch_6.3.1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

so it still ends with error, also leaving all java packages unconfigured.
It works only if I install java runtime separately, before elasticsearch

This happens because elasticsearch package doesn't have direct dependency on jre. If there is one, package manager would configure jre before trying to unpack elasticsearch.

[tvernum]

@stumyp
This is intentional, see #6163 and #15569
We know that position doesn't suit everybody, but this is an area where we don't think it is possible to meet everyone's requirements and our decision making has settled on the current behaviour.

In the interests of avoiding duplicate issues & discussion, I'm going to close this issue.

[rjernst]

@tvernum Those discussions were 4 and 2 years ago, and some things have changed in that time. For example, starting with jdk10 the oracle packages are now gpl, so it might be easier to depend on them now soon (since we recommend the oracle jdk). I think it is at least worth having another discussion, even if it results in the same outcome as the one from 4 years ago (note the one from 2 years was simply closed, referring to the 4 year old issue).

[elasticmachine]

Pinging @elastic/es-core-infra

[stumyp]

I'm sorry, even if this is intentional I do not agree with this intention, one of the major reasons there are package managers is managing dependencies and resolving them if needed. It was tolerable when install scripts did not depend on existence of java binary, now they are and this is breaking change. All people usually want is to run

apt-get install elasticsearch

on stock Debian or Ubuntu freshly installed system and have it all. Not think of dependencies ahead of time and working around them.

There is instruction on Debian web site how to install desired Java version: https://wiki.debian.org/JavaPackage

I just followed it and made .deb package from Oracle JRE. This is what it has as Provides:

Provides: java-runtime-headless, java2-runtime-headless, java5-runtime-headless, java6-runtime-headless, java7-runtime-headless, java8-runtime-headless

Basically, similar to what I suggested as Depends

Advanced users can always switch to desired version of JRE, in Debian-way. By using packages and alternatives.

[ohaleck]

I totally agree with @stumyp . It took me half of my work day trying to figure this out.

If you really really really want to avoid dependencies on java packages, you can always just check for availability of java libs and/or binaries in the pre-installation script and clearly suggest that they must be installed before deb installation can continue.

[cybafelo]

Yes i totally agree with ohaleck and stumyp - the .deb package should at least provide a proper error - like 'install the jre' - or at least include it in the documentation of installing elasticsearch.

Would have had NO idea if this post didn't exist.

[smari]

Just registering agreement here with @stumyp and others who say this is confusing. Thanks for opening this thread!

[jasontedor]

Thank you all for the feedback, and I am sorry for the difficulties that you are experiencing here. We have thought about this quite hard in the last few weeks and have come to the conclusion to change our stance here on not declaring a Java dependency. It will take for us some time to investigate a solution that works for all the distributions that we support, and we are unlikely to make a change like this in a minor release but we intend to have this solved before our next major release. The solution that we are looking for will have two aims:

• a simple <package manager> install elasticsearch should install a version of Java that we support
• there is an out for experienced/opinionated users to <package manager> install elasticsearch without it installing a version of Java that they do not want (e.g., users that want and only want an Oracle-distributed JDK) (we will probably via an alternatives-mechanism; again, we need to investigate what is possible on all the distributions that we support)

We will also aim to clarify the documentation and to make the error that occurs when you <package manager> install elasticsearch in 6.x without a JDK being installed less obtuse.

Again, thank you so much 🙏 for the feedback and thank you for your patience. We will use this issue to track progress.

[stumyp]

Thank you, @jasontedor
question: do you know how Oracle JDK is usually installed? from tar.gz or rpm/deb?

• If it is deb/rpm then I believe these packages can be listed as dependency
• If it is tar.gz archive, then clearly these customers prefer archives over packages so let them install Elasticsearch from tar.gz as well :smile:

[jasontedor]

do you know how Oracle JDK is usually installed? from tar.gz or rpm/deb?

This is unknown to me. There is no Debian package for the Oracle JDK. So these users would be using the tar.gz but maybe re-packaging it with make-jpkg. I think also there are PPAs that folks use with the Oracle JDK although there are some licensing issues here. 🙈

[Lord-Y]

Hi guys,

Got the same issue. ES package need java. So after installing this one http://www.webupd8.org/2014/03/how-to-install-oracle-java-8-in-debian.html everything's working perfectly. But hell yeah the apt error is not explicit at all. I spent hours on it before seeing the light

[Br3nda]

If elasticsearch needs java, please have it as a dependency in the deb.

I just spent ages trying to work out why elastic search won't even install.

Preparing to unpack elasticsearch-6.4.0.deb ...
dpkg: error processing archive elasticsearch-6.4.0.deb (--install):
 new elasticsearch package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 elasticsearch-6.4.0.deb

fixed with this:

sudo apt install openjdk-11-jre-headless

[jasontedor]

@Br3nda Please see my previous comment. We will change this.

[jasontedor]

Relates #33607

[kaddoshi]

Thank you all for the feedback, and I am sorry for the difficulties that you are experiencing here. We have thought about this quite hard in the last few weeks and have come to the conclusion to change our stance here on not declaring a Java dependency. It will take for us some time to investigate a solution that works for all the distributions that we support, and we are unlikely to make a change like this in a minor release but we intend to have this solved before our next major release. The solution that we are looking for will have two aims:

• a simple <package manager> install elasticsearch should install a version of Java that we support
• there is an out for experienced/opinionated users to <package manager> install elasticsearch without it installing a version of Java that they do not want (e.g., users that want and only want an Oracle-distributed JDK) (we will probably via an alternatives-mechanism; again, we need to investigate what is possible on all the distributions that we support)

We will also aim to clarify the documentation and to make the error that occurs when you <package manager> install elasticsearch in 6.x without a JDK being installed less obtuse.

Again, thank you so much for the feedback and thank you for your patience. We will use this issue to track progress.

Especialistas, um erro que acontece em todo mundo, em todas as linguagens com relação a suporte de Tecnologia, principalmente em LINUX, são as informações incompletas sobre os COMANDOS.
Agradeço pois resolvi meu problema com o comando jasontedor, porém com o comando no terminal correto: <package manager> install elasticsearch
correto: sudo apt-get install elasticsearch

Abs !!!

"Experts, an error that happens worldwide, in all languages ​​with regard to Technology support, especially in LINUX, are the incomplete information on the COMMANDS.
Thanks, I solved my problem with the jasontedor command, but with the command in the correct terminal: <package manager> install elasticsearch
correct: sudo apt-get install elasticsearch
Thank you !!!"

[chmarr]

For RPM packages, at least, if you have a "Requires" on a generic capability, and there's a java already installed that matches that capability, then it won't install any other java. If not, it will install the java that's available to it (not sure what happens when there's choices about OpenJDK and Oracle available)

The requirement could be "java", "java-1.8.0", "jre" or "jre-1.8.0", which both openjdk and oraclejava announces as "provides". (I don't know what determines the use of "java" vs "jre".)

So, this will ensure java is installed, or fetch and install it, and doesn't preclude the administrator's choice of which flavor. It does, however, let rpm-install get the install ordering right, which it can't otherwise.

P.s.: It really would be nice if this was fixed in a patch release... or even just a repackaging of the RPM, which is what "Release" is for :)

[stumyp]

So, instead of fixing it on system level. by building proper packages, you decided to just bundle java version you define into the package? Is it wise to make large enough package even larger, plus not leaving a choice to the customer what version to use?

I think this change is making things worse, not better. I'm not even complaining on how you create you packages (fpm is a nightmare).

This just indicates to me that Elasticsearch is not meant to be integrated into the system.

On more general topic, it is harder to me to defend my choice of Elasticsearch, more and more problems appear and such decisions do not make it easier. I really would switch to something else, if there is an alternative...

[jasontedor]

Is it wise to make large enough package even larger, plus not leaving a choice to the customer what version to use?

You still have a choice, you can override JAVA_HOME and the new packages will respect it.