Elasticsearch: JAVA_HOME CMD enviroment variable ignored when starting ES as service

[agonzalezgaliana]

Bug report

Elasticsearch version: 6.2.4

Plugins installed: None

JVM version:

Private JRE:
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)

OS version:

Windows 7 64bit

Description of the problem including expected versus actual behavior:

I'm currently trying to install ElasticSearch as service using a private JRE different from the one that could be installed in my system (and pointed by the SYSTEM variable JAVA_HOME). In the documentation (here) says that there are 2 ways of configuring the service (by means of set command or system properties). Due to the fact that I don't want to interfere with other applications that may use JAVA_HOME system property I decided to set a CMD JAVA_HOME environment variable and install/start the service.

The JAVA_HOME system property is NOT defined since I want to make sure ElasticSearch uses the private JRE.

Expected behavior
The service is properly installed and started by using the private JRE pointed by the JAVA_HOME CMD environment variable and ignoring the JAVA_HOME system property which may point to a non valid JRE.

Actual behavior
The service is properly installed, however it cannot be started.

C:\elasticsearch-6.2.4\bin>elasticsearch-service.bat start
Failed starting 'elasticsearch-service-x64' service

Steps to reproduce:

1. Open CMD window at elasticsearch-6.2.4\bin
2. Define JAVA_HOME CMD envvar pointing to a private JRE set JAVA_HOME=C:\elasticsearch-6.2.4\jre
3. Install service elasticsearch-service.bat install
4. Start service >elasticsearch-service.bat start

Provide logs (if relevant):
elasticsearch-service-x64.2018-05-18.log

[2018-05-18 14:15:28] [info] [ 6908] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-05-18 14:15:28] [info] [ 6908] Service elasticsearch-service-x64 name Elasticsearch 6.2.4 (elasticsearch-service-x64)
[2018-05-18 14:15:28] [info] [ 6908] Service 'elasticsearch-service-x64' installed
[2018-05-18 14:15:28] [info] [ 6908] Commons Daemon procrun finished
[2018-05-18 14:15:39] [info] [ 6352] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-05-18 14:15:39] [info] [ 6352] Starting service 'elasticsearch-service-x64' ...
[2018-05-18 14:15:39] [info] [17032] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-05-18 14:15:39] [info] [17032] Running 'elasticsearch-service-x64' Service...
[2018-05-18 14:15:39] [info] [10696] Starting service...
[2018-05-18 14:15:39] [error] [10696] Failed creating java %JAVA_HOME%\bin\server\jvm.dll
[2018-05-18 14:15:39] [error] [10696] The system cannot find the path specified.
[2018-05-18 14:15:39] [error] [10696] ServiceStart returned 1
[2018-05-18 14:15:39] [error] [10696] The system cannot find the path specified.
[2018-05-18 14:15:39] [info] [17032] Run service finished.
[2018-05-18 14:15:39] [info] [17032] Commons Daemon procrun finished
[2018-05-18 14:15:40] [error] [ 6352] Failed to start 'elasticsearch-service-x64' service
[2018-05-18 14:15:40] [error] [ 6352] The data area passed to a system call is too small.
[2018-05-18 14:15:40] [info] [ 6352] Start service finished.
[2018-05-18 14:15:40] [error] [ 6352] Commons Daemon procrun failed with exit value: 5 (Failed to start service)
[2018-05-18 14:15:40] [error] [ 6352] The data area passed to a system call is too small.

elasticsearch-service-x64-stderr.2018-05-18.log

2018-05-18 14:15:39 Commons Daemon procrun stderr initialized
The data area passed to a system call is too small.
Failed to start service

elasticsearch-service-x64-stdout.2018-05-18.log

2018-05-18 14:15:39 Commons Daemon procrun stdout initialized

[elasticmachine]

Pinging @elastic/es-core-infra

[rjernst]

I think the issue here is the call to the service manager passes java home unexpanded:

--Jvm "%%JAVA_HOME%%%JVM_DLL%"

I believe the double percent signs around JAVA_HOME mean it will not be resolved until the service is started, and the local JAVA_HOME set before running install will not exist.

@elastic/microsoft Is there any reason this can't be resolved at install time?

[jasontedor]

@rjernst This is a deliberate choice so that the we use the system JAVA_HOME and resolve it at runtime so that a user can upgrade the installed Java without having to reinstall the Windows service.

[codebrain]

The JAVA_HOME variable is evaluated at runtime as per:
https://github.com/elastic/windows-installers/blob/master/src/Elastic.Configuration/EnvironmentBased/Java/JavaConfiguration.cs

[Mpdreamz]

@codebrain the OP is using the zip for service installation not the MSI.

[rjernst]

@jasontedor The docs linked in the original issue comment do state JAVA_HOME may be set locally before installing the service:

The Elasticsearch service can be configured prior to installation by setting the following environment variables (either using the set command from the command line, or through the System Properties->Environment Variables GUI).

While we could update the docs to include an exception for JAVA_HOME, I find it odd that the other environment variables there would not also want the same treatment (eg ES_PATH_CONF or ES_JAVA_OPTS). Is it really that much trouble to run service installation again if changing JAVA_HOME?

[jasontedor]

I would support that change, yes: that is, we would no longer use %%JAVA_HOME%% here and require that if you want to change any environment variables you have to manually manage the service or reinstall, we would only use what is defined at install time.

[agonzalezgaliana]

Hi everybody,

may I ask about the status of this issue?

[agonzalezgaliana]

Tested again with version 6.6.0 with the same results. Would it be possible to know the current status? Is there a plan to fix the issue? Thanks.

[agonzalezgaliana]

Finally we were able to bypass the problem by removing the double percent signs around JAVA_HOME in the call to the service manager. In this way we resolve the path before service starts running.

--Jvm "%JAVA_HOME%%JVM_DLL%"