Hydra: [Bug] Hydra installation failing if java is not installed (Docker build)

[lalitpagaria]

🐛 Bug

Description

I am using hydra's master branch and trying to build docker image from python:3.8-slim-buster. The installation of hydra-core is failing which is I presume antlr4 needing java. If using released version 1.0.4 then this issue do not occur.

Checklist

• [X] I checked on the latest version of Hydra (using master)
• [X] I created a minimal repro

To reproduce

Build following Dockerfile -
* Minimal Code/Config snippet to reproduce *

FROM python:3.8-slim-buster

RUN apt-get update && apt-get upgrade -y && apt-get install -y curl git
RUN pip install --upgrade pip
RUN pip install --no-cache-dir git+https://github.com/facebookresearch/hydra.git#egg=hydra-core

* Stack trace/error message *

#7 12.46 Building wheels for collected packages: hydra-core, antlr4-python3-runtime, PyYAML
#7 12.46   Building wheel for hydra-core (PEP 517): started
#7 12.75   Building wheel for hydra-core (PEP 517): finished with status 'error'
#7 12.75   ERROR: Command errored out with exit status 1:
#7 12.75    command: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmp368nyn_3
#7 12.75        cwd: /tmp/pip-install-utmk8rm3/hydra-core_2d93aca6edfb47c6a48f687c3a2baee1
#7 12.75   Complete output (12 lines):
#7 12.75   running bdist_wheel
#7 12.75   running build
#7 12.75   running build_py
#7 12.75   running clean
#7 12.75   Generating parsers with antlr4
#7 12.75   running antlr
#7 12.75   Generating parser for Python3: ['java', '-jar', '/tmp/pip-install-utmk8rm3/hydra-core_2d93aca6edfb47c6a48f687c3a2baee1/build_helpers/bin/antlr-4.8-complete.jar', '-Dlanguage=Python3', '-o', '/tmp/pip-install-utmk8rm3/hydra-core_2d93aca6edfb47c6a48f687c3a2baee1/hydra/grammar/gen/', '-Xexact-output-dir', '-visitor', '/tmp/pip-install-utmk8rm3/hydra-core_2d93aca6edfb47c6a48f687c3a2baee1/hydra/grammar/OverrideLexer.g4']
#7 12.75   /tmp/pip-build-env-5l4uyv22/overlay/lib/python3.8/site-packages/setuptools/dist.py:452: UserWarning: Normalizing '1.1.0dev1' to '1.1.0.dev1'
#7 12.75     warnings.warn(tmpl.format(**locals()))
#7 12.75   ===========================================================================
#7 12.75   | Unable to generate parsers: [Errno 2] No such file or directory: 'java' |
#7 12.75   ===========================================================================
#7 12.75   ----------------------------------------
#7 12.75   ERROR: Failed building wheel for hydra-core
#7 12.75   Building wheel for antlr4-python3-runtime (setup.py): started
#7 13.09   Building wheel for antlr4-python3-runtime (setup.py): finished with status 'done'
#7 13.09   Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.8-py3-none-any.whl size=141229 sha256=9144c973c22ab58faceed86545de9463583cc006044b6f8a7ccb428300281623
#7 13.09   Stored in directory: /tmp/pip-ephem-wheel-cache-xrhpm62p/wheels/c8/d0/ab/d43c02eaddc5b9004db86950802442ad9a26f279c619e28da0
#7 13.10   Building wheel for PyYAML (setup.py): started
#7 13.37   Building wheel for PyYAML (setup.py): finished with status 'done'
#7 13.37   Created wheel for PyYAML: filename=PyYAML-5.3.1-cp38-cp38-linux_x86_64.whl size=44619 sha256=2bba541976b262c690951aeb86e8937f9f19aaa295662b01a2ec2f9fc8314d89
#7 13.37   Stored in directory: /tmp/pip-ephem-wheel-cache-xrhpm62p/wheels/13/90/db/290ab3a34f2ef0b5a0f89235dc2d40fea83e77de84ed2dc05c
#7 13.38 Successfully built antlr4-python3-runtime PyYAML
#7 13.38 Failed to build hydra-core
#7 13.38 ERROR: Could not build wheels for hydra-core which use PEP 517 and cannot be installed directly
------
executor failed running [/bin/sh -c pip install --no-cache-dir git+https://github.com/facebookresearch/hydra.git#egg=hydra-core]: exit code: 1

Expected Behavior

System information

• Hydra Version : Top of the branch
• Python version : 3.8
• Virtual environment type and version : NA
• Operating system : Ubuntu

Additional context

Update: Installing java solve this issue, but it will increase image size and unnecessarily keep java in the image -

FROM python:3.8-slim-buster
RUN mkdir -p /usr/share/man/man1
RUN apt-get update && apt-get upgrade -y && apt-get install -y curl git default-jre
RUN pip install --upgrade pip
RUN pip install --no-cache-dir git+https://github.com/facebookresearch/hydra.git#egg=hydra-core

[omry]

Installing Hydra from source requires Java.
If you install the packages wheel (e.g. pip install hydra-core) it's not needed.

This is not a bug.

[omry]

For the record, you need a modern JRE. Check the requirements of antlr (it should be very easy to get it a compatible Java runtime).

[lalitpagaria]

@omry agree with you but can we please document it.
Also by chance is there intermediate release of Hydra-core like alpha1 possible?

[omry]

I think the error message for this problem is better than any documentation hidden in some corner of the website.

At some point we will likely start having nightly releases (#1112), but this is not super high priority.
Once 1.1 hits a certain maturity level I will probably start making periodic (manual) dev releases for it.

[lalitpagaria]

Thanks for the update looking forward for nightly releases