Homebrew-core: clojure formula dependency is wrong

Created on 21 Feb 2020 · 39Comments · Source: Homebrew/homebrew-core

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

[x] ran brew update and can still reproduce the problem?
[x] ran brew doctor, fixed all issues and can still reproduce the problem?
[X] ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link? https://gist.github.com/puredanger/bf1597cf60ed911fd70348b9953c18f9
[ ] if brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

What you were trying to do (and why)

Install Clojure and use my existing Java installation.

What happened (include command output)

brew install clojure

Command output

Updating Homebrew...
==> Auto-updated Homebrew!
Updated Homebrew from f8a4dd9fc Updated 3 taps (homebrew/core, ==> New Formulae
iam-policy-json-to-terraform ==> Updated Formulae
docker ✔ cypher-shell glib ✔ dartsim gmp ✔ deno imagemagick ✔ libheif ✔ derby netpbm ✔ detekt pandoc ✔ devspace protobuf ✔ dialog readline ✔ dita-ot ruby ✔ ditaa ruby-build ✔ dnscontrol x265 ✔ dwarfutils acpica ec2-ami-tools aliyun-cli ec2-api-tools angle-grinder ejabberd angular-cli elb-tools ansible emscripten antibody erlang antlr@2 apache-flink exploitdb apache-spark fastbit apollo-cli fastqc arduino-cli fetch-crl asciidoctorj ffmpeg ask-cli atari800 file-roller atlassian-cli firebase-cli aurora-cli fluent-bit aws-cfn-tools flume aws-elasticbeanstalk flyway aws-okta fmpp awscli@1 azure-cli fop bagit freeciv balena-cli freeswitch basex frege beagle frege-repl bfg frotz bind galen bison gatsby-cli bitlbee gcviewer boot-clj gdb broot git-annex bundletool byteman git-gui carrot2 gitbucket cassandra giter8 certbot gjs cfn-lint glooctl cfr-decompiler gmime cheat gnu-getopt chronograf golang-migrate circleci gom citus goreleaser clhep gradle clojure grafana clojurescript groovy closure-compiler groovysdk closure-stylesheets grpc cloud-watch cointop gsoap cromwell gtk+3 crowdin hadoop crystal haproxy crystal-icr hcloud csound helm ==> Deleted Formulae
gh /> to 5b126ec85.
homebrew/cask and borkdude/brew).
libcbor raxml-ng
helm@2 nagios-plugins sjk
helmfile nco skaffold
hg-fast-export netlify-cli skinny
dependency-check hugo nifi smali
igv node@12 sn0int
ipython nomad solr
istioctl nrpe [email protected]
itex2mml nushell sonarqube
jack ocrmypdf sonarqube-lts
jadx octant sphinx-doc
jasmin octave sqlcipher
javacc octomap sqoop
jboss-forge opa srtp
jdnssec-tools open-scene-graph ssh-copy-id
jena openapi-generator stanford-corenlp
jenkins openimageio stanford-ner
jetty openjdk@11 stanford-parser
jetty-runner openssh starship
ethereum jflex operator-sdk storm
jhipster orientdb subversion
joshua ott suricata
jruby pacapt swagger2markup-cli
jsonschema2pojo packer swiftformat
jsvc paket tailor
[email protected] juju pandoc-citeproc taskell
jupyterlab percona-toolkit tcl-tk
kaitai-struct-compiler pgweb tee-clc
kawa php teleport
kube-aws [email protected] terraform
kubectx [email protected] terraform_landscape
kumo phpstan terragrunt
fonttools kyma-cli picard-tools tika
languagetool pig tokei
lazygit plantuml tomcat
lcm platformio tomcat-native
lcov pmd tomcat@7
ldc poco tomcat@8
ldns postgresql@10 topgrade
libarchive postgresql@11 tox
libdap [email protected] traefik@1
libpq [email protected] tundra
libpqxx prestodb txr
libstfl prestosql typescript
git-credential-manager liquidctl procs ucloud
macvim procyon-decompiler umlet
mallet proj unifdef
manticoresearch prometheus upscaledb
mariadb-connector-c pulumi vault-cli
mat2 pyenv-virtualenv vert.x
mdcat python-markdown vim
mesa rav1e visp
mgba raylib vnu
micronaut rds-command-line-tools vulkan-headers
mikutter redpen walkmod
mill renameutils weechat
minikube root whois
minio-mc sbt wildfly-as
mit-scheme sbuild wiremock-standalone
mlt scala wpscan
gsettings-desktop-schemas mmark [email protected] wtf
mmctl scdoc xmlsectool
mockserver sdedit yaegi
modules serverless youtube-dl
mpd siege zola
mutt signal-cli zsh
mvnvm simple-scan
zpython

==> Installing dependencies for clojure: openjdk and readline
==> Installing clojure dependency: openjdk
==> Downloading https://homebrew.bintray.com/bottles/openjdk-13.0.2+8_2.catalina.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/65/65adca036393f528e3830cab8b0aafec94be870de087d94cfe098fd593517307?__gda__=exp=1582297799~hmac=b5e6c5

################################################################## 100.0%

==> Pouring openjdk-13.0.2+8_2.catalina.bottle.tar.gz
==> Caveats
For the system Java wrappers to find this JDK, symlink it with
sudo ln -sfn /usr/local/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk

openjdk is keg-only, which means it was not symlinked into /usr/local,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have openjdk first in your PATH run:
echo 'export PATH="/usr/local/opt/openjdk/bin:$PATH"' >> ~/.bash_profile

For compilers to find openjdk you may need to set:
export CPPFLAGS="-I/usr/local/opt/openjdk/include"

==> Summary
🍺 /usr/local/Cellar/openjdk/13.0.2+8_2: 631 files, 314.6MB
==> Installing clojure dependency: readline
==> Downloading https://homebrew.bintray.com/bottles/readline-8.0.4.catalina.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/6a/6ae1c8e7c783f32bd22c6085caa4d838fed7fb386da7e40ca47b87ec9b1237d6?__gda__=exp=1582297855~hmac=db812a

################################################################## 100.0%

==> Pouring readline-8.0.4.catalina.bottle.tar.gz
==> Caveats
readline is keg-only, which means it was not symlinked into /usr/local,
because macOS provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only.

For compilers to find readline you may need to set:
export LDFLAGS="-L/usr/local/opt/readline/lib"
export CPPFLAGS="-I/usr/local/opt/readline/include"

For pkg-config to find readline you may need to set:
export PKG_CONFIG_PATH="/usr/local/opt/readline/lib/pkgconfig"

==> Summary
🍺 /usr/local/Cellar/readline/8.0.4: 48 files, 1.5MB
==> Installing clojure
==> Downloading https://download.clojure.org/install/clojure-tools-1.10.1.510.tar.gz

################################################################## 100.0%

==> ./install.sh /usr/local/Cellar/clojure/1.10.1.510_1
🍺 /usr/local/Cellar/clojure/1.10.1.510_1: 11 files, 18.6MB, built in 4 seconds
==> Caveats
==> openjdk
For the system Java wrappers to find this JDK, symlink it with
sudo ln -sfn /usr/local/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk

openjdk is keg-only, which means it was not symlinked into /usr/local,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have openjdk first in your PATH run:
echo 'export PATH="/usr/local/opt/openjdk/bin:$PATH"' >> ~/.bash_profile

For compilers to find openjdk you may need to set:
export CPPFLAGS="-I/usr/local/opt/openjdk/include"

==> readline
readline is keg-only, which means it was not symlinked into /usr/local,
because macOS provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only.

For compilers to find readline you may need to set:
export LDFLAGS="-L/usr/local/opt/readline/lib"
export CPPFLAGS="-I/usr/local/opt/readline/include"

For pkg-config to find readline you may need to set:
export PKG_CONFIG_PATH="/usr/local/opt/readline/lib/pkgconfig"

What you expected to happen

Clojure formula should be installed but I already have a valid Java set up and I did not want openjdk installed. The clojure formula was recently changed incorrectly from Java >= 1.8 to adoptopenjdk.

I am on the Clojure core team and wrote the clojure brew formula and installer.

Clojure does not require adoptopenjdk. Some users use it with the Oracle JDK or other vendor JDKs. The requirement from a language perspective is any valid Java, >= 1.8, so the previous dependency was correct. Many Clojure users prefer to have control over their JDK version and most users are still using Java 8 as the module system introduced in Java 9 may require changes before migration.

Without the opportunity to do that migration, auto-installing adoptopenjdk at the latest version can cause their application to throw warnings or fail (due to modules that have been broken out of the JDK and are no longer automatically provided). Here is an example issue of how this can manifest to users: https://github.com/clojure-emacs/orchard/issues/20 (this particular issue was not caused by this brew change, but is exactly the experience users with a forced Java update may encounter).

This commit should be reverted.

Step-by-step reproduction instructions (by running `brew install` commands)

brew install clojure

outdated

Source

puredanger

👍37

Most helpful comment

I appreciate the homebrew maintainers' work and respect their policies, and I hope others will as well.

While I have my own opinions, I accept the decision here and Clojure will move their formula into their own tap, so this does not need to be an area of conflict. We've been contemplating this for a while anyways.

puredanger on 21 Feb 2020

👍9

All 39 comments

Homebrew will use brewed dependencies where possible. The fact that other Java versions were previously possible was an unfortunate side-effect of java not being build from source as a formula. The current situation is the intended usage within homebrew. If people want to use a different dependency the best course of action is to maintain it in a tap.

SMillerDev on 21 Feb 2020

😕2

Just a couple of points to highlight one more time:

openjdk installed from dependency is not symlinked anywhere and will be used only if JAVA_HOME is not set
If you'd like to use another Java installation feel free to set JAVA_HOME to any Java you want
Using some random/installed by default Java is not reliable for software shipping (and the thing you have shown in https://github.com/clojure-emacs/orchard/issues/20 exactly the case of using some "random" version of Java which we want to avoid for a default installation)

bayandin on 21 Feb 2020

👍1

I guess this is good reason to not use brew ¯_(ツ)_/¯

jsa-aerial on 21 Feb 2020

I think we will probably just pull this out into our own tap then.

puredanger on 21 Feb 2020

👍2

90% of the clojure community are known to use either JDK 8 or JDK 11 (https://clojure.org/news/2020/02/20/state-of-clojure-2020)

And clojure.org recommends running clojure on JDK8 or JDK11 at this time. It seems very strange to me that brew are packaging clojure against the platforms recomendations.

It's also not a purely theoretical issue I have clojure apps which use the clojure command line tools, and don't run on anything other than JDK 8 due to either dependencies on XMLGregorianCalendar; which was removed in JDK9, or now illegal reflective access calls because of the JDK9 module system.

RickMoynihan on 21 Feb 2020

👍3

If you'd like to use another Java installation feel free to set JAVA_HOME to any Java you want

SMillerDev on 21 Feb 2020

👍1

If you'd like to use another Java installation feel free to set JAVA_HOME to any Java you want

The homepage for homebrew says:

Homebrew installs the stuff you need that Apple (or your Linux system) didn’t.

This is clearly incorrect in this instance.

The Clojure team has expressed what the correct dependency is, and if this is not what is installed then homebrew is clearly not doing the main thing that it's advertised to do. Homebrew will be installing something that the significant majority of users do not need (and is not sanctioned), and then expecting them to fix it.

I find this disappointing and makes me suspicious of how other packages are handled.

admiralbumblebee on 21 Feb 2020

👍5

This is clearly incorrect in this instance.

You need Java to use Clojure, we install that dependency. Ergo "Homebrew installs the stuff you need"

The Clojure team has expressed what the correct dependency is

The team only indicated that the current dependency is incorrect, not which formula is the correct one. If the correct way is not to depend on any Java formula that breaks the flow for new users and is equally unacceptable.

SMillerDev on 21 Feb 2020

👍1

Why push users into a default configuration that is neither recommended by clojure's core team or its community of practitioners? It strikes me that establishing the wrong default is going to cause new and existing users of the language problems.

RickMoynihan on 21 Feb 2020

👍8

Yes, people mentioned that. The next person who only says it's wrong without providing an alternative will be billed for the time I spend on this.

SMillerDev on 21 Feb 2020

👎13 👍1

Yes, people mentioned that. The next person who only says it's wrong without providing an alternative will be billed for the time I spend on this.

The OP says:

the previous dependency was correct.

This commit should be reverted.

That seems like an alternative to me?

admiralbumblebee on 21 Feb 2020

Please can we keep this civil without criticising volunteers for their work, whether we like it or not.

Personally I'd prefer the depedency was left as it was; but if an openjdk release must be forced on people picking a release of either 8 or 11 would I think be preferable to users of this formula.

RickMoynihan on 21 Feb 2020

👍7

I appreciate the homebrew maintainers' work and respect their policies, and I hope others will as well.

puredanger on 21 Feb 2020

👍9

Clojure will move their formula into their own tap, so this does not need to be an area of conflict.

We will not be removing this formula from homebrew-core until it is no longer widely used. Of course Clojure can create their own tap and recommend that if they prefer.

If someone can point out specific, reproducible steps to demonstrate how this configuration is broken (rather than "unsupported", "wrong" or "not recommended") we can consider modifying the formula so that these problems are addressed.

Additionally:

If you'd like to use another Java installation feel free to set JAVA_HOME to any Java you want

I'm not sure how this doesn't address the problem (please clarify if it does not). If you do not wish to use the installed openjdk formula: simply set JAVA_HOME and ignore it.

If this is simply "Homebrew installs a dependency it didn't before and I don't like that": that's par for the course with a package manager.

MikeMcQuaid on 21 Feb 2020

👎2 👍2

Note if you're part of this conversation it's not at all helpful to just 👎 comments without actually attempting to answer the several questions that need answering to reach an adequate resolution.

MikeMcQuaid on 21 Feb 2020

As a more meta-point I realise language communities operate independently of each other and can't be expected to keep up with what all the others are doing but this is not an issue specific to Clojure or Java. NodeJS and several other languages and applications regularly say "but this formula doesn't depend on X!" when it actually does.

It may not depend exclusively on the specific implementation or version we have chosen but that's a wider Homebrew implementation detail. We used to have things that allowed random formulae to depend on any of multiple dependencies to fulfil the dependency (Requirements) but this was extremely error-prone and we're unable to support it adequately so we don't. We are currently moving Java formula to use this openjdk dependency from the JavaRequirement which causes us issues with maintaining our CI environment and requires users to install a cask (or equivalent) for a formula dependency. We are likely to do this for X11 if possible in future, too.

MikeMcQuaid on 21 Feb 2020

Hey @MikeMcQuaid and others, thanks for engaging productively.

Clojure does not actively test against "the latest version of Java", but rather tries to focus on the LTS releases (currently Java 8 or 11, soon 14). We do try to make sure it works on the interim ones, but that's not part of our current test matrix, and not part of what we recommend to users. Also, we try to be JVM-vendor-agnostic so that users on Oracle or other OpenJDK-based distributions should all work.

I don't know enough about what is possible or allowed with Homebrew to have a good handle on the space of options. Could we remove the Java dependency entirely and have the tools check and tell you what to do if it's not present?

With respect to our own tap, there are other possible benefits there but it would be actively harmful to leave an old formula here without updates and only update in a secondary tap. I am the only person building the tar files and publishing updates to the formula here. If I switch to doing that on a Clojure-specific tap, could we blacklist Clojure here with a pointer to the Clojure tap? Otherwise, this will become the default place to look with an increasingly out of date thing to find.

puredanger on 21 Feb 2020

While you might be the first person to currently suggest updates that doesn't necessarily mean that the formula will be outdated when you no longer suggest them. If it's consistently outdated we can always consider a deletion though.

SMillerDev on 22 Feb 2020

👍1

I’m part of the Clojure team, and the one doing 99% of the dev and release work on these tools. I’m going to start pushing updates only to a Clojure tap, so it will be consistently old.

For Clojure users, the best and most up to date source for these updates will be the Clojure tap. It seems like it can only be worse for users to get an old version from core without knowing it’s old. I don’t see how it benefits anyone to not point to the Clojure tap from the start?

puredanger on 22 Feb 2020

👍3

@puredanger, to get back to the original issue that the “dependency is wrong”, would depending on the openjdk@11 formula be the “right” dependency?

reitermarkus on 22 Feb 2020

I think we are talking past each other. What exactly is the problem?

If the issue is that users are getting a copy of openjdk that they don't technically need, and the proposed solution is to change the dependency back to depends_on :java, that's not going to happen. See https://docs.brew.sh/Building-Against-Non-Homebrew-Dependencies for why this is the case.

The fact that depends_on :java existed for so long is largely a due to a lack of maintainer time and is undesired behavior. It causes many problems for our CI and increases our maintenance burden. Migrating to a brewed openjdk has been in the works for a long time. The linked document also provides an alternative if you don't want a brewed jdk (i.e., maintain your own tap). Note that the clojure formula specifically permits setting your own JAVA_HOME which will permit you to use whatever Java you want.

If the issue is that clojure only works with LTS releases of Java, then https://github.com/Homebrew/homebrew-core/issues/50536#issuecomment-589896834 appears to be the right solution, so please open a pull request for that.

jonchang on 22 Feb 2020

👍1

@reitermarkus Yes, if brew must fetch openjdk, let it fetch the LTS release (currently 11, soon 14, basically every 3rd release).

Although, as mentioned before, some projects have a hard dependency on JDK 8 as well, since Java 9 removed a few capabilities.

hindol on 22 Feb 2020

if brew must fetch openjdk

The point of a package manager is that after installing a package, it works, so yes, we must.

some projects have a hard dependency on JDK 8 as well

That is the reason we still allow overriding JAVA_HOME for formulae related to Java development. Still, the default will always be to depend on the latest version possible.

reitermarkus on 22 Feb 2020

@MikeMcQuaid

If you do not wish to use the installed openjdk formula: simply set JAVA_HOME and ignore it.

In my eyes, the problem with that suggestion is, how does one do that universally, on one's machine? How do I ensure that any invocation of java on my machine, from any environment (that doesn't have a prescribed java version, such as redefining JAVA_HOME, obviously), such as app runs, sudo runs, any process calls from any programming language, etc, uses that version? The same problem occurs with JAVA_OPTIONS and is entirely not homebrew's fault - but it still an issue with that suggestion (IMHO).

@Hindol

let it fetch the LTS release (currently 11, soon 14, basically every 3rd release).

(14 isn't an LTS, btw. It's every 3 years not 3rd release, so the next LTS is Java 17, in Sept 2021.)

@reitermarkus

Still, the default will always be to depend on the latest version possible.

In https://github.com/Homebrew/homebrew-core/commit/f51759139edadc7e52d33b924c1b26ea058148f1 @SMillerDev also mentioned "This allows us to test tools with updates". I don't understand exactly how much this change is motivated by testing (there are a few mentions of CI troubles in the thread). But would it be acceptable if, as a default, the dependency resulted in the latest being used, but users had the option to satisfy the constraint of "need java" by their own selection of JDK variant/version?

Unfortunately, the break-neck speed of JDK releases (every 6 months, with only a 1 month overlap) is quite the imposition on the ecosystem, and honouring the 3 year LTS release cadence is more feasible and seems to have become fairly common. Much like the Clojure community, the Scala community (whom I'm a part of) tries to spend time on the non-LTS releases, but we're more certain of the software on the LTS releases. So, in some way, this isn't about "use the latest version" it's more like "use the latest pre-release".

Lastly, much ❤️ to the maintainers of the Homebrew ecosystem. I'm always grateful for all the work that goes in.

dwijnand on 22 Feb 2020

In my eyes, the problem with that suggestion is, how does one do that universally, on one's machine?

Set it in bashrc or equivalent and configure your IDE to also honor it, that covers all cases AFAIK.

But would it be acceptable if, as a default, the dependency resulted in the latest being used, but users had the option to satisfy the constraint of "need java" by their own selection of JDK variant/version?

That's the current situation that you can set with JAVA_HOME.

SMillerDev on 22 Feb 2020

👍1

I don’t see how it benefits anyone to not point to the Clojure tap from the start?

People will not start looking for the clojure tap by default and will install clojure in weird unexpected ways if it doesn't just work with a default install. Your solution only works if people go through your documents, find your tap and then try installing it.

SMillerDev on 22 Feb 2020

👍1

In my eyes, the problem with that suggestion is, how does one do that universally, on one's machine?

Set it in bashrc or equivalent and configure your IDE to also honor it, that covers all cases AFAIK.

>

Wait, how does that cover all cases? How does it even cover sudo? How does it cover a python script calling subprocess.run (https://stackoverflow.com/a/54123892/463761). In my experience, it just doesn't.

dwijnand on 22 Feb 2020

Sudo has some options to retain environment variables and you can always set the variable for the target user as well. All of this is something that Linux users have used for years though. It's a proven concept and Homebrew is just joining everyone else here.

SMillerDev on 22 Feb 2020

👍1

I'm not saying you can't seek and find workarounds at every usage site to try and make it use the right version of Java. But I hope you can see how "Oh, why don't you just set JAVA_HOME instead" isn't so straightforward in truth.

It's a proven concept and Homebrew is just joining everyone else here.

In a sense, so is macOS's java and /usr/libexec/java_home and the new setup actively subverts all that. I understand there's a good reason for it, but the "just set JAVA_HOME" alternative isn't great. (But I thank you for your answers.)

dwijnand on 22 Feb 2020

In my experience, it just doesn't.

@dwijnand I'm afraid it's not our job to explain to you how to configure your software and environment so they work how you want.

I don’t see how it benefits anyone to not point to the Clojure tap from the start?

People will not start looking for the clojure tap by default and will install clojure in weird unexpected ways if it doesn't just work with a default install. Your solution only works if people go through your documents, find your tap and then try installing it.

Additionally: https://formulae.brew.sh and similar tooling does not exist for your tap. People expect brew install clojure to work and we're going to continue to ensure that it does. What is likely to happen is that other users or maintainers will step up and update clojure if you do not wish to do so.

I think we are talking past each other. What exactly is the problem?

This is still happening. No-one can explain to me why this dependency is "wrong" and provide a test-case demonstrating this. Therefore I'm forced to draw the conclusion that there isn't one and this is an upstream preference and what they "support" rather than a hard requirement. Please prove me wrong.

Unless the conversation turns fairly rapidly in this direction this thread is going to get locked; it doesn't seem to be a productive use of anyone's time at this point.

MikeMcQuaid on 22 Feb 2020

👍1

Hi, for a little context I've given non-trivial economical support to Clojure and Homebrew alike. I see how both parts here are defending a reasonable technical approach, pursuing well-intentioned goals.

Also I see some flaws in how this issue was opened, and also in how it was immediately closed. Both facts can skew the whole discussion.

So it might be desirable to start a new issue from scratch, trying to stick to the facts and keep it technical.

As much as I admire Alex Miller (known within Clojure folks for his expertise in all JVM matters), clojure formula dependency is wrong is not a problem statement - it's a perceived symptom.

Likewise, the JAVA_HOME suggestion seems reasonable, and only one person (not OP) has tried to explain what might be wrong with it.

provide a test-case demonstrating this

This also seems necessary.

vemv on 22 Feb 2020

👍4

So it might be desirable to start a new issue from scratch, trying to stick to the facts and keep it technical.

I would rather we continue discussion on the closed issue. This is how Homebrew uses our issue tracker: things that are not immediately/obviously actionable are closed and may be reopened when they converge on a solution (or a new issue opened).

Otherwise: I agree completely, thanks @vemv.

MikeMcQuaid on 23 Feb 2020

👍1

Catching up (thanks @MikeMcQuaid @SMillerDev @reitermarkus). There's really two things going on here and I want to explicitly separate them if I can.

1 - I filed the original issue because the change in dependency was incorrect from the point of view of Clojure. Clojure depends on any Java, >= 1.8 so changing it to be "latest version of adoptopenjdk" was not that. However, I understand better now that the policy in homebrew is "use latest deps built from source", so the dependency change is entirely in keeping with homebrew's policy and is not wrong in that regard. I have not disagreed with that explanation or decision to close this issue on those points. (And indeed, post install, setting JAVA_HOME puts the user in full control of which Java is actually used, so that all works as expected.)

I do think it would be helpful to use openjdk@11 if that's an ok thing to do. It certainly matches better with clojure testing and release strategy. That was suggested above but it's hard for me to tell who is on homebrew team recommending things and who is just chipping in, so I'm not sure if that was a good suggestion or not. If so, I will open a PR for it.

2 - The second thing here is that from the Clojure team's point of view someone externally changed our dependency without any input from us. I understand that's how things work here, but it's disconcerting. As the creators of a thing, we want to have some level of control over how these decisions are made and presented to users. Homebrew is providing an incredibly valuable service to us as the favored package manager on our most popular platform, and I have a ton of love and respect for y'all doing that. What I'm trying to figure out is the best way that we can live in the homebrew ecosystem while also doing great by our users.

It seems like external taps are one big way that could be done, and it might be useful for a variety of reasons - control over the formula, control over releases (without needing to gate on the homebrew team to approve prs), providing tagged versions (@ etc), and external brew commands.

I've done a fair amount of googling and reading of brew docs and looking at what other communities are doing and certainly there seem to be many cases of organizations moving formulas both in and out of core for their own external taps but it's hard to determine the reasons in most of those cases. I would love to have more guidance on what you think about that, how to make that decision and how to work cooperatively so that our users have a clear path to getting the best possible version, with the minimum of confusion. (And I also understand if the answer is, "not our problem". I'm not asking you to solve it, just looking for best guidance.)

Thanks.

puredanger on 23 Feb 2020

Thanks for the detailed response @puredanger :heart:.

And indeed, post install, setting JAVA_HOME puts the user in full control of which Java is actually used, so that all works as expected.

Glad to hear it 👍

I do think it would be helpful to use openjdk@11 if that's an ok thing to do. It certainly matches better with clojure testing and release strategy. That was suggested above but it's hard for me to tell who is on homebrew team recommending things and who is just chipping in, so I'm not sure if that was a good suggestion or not. If so, I will open a PR for it.

I would rather we left it as-is unless there is known breakage.

I would love to have more guidance on what you think about that, how to make that decision and how to work cooperatively so that our users have a clear path to getting the best possible version, with the minimum of confusion.

From our perspective: once something is in Homebrew/core unless it breaks or we're unable to run CI on it for some other reason (e.g. depends on some systemwide change we're unwilling or unable to make to our CI system) we will keep it there indefinitely. We have the technical ability to migrate formulae to taps but this is not something we're willing to do for Homebrew/core essentially for security reasons.

The best way for upstream to ensure that the software is packaged the way they would like is to have the test do block reflect necessary functionality and provide some basic regression testing. This stops us from deliberately or accidentally breaking the functionality.

In this case, if there were issues with the latest Java, changing the dependency and changing the test do such that it produces an error with the "wrong" Java version would avoid us accidentally updating it to the wrong version in future.

I understand that's how things work here, but it's disconcerting. As the creators of a thing, we want to have some level of control over how these decisions are made and presented to users.

As a more meta-point: this isn't something Homebrew does but something most systemwide package managers do. We're responsible not just for packaging your software but ensuring that all software package operates as sensibly and well-integrated as possible. This results in us making decisions for our users on e.g. dependencies to minimise the amount of dependencies they need to install and generally be in keeping with the ethos of our package manager.

If it would be good to have explicit documentation "homebrew for upstreams" or similar then I can look at writing it in the nearish future.

MikeMcQuaid on 23 Feb 2020

@MikeMcQuaid we really want someone to manage their Java installation independently from Clojure. Could we remove the Java dependency entirely?

puredanger on 26 Feb 2020

Could we remove the Java dependency entirely?

No, that would make the default install unusable.

we really want someone to manage their Java installation independently from Clojure.

They can and will by setting JAVA_HOME

SMillerDev on 26 Feb 2020

What @SMillerDev said. Clojure does depend on Java (hence the dependency as it cannot run without it) but there's nothing to stop users from ignoring the openjdk formula entirely and using their own Java with JAVA_HOME.

MikeMcQuaid on 27 Feb 2020

I was very surprised when I saw Homebrew unexpectedly installing openjdk while upgrading Clojure to its latest release.

I have JAVA_HOME set to a Java 11 (adoptopenjdk), but Homebrew instead installs openjdk 13.

I ran the following for troubleshooting in brew irb:

irb(main):001:0> JavaRequirement.new(%w[11+]).send("possible_javas")
=> [nil, #<Pathname:/Users/adamf/.asdf/shims/java>]
irb(main):002:0> JavaRequirement.new(%w[11+]).send("preferred_java")
=> #<Pathname:/Users/adamf/.asdf/shims/java>
irb(main):003:0> JavaRequirement.new(%w[11+]).send("java_home_cmd")
=> nil
irb(main):004:0> JavaRequirement.new(%w[11+]).send("satisfied?")
=> true