Brew: `brew cleanup` fails when cask is defined in multiple taps

[mfriedenhagen]

• [X] are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh (https://discourse.brew.sh/t/brew-cleanup-chokes-cask-adoptopenjdk8-exists-in-multiple-taps/4786)
• [ ] ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Homebrew/homebrew-core: https://github.com/Homebrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
• [X] ran brew update and can still reproduce the problem?
• [X] ran brew doctor, fixed all issues and can still reproduce the problem?
• [X] ran brew config and brew doctor and included their output with your issue?

$ brew config 
HOMEBREW_VERSION: 2.1.2-12-g37f4b92
ORIGIN: https://github.com/Homebrew/brew
HEAD: 37f4b92644a255ef76e2e1c9df686211f610e82e
Last commit: 31 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: cedba1f1b4daf1605ff985fcaa7995fe2eb0f527
Core tap last commit: 4 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_VISUAL: subl -w
CPU: quad-core 64-bit kabylake
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1001
Git: 2.21.0 => /usr/local/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 11.0.3, 1.8.0_212, 1.8.0_202
macOS: 10.14.4-x86_64
CLT: 10.2.1.0.1.1554506761
Xcode: N/A
CLT headers: 10.2.1.0.1.1554506761
$ brew doctor 
Your system is ready to brew.

What you were trying to do (and why)

• I added both homebrew/cask-versions and adoptopenjdk/openjdk as taps
• In the past I installed adoptopenjdk8 via the adoptopenjdk tap. Now it is available in cask-versions as well.
• I run the following once in a while:

brew cask upgrade
brew upgrade
brew cleanup

• Because the cask adoptopenjdk8 is available in both taps, I get:

$ brew cleanup
Error: Cask adoptopenjdk8 exists in multiple taps:
  homebrew/cask-versions/adoptopenjdk8
  adoptopenjdk/openjdk/adoptopenjdk8

• As I installed adoptopenjdk8 initially without a tap prefix, I already ran brew cask uninstall adoptopenjdk/openjdk/adoptopenjdk8 and brew cask install adoptopenjdk/openjdk/adoptopenjdk8 afterwards. But the error persists.

Another thing I do not understand:

$ brew cask list --full-name
adoptopenjdk8
apache-directory-studio
fliqlo
knockknock
vagrant
virtualbox
virtualbox-extension-pack
adoptopenjdk/openjdk/adoptopenjdk11

As can be seen adoptopenjdk11 is (correctly?!?) prefixed with the tap, but adoptopenjdk8 is not.

What happened (include command output)

• brew cleanup fails.
• Even worse it seems that even brew upgrade fails sometime, there seems to be a sort of automatic cleanup which happens after a specific amount of days:

$ brew upgrade
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/cask).
No changes to formulae.

==> Upgrading 1 outdated package:
derailed/k9s/k9s 0.6.5 -> 0.6.6
==> Upgrading derailed/k9s/k9s 
==> Downloading https://github.com/derailed/k9s/releases/download/0.6.6/k9s_0.6.6_Darwin_x86_64.tar.gz
==> Downloading from https://github-production-release-asset-2e65be.s3.amazonaws.com/167596393/718b3c80-723e-11e9-82df-3ff559a7fde5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190509%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190509T
######################################################################## 100.0%
🍺  /usr/local/Cellar/k9s/0.6.6: 5 files, 39.8MB, built in 6 seconds
==> `brew cleanup` has not been run in 30 days, running now...
Removing: /usr/local/Cellar/k9s/0.6.5... (5 files, 39.8MB)
Error: Cask adoptopenjdk8 exists in multiple taps:
  homebrew/cask-versions/adoptopenjdk8
  adoptopenjdk/openjdk/adoptopenjdk8

What you expected to happen

I can run brew cask upgrade && brew upgrade && brew cleanup successfully.

Step-by-step reproduction instructions (by running brew commands)

• I actually did not remove brew completely, but as far as I understand this should reproduce this.

brew tap homebrew/cask-versions 
brew tap adoptopenjdk/openjdk
brew cask install adoptopenjdk/openjdk/adoptopenjdk8
brew cleanup # <-- chokes

See AdoptOpenJDK/homebrew-openjdk#106 as well.

[MikeMcQuaid]

CC @Homebrew/cask

[MikeMcQuaid]

(Not CCing for this specific Java issue but because it should be possible to have multiple taps with the same name and not have brew cleanup go 💥.

@mfriedenhagen Can I see the output of brew cleanup --debug in that case? Thanks!

[claui]

@MikeMcQuaid Yes, we absolutely need to fix this.

In the meantime, the issue can be worked around by removing the cached download:

rm -f ~/Library/Caches/Homebrew/Cask/adoptopenjdk8* \
  ~/Library/Caches/Homebrew/downloads/*-OpenJDK8*

[amyspark]

tl;dr: Casks are loaded by the about-to-be-cleaned download's file name. (Not a good idea at all.)

amalia@Sakura:~ $ brew cask info adoptopenjdk8 --debug
Error: Cask adoptopenjdk8 exists in multiple taps:
  homebrew/cask-versions/adoptopenjdk8
  adoptopenjdk/openjdk/adoptopenjdk8
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:201:in `for'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:178:in `load'
/usr/local/Homebrew/Library/Homebrew/cask/cmd/abstract_command.rb:52:in `block in casks'
/usr/local/Homebrew/Library/Homebrew/cask/cmd/abstract_command.rb:52:in `map'
/usr/local/Homebrew/Library/Homebrew/cask/cmd/abstract_command.rb:52:in `casks'
/usr/local/Homebrew/Library/Homebrew/cask/cmd/info.rb:20:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/cmd/abstract_command.rb:36:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:92:in `run_command'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:158:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:123:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:9:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:102:in `<main>'
Error: Kernel.exit
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:162:in `exit'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:162:in `rescue in run'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:146:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:123:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:9:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:102:in `<main>'


amalia@Sakura:~ $ brew cleanup --debug
<snip>
Error: Cask adoptopenjdk8 exists in multiple taps:
  homebrew/cask-versions/adoptopenjdk8
  adoptopenjdk/openjdk/adoptopenjdk8
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:201:in `for'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:178:in `load'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:99:in `stale_cask?'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:41:in `stale?'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:296:in `block in cleanup_cache'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:280:in `each'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:280:in `cleanup_cache'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:170:in `clean!'
/usr/local/Homebrew/Library/Homebrew/cmd/cleanup.rb:43:in `cleanup'
/usr/local/Homebrew/Library/Homebrew/brew.rb:102:in `<main>'

[amyspark]

Ok, I've kinda finished triaging this. Casks' downloads are stored and referenced by only their token. This triggers a lookup failure when trying to access the related cask, for obvious reasons.

The fix should be twofold: first, the downloader strategy should use the full_name to generate the cache file name. Then, all moving parts in brew cleanup that rely on cache &:children must be updated to account for this. (The latter part will be certainly difficult.)

PD: can someone check if cleaning clashing formulae triggers the same bug?

[MikeMcQuaid]

PD: can someone check if cleaning clashing formulae triggers the same bug?

It doesn't.

The fix should be twofold: first, the downloader strategy should use the full_name to generate the cache file name.

We can't modify the cache in this way again. An alternative workaround would be to use the list of installed casks to prioritise casks from a particular tap.

[mfriedenhagen]

Where does brew store the information about installed formulas and casks?

[MikeMcQuaid]

The Cellar and Caskroom respectively.

[amyspark]

Quick 'n dirty fix:

diff --git a/Library/Homebrew/cleanup.rb b/Library/Homebrew/cleanup.rb
index 2679b4b76..4b43f1a6a 100644
--- a/Library/Homebrew/cleanup.rb
+++ b/Library/Homebrew/cleanup.rb
@@ -96,7 +96,7 @@ module CleanupRefinement
       return false unless name = basename.to_s[/\A(.*?)\-\-/, 1]

       cask = begin
-        Cask::CaskLoader.load(name)
+        Cask::Cask.search(name, &:token).sort_by(&:token).first
       rescue Cask::CaskUnavailableError
         return false
       end

BTW, Cask list both versions as installed, so I preferred to sort them just like brew search does.

[MikeMcQuaid]

Cask list both versions as installed, so I preferred to sort them just like brew search does.

Can you elaborate on this? It shows both versions as installed even if only one is?

[amyspark]

Yes @MikeMcQuaid , e.g. after installing homebrew/cask-versions/adoptopenjdk8. Note that I've tapped AdoptOpenJDK's tap too.

bamalia@Sakura:~ $ brew search adoptopenjdk8
==> Casks
adoptopenjdk8 ✔            adoptopenjdk8-jre          adoptopenjdk8-openj9-jre
adoptopenjdk8 ✔            adoptopenjdk8-openj9
amalia@Sakura:~ $ brew search adoptopenjdk8 -v
==> Casks
adoptopenjdk8 ✔            adoptopenjdk8-jre          adoptopenjdk8-openj9-jre
adoptopenjdk8 ✔            adoptopenjdk8-openj9
amalia@Sakura:~ $ brew cask info homebrew/cask-versions/adoptopenjdk8
adoptopenjdk8: 8,212:b03
https://adoptopenjdk.net/
/usr/local/Caskroom/adoptopenjdk8/8,212:b03 (97.7MB)
From: https://github.com/Homebrew/homebrew-cask-versions/blob/master/Casks/adoptopenjdk8.rb
==> Name
AdoptOpenJDK 8
==> Artifacts
OpenJDK8U-jdk_x64_mac_hotspot_8u212b03.pkg (Pkg)
amalia@Sakura:~ $ brew cask info adoptopenjdk/openjdk/adoptopenjdk8
adoptopenjdk8: 8,212:b03
https://adoptopenjdk.net/
/usr/local/Caskroom/adoptopenjdk8/8,212:b03 (97.7MB)
From: https://github.com/adoptopenjdk/homebrew-openjdk/blob/master/Casks/adoptopenjdk8.rb
==> Name
AdoptOpenJDK 8
==> Artifacts
OpenJDK8U-jdk_x64_mac_hotspot_8u212b03.pkg (Pkg)

[MikeMcQuaid]

@amyspark Weird! I guess that's going to be related to the underlying bug here and also probably needs addressed.

[reitermarkus]

It shows both as installed because we don't store which tap it was installed from in Caskroom.

[Archez]

The same thing is happening for the chromedriver-beta cask as well.
chromedriver exists on homebrew/cask
chromedriver-beta exists on homebrew/cask-versions and caskroom/versions

What happens in my case is I just want to install chromedriver; however, the install fails when it performs the update check as I believe both packages have the same starting string, so the conflict clash happens.

$ brew cask install chromedriver
Updating Homebrew...
Error: Cask chromedriver-beta exists in multiple taps:
  homebrew/cask-versions/chromedriver-beta
  caskroom/versions/chromedriver-beta

This clash happens regardless if I run brew cask install chromedriver or brew cask install homebrew/cask/chromedriver so I essentially am unable to install this cask without removing one of the taps that contains chromedriver-beta.

[reitermarkus]

@Archez, you seem to have an old tap, run brew untap caskroom/versions.

[EvaWang369]

I uninstalled and installed brew, then installed kafka (brew install kafka). it works.

[dhtml]

@Archez, you seem to have an old tap, run brew untap caskroom/versions.

This seems to work for me on mojave setup.