Homebrew-core: brew link seems to link latest alphabetical version rather than highest version

[chrisdarroch]

If I have the following things in my Cellar:

• /usr/local/Cellar/docker/1.9.1
• /usr/local/Cellar/docker/1.11.1

If 1.9.1 is currently linked on my machine, I can run brew unlink docker and it will remove the synlinks for 1.9.1.

If I then run brew link docker, it re-links 1.9.1.

I suspect this is because brew link is picking up folders alphabetically as opposed to by highest version number.

Environment:

OS X 10.10.5
zsh
Homebrew 0.9.9 (git revision ed9bb; last commit 2016-07-13)
Homebrew/homebrew-core (git revision 08fb; last commit 2016-07-14)

[chrisdarroch]

I avoided the problem by running brew unlink and then brew prune before running brew link.

I raised this because one of brew install and brew upgrade advise to "run brew unlink docker && brew link docker" to fix the upgrade.

[UniqMartin]

This is not what should be happening and I can't quite reproduce this locally. Normally, when brew link sees two different versions of the formula, it simply refuses to do its job, because it can't know which one you might want to link:

$ brew link docker
Error: docker has multiple installed versions

But even if this wasn't the case, it's definitely sorting the installed versions by doing a proper version comparison, not just a simple string search. I'm truly curious how you ended up with the behavior you were able to observe.

[ilovezfs]

@UniqMartin you can create the situation artificially like this:

bash-4.3$ ls
2.9.1   2.9.2
bash-4.3$ brew info git|grep '*'
/usr/local/Cellar/git/2.9.2 (1,428 files, 32.8M) *
bash-4.3$ brew unlink git
Unlinking /usr/local/Cellar/git/2.9.2... 211 symlinks removed
bash-4.3$ mv 2.9.2 ..
bash-4.3$ brew link git
Linking /usr/local/Cellar/git/2.9.1... 211 symlinks created
bash-4.3$ mv ../2.9.2 .
bash-4.3$ brew info git|grep '*'
/usr/local/Cellar/git/2.9.1 (1,427 files, 32.8M) *
bash-4.3$ brew unlink git
Unlinking /usr/local/Cellar/git/2.9.1... 211 symlinks removed
bash-4.3$ brew link git
Linking /usr/local/Cellar/git/2.9.1... 211 symlinks created
bash-4.3$ brew upgrade git
Checking for Homebrew updates...
Error: git 2.9.2 already installed
bash-4.3$ brew unlink git
Unlinking /usr/local/Cellar/git/2.9.1... 211 symlinks removed
bash-4.3$ brew link git
Linking /usr/local/Cellar/git/2.9.1... 211 symlinks created
bash-4.3$ brew unlink git
Unlinking /usr/local/Cellar/git/2.9.1... 211 symlinks removed
bash-4.3$ readlink /usr/local/opt/git
../Cellar/git/2.9.1
bash-4.3$ rm /usr/local/opt/git
bash-4.3$ brew link git
Linking /usr/local/Cellar/git/2.9.2... 211 symlinks created
bash-4.3$ brew info git|grep '*'
/usr/local/Cellar/git/2.9.2 (1,428 files, 32.8M) *
bash-4.3$

The underlying cause is the opt link that brew unlink doesn't remove.

[ilovezfs]

@UniqMartin By the way, the real-world cause is almost always a failed upgrade, I believe:

bash-4.3$ pwd
/usr/local/Cellar/pcre
bash-4.3$ brew info pcre | grep '*'
/usr/local/Cellar/pcre/8.39 (203 files, 5.5M) *
bash-4.3$ ls
8.38    8.39
bash-4.3$ brew uninstall pcre
Uninstalling /usr/local/Cellar/pcre/8.39... (203 files, 5.5M)
pcre 8.38 is still installed.
Remove them all with `brew uninstall --force pcre`.
bash-4.3$ brew link pcre
Linking /usr/local/Cellar/pcre/8.38... 133 symlinks created 
bash-4.3$ readlink /usr/local/bin/pcregrep 
../Cellar/pcre/8.38/bin/pcregrep
bash-4.3$ rm /usr/local/bin/pcregrep 
bash-4.3$ touch /usr/local/bin/pcregrep 
bash-4.3$ brew info pcre | grep '*'
/usr/local/Cellar/pcre/8.38 (203 files, 5.4M) *
bash-4.3$ readlink /usr/local/opt/pcre
../Cellar/pcre/8.38
bash-4.3$ brew upgrade pcre
Checking for Homebrew updates...
==> Upgrading 1 outdated package, with result:
pcre 8.39
==> Upgrading pcre
==> Downloading https://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.39.tar.bz2
######################################################################## 100.0%
==> ./configure --prefix=/usr/local/Cellar/pcre/8.39 --enable-utf8 --enable-pcre8 --enable-pcre16 --e
==> make
==> make test
==> make install
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/pcregrep
Target /usr/local/bin/pcregrep
already exists. You may want to remove it:
  rm '/usr/local/bin/pcregrep'

To force the link and overwrite all conflicting files:
  brew link --overwrite pcre

To list all files that would be deleted:
  brew link --overwrite --dry-run pcre

Possible conflicting files are:
/usr/local/bin/pcregrep
==> Summary
🍺  /usr/local/Cellar/pcre/8.39: 203 files, 5.5M, built in 1 minute
bash-4.3$ readlink /usr/local/opt/pcre
../Cellar/pcre/8.38
bash-4.3$ brew link --overwrite --dry-run pcre
Would remove:
/usr/local/bin/pcregrep
bash-4.3$ brew link --overwrite pcre
Linking /usr/local/Cellar/pcre/8.38... 133 symlinks created
bash-4.3$ readlink /usr/local/bin/pcregrep 
../Cellar/pcre/8.38/bin/pcregrep
bash-4.3$ readlink /usr/local/opt/pcre
../Cellar/pcre/8.38
bash-4.3$ brew unlink pcre
Unlinking /usr/local/Cellar/pcre/8.38... 133 symlinks removed
bash-4.3$ brew link pcre
Linking /usr/local/Cellar/pcre/8.38... 133 symlinks created
bash-4.3$ brew unlink pcre
Unlinking /usr/local/Cellar/pcre/8.38... 133 symlinks removed
bash-4.3$ readlink /usr/local/opt/pcre
../Cellar/pcre/8.38
bash-4.3$ rm /usr/local/opt/pcre
bash-4.3$ brew link pcre
Linking /usr/local/Cellar/pcre/8.39... 133 symlinks created
bash-4.3$ brew info pcre | grep '*'
/usr/local/Cellar/pcre/8.39 (203 files, 5.5M) *
bash-4.3$

[UniqMartin]

Thanks for the analysis, @ilovezfs! It seems like Tap#unlink doesn't have a matching counterpart to this line in Tap#link. (Those are the methods that do most of the work for brew link and brew unlink.) Should Tap#unlink be calling Tap#remove_opt_record? I'm pretty confident this is the case, but haven't poked this part of Homebrew very much. @xu-cheng Thoughts?

[ilovezfs]

I think one corner case here is keg only stuff where the answer about the opt link can go either way depending on the user's intent when running brew unlink kegonlyfoo (e.g. after a brew link -f kegonlyfoo she wants to undo).

[ilovezfs]

And of course the flip side of that, which is really saying the same thing, where if brew unlink is changed to the remove the opt link, how are you supposed to recreate it for something that's keg only, since brew link will error out and brew link -f isn't going to do what you're trying to do?

[xu-cheng]

Should Tap#unlink be calling Tap#remove_opt_record?

No, it should not. It's by design that after Tap#unlink should not remove opt path for following reasons:

• brew unlink means only unlinked from prefix, but it should stil preserve the opt access.
• Keeping opt path helps to remember which keg is linked previously. So in the case of users using brew switch to change active version following with brew unlink to temporary unlink, users should be able to do brew link again targeted at previous chose keg.

However, there is indeed a bug in case of brew upgrade as shown by @ilovezfs, that a link conflict will cause the opt link staying the old path forever. And the solution is pretty simple. i.e. Run optlink before actual link inside Keg#link. Will submit a PR

[UniqMartin]

That's true. There are obviously some scenarios I haven't thought through, particularly the keg-only case and I have to admit that I don't have a good solution for that.

Maybe in this case the more appropriate thing to do is to modify brew link itself and how it determines which of the (possibly many) installed kegs it is supposed to link. It's relying on ARGV.kegs for this and this one in turns favors the opt link if it exist, even if there are newer versions of a formula. But now I have to admit that I'm not entirely clear about the semantics of the opt link. Is it supposed to always reflect the latest installed version? It's certainly not there to represent the linked keg, as that's what the symlinks in Library/LinkedKegs are for and the documentation of Formula#opt_prefix doesn't clarify it completely (for me).

[ilovezfs]

@xu-cheng should a failed upgrade remove the new version's opt link or not? And then should it temporarily relink the old optlink or not? That would have implications for whether a failed brew upgrade breaks other things in the Cellar that are linked to the opt libs or if they're left unbroken but pointing to the old version until a fully successful brew link of the new version.

[xu-cheng]

PR:

https://github.com/Homebrew/brew/pull/533

should a failed upgrade remove the new version's opt link or not? And then should it temporarily relink the old optlink or not?

Unless it's a uninstall, opt link should never be unlinked, it can only be overwrote to a new location. And if a upgrade is successful built/poured, it should be where the optlink pointed to regardless whether it's linked to prefix.

That would have implications for whether a failed brew upgrade breaks other things in the Cellar that are linked to the opt libs or if they're left unbroken but pointing to the old version until a fully successful brew link of the new version.

For opt path, it will never be broken. And since we link libraries to opt path instead of prefix path, everything should work fine. The only difference is for user's script/project etc not for Homebrew's formulae.

[ilovezfs]

This is now fixed! Thanks for reporting the issue @chrisdarroch. This has caused trouble for many users for quite some time now.