Brew: Brew upgrade ignores all options

[amckinlay]

Currently brew install -h does not have a syntax for formula build options (--with/without-*):

brew install [--debug] [--env=std|super] [--ignore-dependencies] [--only-dependencies] [--cc=compiler] [--build-from-source] [--devel|--HEAD] [--keep-tmp] formula
brew install --interactive [--git] formula

This can cause confusion, because other commands use similar syntax but do not accept formula build options.

For example, brew reinstall formula (syntax from brew reinstall -h) accepts build options, but brew upgrade [install-options] [--cleanup] [--fetch-HEAD] [formulae] does not (despite specifying that it accepts install-options). brew upgrade exacerbates this irregularity by ignoring invalid arguments -- build options are simply ignored without warning. It can also be confusing that brew upgrade accepts -s but not build options.

I first thought that brew upgrade ignoring build options was a bug.

[MikeMcQuaid]

I first thought that brew upgrade ignoring build options was a bug.

Ignoring valid build options is a bug. Ignoring invalid ones is not. They should be probably made non-silent, though.

[MikeMcQuaid]

I'm rereading this and not sure I understand the issue any more. @amckinlay To help us debug this issue can you explain:

• What you were trying to do (and why)
• What happened
• What you expected to happen
• Step-by-step reproduction instructions (with as minimal input data as possible)

[amckinlay]

@MikeMcQuaid Rereading the issue I can see how my explanation is confusing. I don't even know if some of the claims are correct.

Anyway, I encountered the issue because I wanted to upgrade a formula that had new build options available. I wanted to enable these build options upon brew upgrade of the formula. E.g., brew upgrade <formula> --with-new-option. I took me a while to realize after performing the upgrade that the new build options were not enabled because brew does not warn about invalid options.

It is strange that brew upgrade does not accept build options even though it accepts -s (at least as I remember).

Anywho, I think there should be a syntax item in the syntax description describing where build options go. E.g., brew install [--debug] ... [<build-options>] ... formula. And for commands that do not accept build options to not include it. So at least if the command does not complain about invalid options, you are not left guessing why the command is not accepting options. At first I thought it was a bug.

Hope that helps. I looked at the code, and the way brew heterogeneously splits out the processing between shell and ruby is too confusing for me to be able to submit a PR.

[MikeMcQuaid]

It is strange that brew upgrade does not accept build options even though it accepts -s (at least as I remember).

brew upgrade does accept build options (or at least should).

I looked at the code, and the way brew heterogeneously splits out the processing between shell and ruby is too confusing for me to be able to submit a PR.

None of the invalid option handling is done in Bash FWIW.

[MikeMcQuaid]

Can you confirm that brew upgrade always ignores all options? Thanks!

[amckinlay]

@MikeMcQuaid Yes, it ignores all options for building except -s.

[MikeMcQuaid]

@amckinlay Thanks! Can you provide the sample commands you've used for testing?

[masongup-mdsol]

I'm seeing this problem too, using commands as follows:

$ brew upgrade vim --without-python
==> Upgrading 1 outdated package, with result:
vim 8.0.0647
==> Upgrading vim --with-override-system-vi
==> Using the sandbox
==> Downloading https://github.com/vim/vim/archive/v8.0.0647.tar.gz
Already downloaded: /Users/mgup/Library/Caches/Homebrew/vim-8.0.0647.tar.gz
==> ./configure --prefix=/usr/local --mandir=/usr/local/Cellar/vim/8.0.0647/share/man --enable-multibyte --with-tlib=ncurses --enable-cscope --with-compiledby=Homebrew --enable-perlinterp --enable-pythoninterp --enable-rubyinterp --d

Where vim 8.0.0086 is already installed.

I'm running into this because I keep getting build error if_python.c:67:10: fatal error: 'Python.h' file not found, and homebrew doesn't seem to respond to my attempts to change the python options. The --without-python option doesn't work, and --with-python3 and --with-custom-python seem to be ignored also.

[ilovezfs]

@masongup-mdsol it's not actually a feature of upgrade at this point. Please open an issue in https://github.com/Homebrew/homebrew-core/issues about the problem you're having with vim.

[masongup-mdsol]

No problem

[RandomDSdevel]

@MikeMcQuaid: Shouldn't you or @ilovezfs change this issue's title back to what it was before now that @masongup-mdsol has forked discussion of his issue over to a different thread?

(Never mind, I confused what the name change meant and the context in which it occurred in the first place, so you can ignore this post, hence me striking through what it contained before I added this disclaimer.)

[RandomDSdevel]

Another thought that occurred to me the other day is this: what if this issue is the result of a corner cased that was overlooked when creating the code designed to manage going through iterations of the strategy originally introduced into Homebrew in #1052 when/after that PR landed?

P. S.: Oh, wait: I'm mixing this issue and the long-closed/-locked #1656 I originally filed from which this issue's discussion was forked up in my head. As such, I admit that this post was somewhat non sequitur and thus doubt it has anything to do with this issue thread here.

[iansinnott]

👍 This seems to be an issue for upgrading packages as @masongup-mdsol noted above. I'd just like to add another example:

brew upgrade yarn --ignore-dependencies

Brew will _still_ try to install the latest node even with the ignore dependencies option.

Brew info:

Homebrew 1.2.3
Homebrew/homebrew-core (git revision 4b3f; last commit 2017-07-01)

Of course for anyone else running in to the upgrade issue a quick workaround is to simply reinstall:

brew uninstall yarn && brew install yarn --ignore-dependencies

[jrz]

Can we not have a file that saves the options for all packages? It's super annoying to keep running into problems when doing a brew upgrade or reinstall

[ilovezfs]

meet https://github.com/Homebrew/homebrew-bundle

[amyspark]

All, I'd like to sum up what's happening here. After a given Formula was installed, brew upgrade:

1. ignores all build flags except those that were used when installing
2. forgets to use --ignore-dependencies if it was specified when installing

(EDIT: This explanation has been updated because I couldn't find the source of upgrade.rb's build flags.)

For (1), install.rb gets the formulae through ARGV.formulae :
https://github.com/Homebrew/brew/blob/dfe0a1eb898dcee72dc113737c79bff588383130/Library/Homebrew/extend/ARGV.rb#L41-L50
And the Formulary.factory (after lots of hoops 😩 ) reads the flags through software_spec.rb:
https://github.com/Homebrew/brew/blob/dfe0a1eb898dcee72dc113737c79bff588383130/Library/Homebrew/software_spec.rb#L43

On the other hand, upgrade only uses the ones stored in the Tab because it loads the formulae via ARGV.resolved_formulae, which explicitly overrides the build options with the Tab's:
https://github.com/Homebrew/brew/blob/dfe0a1eb898dcee72dc113737c79bff588383130/Library/Homebrew/extend/ARGV.rb#L57-L61

For (2), --ignore-dependencies is not stored anywhere in the Tab,
https://github.com/Homebrew/brew/blob/dfe0a1eb898dcee72dc113737c79bff588383130/Library/Homebrew/tab.rb#L20-L50
It's also never taken into account by upgrade (see the upgrade.rb block above), but it is read by install (see the install.rb block above too).

Please let me know if I've understood this correctly, and if so, I'll try to fix it.
Thanks for your comments!

[MikeMcQuaid]

ignores all build flags except those that were used when installing

This is neither intentional or unintentional but doesn't seem like terrible behaviour.

forgets to use --ignore-dependencies if it was specified when installing

--ignore-dependencies should be seen as a developer flag; most things will not work without their dependencies installed.

Arguably in both cases the only issue is the documentation being ambiguous that this behaviour brew upgrade --different-option is not supported.

[MikeMcQuaid]

But I guess if we can make brew upgrade --different-option work (given brew reinstall --different-option andbrew install --different-option` both work). @amyspark let me know if you plan to work on that otherwise I will, thanks.

[amyspark]

@MikeMcQuaid I've got a patch ready, I'll submit it asap.