Homebrew-core: vim: installation fails when using pyenv

[johnpneumann]

Please follow the general troubleshooting steps first:

• [x] Ran brew update and retried your prior step?
• [x] Ran brew doctor, fixed as many issues as possible and retried your prior step?
• [x] If you're seeing permission errors tried running sudo chown -R $(whoami) $(brew --prefix)?
  

  Bug reports:

This issue was originally reported to pyenv, here: https://github.com/yyuu/pyenv/issues/589

When running brew install/upgrade vim and pyenv, an error occurs when attempting to upgrade the package. This appears to be due to incompatible unicode formats.

https://gist.github.com/d9f660ae9f77b8ae449335df283247be

Steps to reproduce

Requirements:

• pyenv
• vim uninstalled

Install

pyenv global 2.7.11
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/e0de5d3e7bf9976ea9232b083add47fc9106e09e/Formula/vim.rb --with-python --with-cscope --with-lua --with-ruby --with-luajit --with-perl

Upgrade

pyenv global system
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/e0de5d3e7bf9976ea9232b083add47fc9106e09e/Formula/vim.rb --with-python --with-cscope --with-lua --with-ruby --with-luajit --with-perl
pyenv global 2.7.11
brew upgrade vim

Not sure what the best approach here is. Checking to see if pyenv is available seems hacky, but workable.

[adrianliaw]

+1
And I need version 7.4.1830 because of Nvim-R plugin

[MikeMcQuaid]

I'm not sure how to suggest fixing this beyond "don't use pyenv when building vim".

[MikeMcQuaid]

And I need version 7.4.1830 because of Nvim-R plugin

@adrianliaw Then submit a pull request, thanks.

[tylucaskelley]

Bumping this, don't think it's fair to just say "don't use pyenv" when it's the largest and most well-supported Python environment management tool.

[UniqMartin]

For what it's worth, the suggestion was “don't use pyenv when building vim” not “don't use pyenv”. That's quite a difference. If you have any practical suggestions on how to improve the situation, we're happy to hear them, or better yet, to receive a pull request with an implementation of the suggestion. We're a small team and not all of us are intimately familiar with the Python ecosystem or pyenv, thus we rely on contributions from the community to resolve these kinds of problems.

[chdiza]

I don't see any evidence in this thread that this is a Homebrew problem. Does it happen when building Vim outside of Homebrew? Vim's been having Perl-linking issues lately, and is notoriously flaky about linking to scripting languages in general on all kinds of platforms.

Furthermore, for all I can tell, this is a pyenv problem and not a Vim problem _or_ a HB problem. At the bottom of the first linked bug report a user posts a fix, and it involves tweaking _building python with pyenv_. And earlier in that same thread, a pyenv _maintainer_ says to link a Homebrewed Vim against system python.

[MikeMcQuaid]

@tylucaskelley I think it's fair to say that as the implication is "don't use pyenv" as much as "not using pyenv is an acceptable workaround but we'll accept pull requests".

[qtfkwk]

I've been looking at this issue for a while and stumbled onto a similar general fix as the OP and works when upgrading all Homebrew packages and there happens to be a update for Vim (or not):

original=$(pyenv global)
pyenv global system
brew upgrade --all
pyenv global $original

My situation is complicated the fact that I'm trying to do this from within Python code but that's a completely different topic.

Agree that this isn't really a Homebrew or Vim issue, unless the suggestion is that they should be pyenv-aware? Argument against is that there are many such similar version managers and also for other languages (Perl and Ruby come to mind) and I don't see how/why either Homebrew or Vim could (or should) handle them all.

I did have success running brew install vim --without-python from within Python code. This is perhaps not ideal since it sidesteps the issue entirely, but if I'm not using Vim's Python integration... why bother with it, especially when it's causing errors?

If I happen to stumble onto a better solution, I'll post it back here.

[brennebeck]

Thanks @qtfkwk. Telling pyenv to use the system python (pyenv shell system) worked like a charm.

[tylucaskelley]

@MikeMcQuaid apologies for my somewhat aggressive tone on my previous comment. @qtfkwk thank you for the workaround! Good enough for me 😄

[akerl]

Is there not a way at the Formula level to have homebrew build it using the system python? I'll admit to being intrigued that it was using pyenv in the first place when building.

Given the crazy number of weird things that myself and other users could do to muck up their python/ruby/whatever environments at the user / dotfile level, I assumed that Homebrew would basically wipe the env / PATH when building, since it would be a nightmare trying to ensure that homebrew formula were resilient to whatever user env folks have set up.

[MikeMcQuaid]

@tylucaskelley No worries, appreciate the apology!

[qtfkwk]

Here's an alternative method without the system-wide side effects of using pyenv global:

pyenv shell system
brew upgrade --all
pyenv shell --unset

In my (possibly niche) situation (as alluded to above, trying to run from within Python code, for the ratom homebrew plugin), the following is working. Btw, this is needed because pyenv shell won't work here. Also note that prefixing the homebrew upgrade command with pyenv exec fixes an issue where the upgrade command still fails, possibly related to which python still indicating pyenv's 2.7.11 version, which is due to the PATH environment variable still being set to its former value, caused by my generally running with pyenv global 2.7.11 instead of the system Python.

import os
import ratom
ratom.args()
v = os.environ['PYENV_VERSION']
os.environ['PYENV_VERSION'] = 'system'
ratom.run('pyenv exec brew upgrade --all')
os.environ['PYENV_VERSION'] = v

Also should point out for testing purposes, rather than waiting around for a vim update, you can substitute brew upgrade --all for brew uninstall vim and brew install vim.

[qtfkwk]

Regarding @akerl 's idea to address this in the vim formula... this is exactly what we were trying to avoid in that homebrew can't (or shouldn't be expected to) support edge cases in the vim formula for users who happen to use pyenv. To do it for this case would suggest that they should support all things like pyenv (such as rbenv and perlbrew) in all kinds of weird situations and unrelated formulas they happen to affect (like vim). IMHO this would lead to easier use in these edge cases but less stability overall in homebrew. Not fixing it in the formula allows/forces us to fix the problem closer to its source. I think that homebrew is doing the right thing by respecting the environment, rather than trying to second-guess or force the user into using its own idea of the "right" python.

[tdsmith]

This is ± a bug or a violated expectation wrt the vim build scripts; it's mixing and matching the (ucs4) headers from the pyenv python with -framework Python, which is picking up (ucs2) system Python. I don't think vim has an expectation about ucs2 vs ucs4 but Python is picking up a discordant config header which is leading to bad news. This is probably fixable and I think it would be a useful thing to fix.

[tdsmith]

(I think the proximate cause is that vim is expecting Python to be built as a framework on OS X and pyenv builds with a Unix layout but I need to take a closer look)

[MikeMcQuaid]

We'll accept PRs for this but we're not actively working on it at this time. We don't support using pyenv.

[MikeMcQuaid]

@tdsmith Are you currently actively working on this?

[MikeMcQuaid]

@tdsmith and: is it a Homebrew or upstream issue?

[tdsmith]

On Wed, Aug 10, 2016, 11:42 Mike McQuaid [email protected] wrote:

@tdsmith https://github.com/tdsmith and: is it a Homebrew or upstream
issue?

I think it's an integration issue; I want to support pyenv because we (I)
recommend it for Python 3.4 etc. Probably this story ends with a PR against
either macvim or pyenv. We can take this off the tracker if you prefer but
I do intend to resolve it.

[MikeMcQuaid]

@tdsmith I'd rather we kept upstream issues off the tracker as they aren't really Homebrew issues.

[tdsmith]

In the meantime, we should at least detect and report an incompatible build
instead of dying midway through; I'm pretty sure this is related to
framework builds vs flat builds and I'll test and propose a PR to close
this issue that way.

On Fri, Aug 12, 2016, 16:15 Mike McQuaid [email protected] wrote:

@tdsmith https://github.com/tdsmith I'd rather we kept upstream issues
off the tracker as they aren't really Homebrew issues.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/Homebrew/homebrew-core/issues/1165#issuecomment-239561554,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKnQak5Y6yhrhxzHSEs2I3FEPme0bchks5qfOJegaJpZM4IepXz
.

[tdsmith]

This was fixed by a PR against vim: vim/vim#980