Homebrew-core: Wrong Python directory when installing MacVim from bottle

Created on 30 Sep 2016 · 11Comments · Source: Homebrew/homebrew-core

Please follow the general troubleshooting steps first:

[x] Ran brew update and retried your prior step?
[x] Ran brew doctor, fixed as many issues as possible and retried your prior step?
[x] If you're seeing permission errors tried running sudo chown -R $(whoami) $(brew --prefix)?

After updating the homebrew version of python from 2.7.12 to 2.7.12_1 (https://github.com/Homebrew/homebrew-core/commit/6b08d86854de8668cb873f8fb27acd61296e19a5), the path to the homebrew python that MacVim uses is incorrect when installing MacVim from a bottle. This results in the following error when starting up MacVim and attempting to execute some python vimscript:

Error detected while processing function <SNR>36_InitializePythonBuiltin:
line   23:
E887: Sorry, this command is disabled, the Python's site module could not be loaded.

Notice the python path in the following commands. In the former, python is installed to /usr/local/Cellar/python/2.7.12_1, but MacVim is looking for it in /usr/local/Cellar/python/2.7.12.

# brew --config
HOMEBREW_VERSION: 1.0.5-43-g41b2df8
ORIGIN: https://github.com/Homebrew/brew
HEAD: 41b2df8e338c3fe345afc89b1bb394e824213561
Last commit: 8 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: a5fba74ccdfc3f54aceb9defccd653beb64134af
Core tap last commit: 73 minutes ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: quad-core 64-bit broadwell
Homebrew Ruby: 2.0.0-p648
Clang: 8.0 build 800
Git: 2.10.0 => /usr/local/bin/git
Perl: /usr/bin/perl
Python: /usr/local/bin/python => /usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Ruby: /usr/local/var/ruby-build/ruby-2.3.0/bin/ruby
Java: 1.8.0_77
macOS: 10.12-x86_64
Xcode: 8.0
CLT: 8.0.0.0.1.1472435881
X11: N/A

# mvim --version
VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Sep 16 2016 20:34:19)
MacOS X (unix) version
Included patches: 1-3
Compiled by Homebrew
Huge version with MacVim GUI.  Features included (+) or not (-):
+acl             +find_in_path    -mouse_sysmouse  -tag_any_white
+arabic          +float           +mouse_urxvt     +tcl
+autocmd         +folding         +mouse_xterm     +termguicolors
+balloon_eval    -footer          +multi_byte      +terminfo
+browse          +fork()          +multi_lang      +termresponse
++builtin_terms  +fullscreen      -mzscheme        +textobjects
+byte_offset     -gettext         +netbeans_intg   +timers
+channel         -hangul_input    +num64           +title
+cindent         +iconv           +odbeditor       +toolbar
+clientserver    +insert_expand   +packages        +transparency
+clipboard       +job             +path_extra      +user_commands
+cmdline_compl   +jumplist        +perl            +vertsplit
+cmdline_hist    +keymap          +persistent_undo +virtualedit
+cmdline_info    +lambda          +postscript      +visual
+comments        +langmap         +printer         +visualextra
+conceal         +libcall         +profile         +viminfo
+cryptv          +linebreak       +python          +vreplace
+cscope          +lispindent      -python3         +wildignore
+cursorbind      +listcmds        +quickfix        +wildmenu
+cursorshape     +localmap        +reltime         +windows
+dialog_con_gui  -lua             +rightleft       +writebackup
+diff            +menu            +ruby            -X11
+digraphs        +mksession       +scrollbind      -xfontset
+dnd             +modify_fname    +signs           +xim
-ebcdic          +mouse           +smartindent     -xpm
+emacs_tags      +mouseshape      +startuptime     -xsmp
+eval            +mouse_dec       +statusline      -xterm_clipboard
+ex_extra        -mouse_gpm       -sun_workshop    -xterm_save
+extra_search    -mouse_jsbterm   +syntax          
+farsi           +mouse_netterm   +tag_binary      
+file_in_path    +mouse_sgr       +tag_old_static  
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/Applications/MacVim.app/Contents/Resources/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_MACVIM -Wall -Wno-unknown-pragmas -pipe  -DMACOS_X_UNIX  -F/usr/local/Cellar/python/2.7.12/Frameworks -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: clang   -L. -L/usr/local/lib -L. -L/usr/local/lib -F/usr/local/Cellar/python/2.7.12/Frameworks -L/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython2.7 -framework CoreFoundation -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon       -lm  -lncurses -liconv -framework Cocoa   -fstack-protector  -L/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE -lperl -F/usr/local/Cellar/python/2.7.12/Frameworks -framework Python  -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -framework Ruby

I've tried reinstalling MacVim several times (brew reinstall macvim, brew rm macvim && brew install macvim), but the same issue happens every time. However, if I build MacVim from source with homebrew (brew install -s macvim), the path is correct, and the error does not appear.

Thanks, and please let me know if I can provide more information.

python

Source

theunraveler

👍3 😕1

Most helpful comment

This patch seems to work. It's also possible we should be rewriting the Python sysconfig data to return our favorite paths instead:

diff --git a/Formula/vim.rb b/Formula/vim.rb
index ee8d430..0448b5e 100644
--- a/Formula/vim.rb
+++ b/Formula/vim.rb
@@ -101,6 +101,16 @@ class Vim < Formula
                           "--enable-cscope",
                           "--with-compiledby=Homebrew",
                           *opts
+
+    # Remember a less fragile path for Python
+    begin
+      inreplace "src/auto/config.mk",
+                %r(PYTHON_HOME='"#{HOMEBREW_CELLAR}/(python3?)/[^/]+/(.*?)"'),
+                %(PYTHON_HOME='"#{HOMEBREW_PREFIX}/opt/\\1/\\2"')
+    rescue InreplaceError
+      ohai "No Python Cellar references found in config.mk (this is okay!)"
+    end
+
     system "make"
     # Parallel install could miss some symlinks
     # https://github.com/vim/vim/issues/1031

tdsmith on 3 Oct 2016

❤1 🎉1 👍1

All 11 comments

I have the same issue, thanks for the tip, brew install -s macvim did the trick for me as a workaround.

sylvainsf on 30 Sep 2016

👍1

Does this cause actual runtime issues? We rewrite the linkage on things to not point to Cellar paths, such as:

/usr/local/bin/vim:
    /usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
    /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 22.0.0)
    /usr/local/opt/luajit/lib/libluajit-5.1.2.dylib (compatibility version 2.0.0, current version 2.0.4)
    /usr/local/opt/perl/lib/perl5/5.24.0/darwin-thread-multi-2level/CORE/libperl.dylib (compatibility version 5.24.0, current version 5.24.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
    /usr/lib/libutil.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/opt/python/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.0)
    /usr/local/opt/ruby/lib/libruby.2.3.0.dylib (compatibility version 2.3.1, current version 2.3.0)

So the linkage doesn't break across revisions.

Vim and MacVim remember the paths they were compiled with, but aren't aware that we fix linkage postinstall to be more resistant to Cellar changes, so whilst the output may look concerning I'm unsure if there's any real runtime impact here?

DomT4 on 1 Oct 2016

👍2

The reports here show that there are runtime issues.

The problem isn't the install_name; see stringswhich vim| grep Cellar. Maybe we can convince vim to remember something else by rewriting a header after configure or something.

tdsmith on 3 Oct 2016

😕1 👍1

Yes--as @tdsmith said, there are runtime issues whenever you try to execute python code in vimscript.

Error detected while processing function <SNR>36_InitializePythonBuiltin:
line   23:
E887: Sorry, this command is disabled, the Python's site module could not be loaded.

theunraveler on 3 Oct 2016

👍1

Grepping through the source tree after running configure I'm betting the thing we want to redefine is PYTHON_HOME:

./src/auto/config.mk:PYTHON_CFLAGS  = -I/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/include/python2.7 -DPYTHON_HOME='"/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7"' -fPIE

tdsmith on 3 Oct 2016

🎉1 😄1 👍1

This patch seems to work. It's also possible we should be rewriting the Python sysconfig data to return our favorite paths instead:

diff --git a/Formula/vim.rb b/Formula/vim.rb
index ee8d430..0448b5e 100644
--- a/Formula/vim.rb
+++ b/Formula/vim.rb
@@ -101,6 +101,16 @@ class Vim < Formula
                           "--enable-cscope",
                           "--with-compiledby=Homebrew",
                           *opts
+
+    # Remember a less fragile path for Python
+    begin
+      inreplace "src/auto/config.mk",
+                %r(PYTHON_HOME='"#{HOMEBREW_CELLAR}/(python3?)/[^/]+/(.*?)"'),
+                %(PYTHON_HOME='"#{HOMEBREW_PREFIX}/opt/\\1/\\2"')
+    rescue InreplaceError
+      ohai "No Python Cellar references found in config.mk (this is okay!)"
+    end
+
     system "make"
     # Parallel install could miss some symlinks
     # https://github.com/vim/vim/issues/1031

tdsmith on 3 Oct 2016

❤1 🎉1 👍1

@tdsmith I think it's worth checking if it's present rather than rescuing. Additionally, just to check, can we set that as an environment variable instead?