pip shows matplotlib (1.3.1) installed in the operating system by default but fails to be updated to (2.0.0) using pip install --upgrade matplotlib

[Albert-Z-Guo]

• Pip version: 9.0.1
• Python version: 2.7.11
• Operating system: macOS Sierra (10.12.3)

Description:

pip shows matplotlib (1.3.1) installed in the operating system by default but fails to be updated to the latest version (2.0.0) using pip install --upgrade matplotlib

What I've run:

$ pip show matplotlib

Name: matplotlib
Version: 1.3.1
Summary: Python plotting package
Home-page: http://matplotlib.org
Author: John D. Hunter, Michael Droettboom
Author-email: [email protected]
License: BSD
Location: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
Requires: numpy, python-dateutil, tornado, pyparsing, nose

I then tried:
$ pip install --upgrade matplotlib

Collecting matplotlib
Using cached matplotlib-2.0.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Requirement already up-to-date: python-dateutil in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Requirement already up-to-date: subprocess32 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Requirement already up-to-date: cycler>=0.10 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Requirement already up-to-date: six>=1.10 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Requirement already up-to-date: pytz in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Requirement already up-to-date: functools32 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Requirement already up-to-date: numpy>=1.7.1 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Requirement already up-to-date: pyparsing!=2.0.0,!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from matplotlib)
Installing collected packages: matplotlib
Found existing installation: matplotlib 1.3.1
Uninstalling matplotlib-1.3.1:
Exception:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 778, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
copytree(src, real_dst, symlinks=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/__init__.py', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/__init__.py', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/__init__.pyc', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/__init__.pyc', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/_cm.py', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/_cm.py', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/_cm.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/_cm.pyc',
...

the error message is very long and in the end it's like this:

[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/type1font.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/units.py', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/units.py', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/units.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/units.pyc', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/units.pyc', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/units.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/widgets.py', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/widgets.py', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/widgets.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/widgets.pyc', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/widgets.pyc', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/widgets.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib', '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib', "[Errno 1] Operation not permitted: '/var/folders/rs/ss84ccns2r197b9k04gw6_k80000gn/T/pip-qdKKlO-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib'")]

[pfmoore]

The Location: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python line shows that the matplotlib 1.3.1 instance is in an operating-system owned location. The error Operation not permitted suggests that you don't have permission to write to OS-owned locations. So assuming that's the issue, pip is correctly reporting the problem.

That seems right to me, as I'd expect you to upgrade your OS python using OS-supplied packages. But I know nothing about how OSX works, so I don't really know how the system Python is managed.

I believe others when using Python on OSX, use virtualenvs, or other distributions of Python (macports?). But you'd need advice from OSX users on the best solution in that case.

[Albert-Z-Guo]

OKay. But how does pip manage packages in different locations?

When I upgrade packages, pip usually upgrades them in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
and all the packages I have are installed there.

Previously, I had a version of matplotlib (2.0.0) in:
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

But this time, after I uninstall matplotlib there, pip refers to the system package of matplotlib (1.3.1) in:
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
And I couldn't install the latest version of matplotlib again in
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

[pfmoore]

All pip does is use the installation scheme that the underlying Python interpreter uses, via sysconfig.get_paths() (it uses the "*_user" scheme if --user is set, otherwise it uses the appropriate platform default scheme ("nt" on Windows, I presume "posix_prefix" on Unix).

Is this installation the standard OS installation of Python on OSX? If so, I'm not clear why you're getting some things installed in /System/Library and some in /Library. That's probably a peculiarity of the Python environment, maybe something Apple added.

You probably need help from someone who knows how the OSX Python is set up. I don't use OSX so I can only really comment on pip's generic behaviour, which doesn't seem to be the issue here.

[Albert-Z-Guo]

I removed my Python 2.7.11 and installed the latest version through MacPorts. The above issue disappear when installing dependent packages through MacPorts.

[pradyunsg]

Basically, you're trying to upgrade a system package when you (rightly) don't have the permissions to do so. System packages should be managed by the OS.

If you want to install a newer matplotlib, you can use --user to do so: pip install --user --upgrade matplotlib.