Pip: _pip install_ broken beyond normal repair by Jupyter Notebook Extensions (AttributeError: _DistInfoDistribution__dep_map)
Environment
OS: Ubuntu 16.04 (nvidia/cuda:8.0-cudnn6-devel)
Python version: 3.6.5
Conda version:
conda 4.5.10
Pip version:
pip 18.0
Description
_Pip install_ stopped working during docker build of a complex docker container (based on Kaggle/docker-python) and it cannot be repaired by simple methods such as reinstalling / downgrading with pip itself, with conda or a Miniconda script, unless you change the default installation location (/opt/conda) to an alternative folder when using Minconda script to re-install pip (effectively moving all subsequent libraries installs with pip to a separate location). Note that _pip uninstall_ still works (and can be used to remove pip - apparently not effectively enough).
Expected behavior
_pip install_ should be working even in these conditions, i.e. _. pip install scipy_ should succeed regardless of the complexity of the installation.
How to Reproduce
I managed to make it reproducible by providing a functional but only partially completed docker container produced by _docker build_ just before the error occured (I verified that it is reproducible on two different servers).
The bug can be reproduced by executing _pip install_ in bash in this archived docker image that captures this bug (no login requried):
docker pull mirekphd/tst-ml-gpu-py36-cuda8-failing:001766c55ca8
docker run -it --rm --name test -u 0 mirekphd/tst-ml-gpu-py36-cuda8-failing:001766c55ca8 bash
pip install scipy
Possible causes
The last completed operation during build and hence the possible culprit was Jupyter Notebook (or Notebook Extensions) installation with conda (the only installer that makes Jupyter Notebook Extensions work in Jupyter Notebook - pip installation made earlier did not suffice). When I moved Jupyter (Notebook, Lab and Notebook Extensions) installation to the end of the build process (i.e. after all other libraries), the build completed and all libraries can be loaded (from a single standard location in /opt/conda). However, Jupyter section was once again the last one to succeed with _pip install_, and pip still gets broken, raising the same error (which can be even considered a feature not a bug as it helps maintain library / ML models reproducibility:). I also notice that the same error is raised by _pip show_ and pipdeptree (e.g. !pipdeptree --reverse --packages xgboost) but not by _pip freeze_.
An intial workaround
A complex but universal workaround I found was to change the pip installation location into an alternative folder when using Miniconda script to re-install pip.
Dockerfile excerpt that repairs this issue and allows the build to succeed:
# ARG MINICONDA_VER=4.5.4 # (same as defined above)
# ARG PIP_VER=18.0 # (same as defined above)
# define alternative pip location
ENV PIP_RESCUE_FOLDER=/tmp/opt/conda
RUN mkdir -p ${PIP_RESCUE_FOLDER} && cd ${PIP_RESCUE_FOLDER} && \
# download Miniconda installation script including pip binary
wget https://repo.continuum.io/miniconda/Miniconda3-${MINICONDA_VER}-Linux-x86_64.sh && \
# note we install pip (and entire Miniconda) in the secondary location by specifying the "prefix folder",
# using parameter -p (hence PREFIX=/tmp/opt/conda and not /opt/conda as it is usually done)
/bin/bash Miniconda3-${MINICONDA_VER}-Linux-x86_64.sh -f -b -p ${PIP_RESCUE_FOLDER} && \
# /tmp/opt/conda/bin/pip --version
# pip 10.0.1 from /tmp/opt/conda/lib/python3.6/site-packages/pip (python 3.6)
# update secondary pip to the desired version
/tmp/opt/conda/bin/pip install --upgrade pip==${PIP_VER} && \
# overwrite pip in primary location with the one just installed into secondary one
cp /tmp/opt/conda/bin/pip /opt/conda/bin && \
cp -R /tmp/opt/conda/lib/python3.6/site-packages/pip /opt/conda/lib/python3.6/site-packages
# to make the libraries installed by secondary pip (into the pip rescue folder) work in python
# we need to add this secondary bin folder to PATH
ENV PATH=$PIP_RESCUE_FOLDER/bin:$PATH
# to verify it works:
# $python
# >>> import openshift as oc
# >>> oc
# <module 'openshift' from '/tmp/opt/conda/lib/python3.6/site-packages/openshift/__init__.py'>
Output
The expected error message is:
root@2dba6ccb6b94:/# pip install scipy --no-cache-dir
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Requirement already satisfied: numpy>=1.8.2 in /opt/conda/lib/python3.6/site-packages (from scipy) (1.15.2)
Exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
return self.__dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
raise exc
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
loc, tokens = self._parse( instring, 0 )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
super(Requirement, self).__init__(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
status = self.run(options, args)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
self._warn_about_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
package_set, _dep_info = check_install_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
package_set = create_package_set_from_installed()
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
package_set[name] = PackageDetails(dist.version, dist.requires())
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
dm = self._dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
self.__dep_map = self._compute_dependencies()
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
reqs.extend(parse_requirements(req))
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
yield Requirement(line)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"
root@2dba6ccb6b94:/#
mirekphd
All 18 comments
The culprit that broke _pip install_ was not Jupyter Notebook itself (already installed successfully using pip earlier in the build sequence), but its Extensions / Configurator installed with conda (!) in the versions specified below:
# Jupyter notebook extensions
# caution: Jupyter notebook has to be pinned, because Jupyter notebook extensions
# not always work with latest versions of notebook
# pin Jupyter Notebook
ARG NB_VER=5.6.0 聽
# pin Jupyter notebook extensions
ARG NB_EXT_VER=0.5.0
# pin Jupyter notebook extensions configurator
ARG NB_EXT_CFG_VER=0.4.0
# caution: jupyter notebook extensions do not work unless installed with conda (pip does not suffice)
RUN conda install -c conda-forge notebook=${NB_VER} \
jupyter_contrib_nbextensions=${NB_EXT_VER} \
jupyter_nbextensions_configurator=${NB_EXT_CFG_VER}
Note that not installing these extensions (which prevents pip from breaking down) is not the same as simply uninstalling them (which does not work - if these extensions are installed even once, pip gets broken beyond repair). If that breakage happened outside boundaries of an isolated container environment, it would be pretty serious.
mirekphd
on 1 Oct 2018
It looks like #3704 fixed a similar issue for pip freeze, but this error is coming from the new functionality that checks all installed packages on every pip install. I think a similar try/except is needed around this line:
I'd also say that the whole check operation should be try/excepted, because bugs in that shouldn't cause pip install to blow up. I think this would be the relevant line:
I think it's a bug in Flit causing at least some of these problems - sorry about that, and hopefully I've got it fixed now. But clearly an installed package with broken metadata shouldn't break pip like this.
takluyver
on 2 Oct 2018
I've had a go at fixing this in #5842 .
takluyver
on 2 Oct 2018
I've found this particular issue in older versions of pip as well, including 10 and 9.
astrojuanlu
on 2 Oct 2018
An update: in my complex python environment upgrading to the the latest versions of all libraries that appeared since the original bug report (including pip 18.0->18.1 and testpath 0.4 -> 0.42) did not help - installing latest Jupyter+extensions+configurator breaks down pip beyond normal repair, although error messages are different this time. In simple environments such as _jupyter/base-notebook_ this error would go unnoticed.
And it's not testpath's fault. If jupyter alone is installed (without extensions and configurator), than no version of testpath can break it (tried versions from 0.3.1. to 0.4.2).
More details
[root@mx01 ml-gpu-py36-cuda8]# docker exec -it ml-gpu-py36-cuda8 bash
root@0d9fb3d3f1ff:~# conda install -y -c conda-forge notebook jupyter_contrib_nbextensions jupyter_nbextensions_configurator
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.10
latest version: 4.5.11
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: /opt/conda
added / updated specs:
- jupyter_contrib_nbextensions
- jupyter_nbextensions_configurator
- notebook
The following packages will be downloaded:
package | build
---------------------------|-----------------
libsodium-1.0.16 | h470a237_1 536 KB conda-forge
mistune-0.8.3 | py36h470a237_2 873 KB conda-forge
ipykernel-5.0.0 | pyh24bf2e0_1 76 KB conda-forge
zeromq-4.2.5 | hfc679d8_6 939 KB conda-forge
jupyter_core-4.4.0 | py_0 44 KB conda-forge
pyzmq-17.1.2 | py36hae99301_0 1.0 MB conda-forge
jinja2-2.10 | py_1 89 KB conda-forge
entrypoints-0.2.3 | py36_1002 9 KB conda-forge
html5lib-1.0.1 | py_0 89 KB conda-forge
webencodings-0.5.1 | py_1 12 KB conda-forge
terminado-0.8.1 | py36_1001 23 KB conda-forge
notebook-5.7.0 | py36_1000 7.3 MB conda-forge
testpath-0.4.2 | py36_1000 91 KB conda-forge
jupyter_contrib_nbextensions-0.5.0| py36_0 19.4 MB conda-forge
pandocfilters-1.4.2 | py_1 9 KB conda-forge
jupyter_highlight_selected_word-0.2.0| py36_1000 13 KB conda-forge
jupyter_nbextensions_configurator-0.4.0| py36_0 470 KB conda-forge
jupyter_contrib_core-0.3.3 | py_2 15 KB conda-forge
jupyter_latex_envs-1.4.4 | py36_0 753 KB conda-forge
jsonschema-2.6.0 | py36_1002 62 KB conda-forge
pandoc-2.3.1 | 0 20.7 MB conda-forge
lxml-4.2.5 | py36hc9114bc_0 6.0 MB conda-forge
send2trash-1.5.0 | py_0 12 KB conda-forge
libxslt-1.1.32 | h88dbc4e_2 511 KB conda-forge
prometheus_client-0.4.0 | py_0 32 KB conda-forge
jupyter_client-5.2.3 | py_1 59 KB conda-forge
bleach-3.0.0 | py_0 108 KB conda-forge
nbconvert-5.3.1 | py_1 320 KB conda-forge
nbformat-4.4.0 | py_1 91 KB conda-forge
------------------------------------------------------------
Total: 59.6 MB
The following NEW packages will be INSTALLED:
bleach: 3.0.0-py_0 conda-forge
entrypoints: 0.2.3-py36_1002 conda-forge
html5lib: 1.0.1-py_0 conda-forge
ipykernel: 5.0.0-pyh24bf2e0_1 conda-forge
jinja2: 2.10-py_1 conda-forge
jsonschema: 2.6.0-py36_1002 conda-forge
jupyter_client: 5.2.3-py_1 conda-forge
jupyter_contrib_core: 0.3.3-py_2 conda-forge
jupyter_contrib_nbextensions: 0.5.0-py36_0 conda-forge
jupyter_core: 4.4.0-py_0 conda-forge
jupyter_highlight_selected_word: 0.2.0-py36_1000 conda-forge
jupyter_latex_envs: 1.4.4-py36_0 conda-forge
jupyter_nbextensions_configurator: 0.4.0-py36_0 conda-forge
libsodium: 1.0.16-h470a237_1 conda-forge
libxslt: 1.1.32-h88dbc4e_2 conda-forge
lxml: 4.2.5-py36hc9114bc_0 conda-forge
mistune: 0.8.3-py36h470a237_2 conda-forge
nbconvert: 5.3.1-py_1 conda-forge
nbformat: 4.4.0-py_1 conda-forge
notebook: 5.7.0-py36_1000 conda-forge
pandoc: 2.3.1-0 conda-forge
pandocfilters: 1.4.2-py_1 conda-forge
prometheus_client: 0.4.0-py_0 conda-forge
pyzmq: 17.1.2-py36hae99301_0 conda-forge
send2trash: 1.5.0-py_0 conda-forge
terminado: 0.8.1-py36_1001 conda-forge
testpath: 0.4.2-py36_1000 conda-forge
webencodings: 0.5.1-py_1 conda-forge
zeromq: 4.2.5-hfc679d8_6 conda-forge
Downloading and Extracting Packages
libsodium-1.0.16 | 536 KB | #################################################################################################################################### | 100%
mistune-0.8.3 | 873 KB | #################################################################################################################################### | 100%
ipykernel-5.0.0 | 76 KB | #################################################################################################################################### | 100%
zeromq-4.2.5 | 939 KB | #################################################################################################################################### | 100%
jupyter_core-4.4.0 | 44 KB | #################################################################################################################################### | 100%
pyzmq-17.1.2 | 1.0 MB | #################################################################################################################################### | 100%
jinja2-2.10 | 89 KB | #################################################################################################################################### | 100%
entrypoints-0.2.3 | 9 KB | #################################################################################################################################### | 100%
html5lib-1.0.1 | 89 KB | #################################################################################################################################### | 100%
webencodings-0.5.1 | 12 KB | #################################################################################################################################### | 100%
terminado-0.8.1 | 23 KB | #################################################################################################################################### | 100%
notebook-5.7.0 | 7.3 MB | #################################################################################################################################### | 100%
testpath-0.4.2 | 91 KB | #################################################################################################################################### | 100%
jupyter_contrib_nbex | 19.4 MB | #################################################################################################################################### | 100%
pandocfilters-1.4.2 | 9 KB | #################################################################################################################################### | 100%
jupyter_highlight_se | 13 KB | #################################################################################################################################### | 100%
jupyter_nbextensions | 470 KB | #################################################################################################################################### | 100%
jupyter_contrib_core | 15 KB | #################################################################################################################################### | 100%
jupyter_latex_envs-1 | 753 KB | #################################################################################################################################### | 100%
jsonschema-2.6.0 | 62 KB | #################################################################################################################################### | 100%
pandoc-2.3.1 | 20.7 MB | #################################################################################################################################### | 100%
lxml-4.2.5 | 6.0 MB | #################################################################################################################################### | 100%
send2trash-1.5.0 | 12 KB | #################################################################################################################################### | 100%
libxslt-1.1.32 | 511 KB | #################################################################################################################################### | 100%
prometheus_client-0. | 32 KB | #################################################################################################################################### | 100%
jupyter_client-5.2.3 | 59 KB | #################################################################################################################################### | 100%
bleach-3.0.0 | 108 KB | #################################################################################################################################### | 100%
nbconvert-5.3.1 | 320 KB | #################################################################################################################################### | 100%
nbformat-4.4.0 | 91 KB | #################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: | + /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''jupyter_highlight_selected_word'\'', sys_prefix=True, logger=logging.getLogger())'
\ + /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''latex_envs'\'', sys_prefix=True, logger=logging.getLogger())'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
m, nbexts = _get_nbextension_metadata(module)
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
m = import_item(module)
File "/opt/conda/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
return __import__(parts[0])
File "/opt/conda/lib/python3.6/site-packages/latex_envs/__init__.py", line 3, in <module>
from . import latex_envs
File "/opt/conda/lib/python3.6/site-packages/latex_envs/latex_envs.py", line 19, in <module>
from IPython.display import HTML, display,JSON
File "/opt/conda/lib/python3.6/site-packages/IPython/__init__.py", line 55, in <module>
from .terminal.embed import embed
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/embed.py", line 16, in <module>
from IPython.terminal.interactiveshell import TerminalInteractiveShell
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 18, in <module>
from prompt_toolkit.document import Document
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
from .interface import CommandLineInterface
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/interface.py", line 19, in <module>
from .application import Application, AbortAction
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/__init__.py", line 2, in <module>
from .application import Application
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/application.py", line 3, in <module>
from prompt_toolkit.buffer import Buffer
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 9, in <module>
from .completion import Completer, Completion, CompleteEvent
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/__init__.py", line 2, in <module>
from .base import Completion, Completer, ThreadedCompleter, DummyCompleter, DynamicCompleter, CompleteEvent, merge_completers, get_common_complete_suffix
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/base.py", line 4, in <module>
from prompt_toolkit.eventloop import generator_to_async_generator, AsyncGeneratorItem
ImportError: cannot import name 'generator_to_async_generator'
failed
ERROR conda.core.link:_execute(502): An error occurred while installing package 'conda-forge::jupyter_latex_envs-1.4.4-py36_0'.
LinkError: post-link script failed for package conda-forge::jupyter_latex_envs-1.4.4-py36_0
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.jupyter_latex_envs-post-link.sh
==> script messages <==
+ /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''latex_envs'\'', sys_prefix=True, logger=logging.getLogger())'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
m, nbexts = _get_nbextension_metadata(module)
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
m = import_item(module)
File "/opt/conda/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
return __import__(parts[0])
File "/opt/conda/lib/python3.6/site-packages/latex_envs/__init__.py", line 3, in <module>
from . import latex_envs
File "/opt/conda/lib/python3.6/site-packages/latex_envs/latex_envs.py", line 19, in <module>
from IPython.display import HTML, display,JSON
File "/opt/conda/lib/python3.6/site-packages/IPython/__init__.py", line 55, in <module>
from .terminal.embed import embed
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/embed.py", line 16, in <module>
from IPython.terminal.interactiveshell import TerminalInteractiveShell
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 18, in <module>
from prompt_toolkit.document import Document
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
from .interface import CommandLineInterface
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/interface.py", line 19, in <module>
from .application import Application, AbortAction
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/__init__.py", line 2, in <module>
from .application import Application
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/application.py", line 3, in <module>
from prompt_toolkit.buffer import Buffer
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 9, in <module>
from .completion import Completer, Completion, CompleteEvent
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/__init__.py", line 2, in <module>
from .base import Completion, Completer, ThreadedCompleter, DummyCompleter, DynamicCompleter, CompleteEvent, merge_completers, get_common_complete_suffix
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/base.py", line 4, in <module>
from prompt_toolkit.eventloop import generator_to_async_generator, AsyncGeneratorItem
ImportError: cannot import name 'generator_to_async_generator'
Attempting to roll back.
Rolling back transaction: done
LinkError: post-link script failed for package conda-forge::jupyter_latex_envs-1.4.4-py36_0
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.jupyter_latex_envs-post-link.sh
==> script messages <==
+ /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''latex_envs'\'', sys_prefix=True, logger=logging.getLogger())'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
m, nbexts = _get_nbextension_metadata(module)
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
m = import_item(module)
File "/opt/conda/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
return __import__(parts[0])
File "/opt/conda/lib/python3.6/site-packages/latex_envs/__init__.py", line 3, in <module>
from . import latex_envs
File "/opt/conda/lib/python3.6/site-packages/latex_envs/latex_envs.py", line 19, in <module>
from IPython.display import HTML, display,JSON
File "/opt/conda/lib/python3.6/site-packages/IPython/__init__.py", line 55, in <module>
from .terminal.embed import embed
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/embed.py", line 16, in <module>
from IPython.terminal.interactiveshell import TerminalInteractiveShell
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 18, in <module>
from prompt_toolkit.document import Document
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
from .interface import CommandLineInterface
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/interface.py", line 19, in <module>
from .application import Application, AbortAction
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/__init__.py", line 2, in <module>
from .application import Application
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/application.py", line 3, in <module>
from prompt_toolkit.buffer import Buffer
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 9, in <module>
from .completion import Completer, Completion, CompleteEvent
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/__init__.py", line 2, in <module>
from .base import Completion, Completer, ThreadedCompleter, DummyCompleter, DynamicCompleter, CompleteEvent, merge_completers, get_common_complete_suffix
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/base.py", line 4, in <module>
from prompt_toolkit.eventloop import generator_to_async_generator, AsyncGeneratorItem
ImportError: cannot import name 'generator_to_async_generator'
root@0d9fb3d3f1ff:~# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@0d9fb3d3f1ff:~# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@0d9fb3d3f1ff:~# pip install --upgrade pip
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
Downloading https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl (1.3MB)
100% |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 1.3MB 10.8MB/s
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@0d9fb3d3f1ff:~# conda install --upgrade pip
usage: conda [-h] [-V] command ...
conda: error: unrecognized arguments: --upgrade
root@0d9fb3d3f1ff:~# conda install pip
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.10
latest version: 4.5.11
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: /opt/conda
added / updated specs:
- pip
The following packages will be downloaded:
package | build
---------------------------|-----------------
pip-18.1 | py36_1000 1.8 MB conda-forge
The following packages will be UPDATED:
pip: 10.0.1-py36_0 defaults --> 18.1-py36_1000 conda-forge
Proceed ([y]/n)? y
Downloading and Extracting Packages
pip-18.1 | 1.8 MB | #################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
root@0d9fb3d3f1ff:~# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@9fa47a5e4577:~# pip uninstall pip
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Uninstalling pip-18.0:
Would remove:
/opt/conda/bin/pip
/opt/conda/bin/pip3
/opt/conda/bin/pip3.6
/opt/conda/lib/python3.6/site-packages/pip-18.0.dist-info/*
/opt/conda/lib/python3.6/site-packages/pip/*
Proceed (y/n)? y
Successfully uninstalled pip-18.0
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
root@9fa47a5e4577:~# conda install pip
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.10
latest version: 4.5.11
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: /opt/conda
added / updated specs:
- pip
The following packages will be downloaded:
package | build
---------------------------|-----------------
pip-18.1 | py36_1000 1.8 MB conda-forge
The following packages will be UPDATED:
pip: 10.0.1-py36_0 defaults --> 18.1-py36_1000 conda-forge
Proceed ([y]/n)? y
Downloading and Extracting Packages
pip-18.1 | 1.8 MB | ################################################################################################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
root@9fa47a5e4577:~# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@9fa47a5e4577:~# conda remove mistune
Solving environment: failed
PackagesNotFoundError: The following packages are missing from the target environment:
- mistune
root@9fa47a5e4577:~# pip uninstall mistune
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 143, in main
status = self.run(options, args)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/uninstall.py", line 75, in run
auto_confirm=options.yes, verbose=self.verbosity > 0,
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 682, in uninstall
uninstalled_pathset = UninstallPathSet.from_dist(dist)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/req/req_uninstall.py", line 360, in from_dist
for path in uninstallation_paths(dist):
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/req/req_uninstall.py", line 50, in unique
for item in fn(*args, **kw):
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/req/req_uninstall.py", line 67, in uninstallation_paths
r = csv.reader(FakeFile(dist.get_metadata_lines('RECORD')))
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1411, in get_metadata_lines
return yield_lines(self.get_metadata(name))
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1407, in get_metadata
value = self._get(self._fn(self.egg_info, name))
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1519, in _get
with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/RECORD'
The error message there (EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA') looks different enough that I'd suggest filing a separate issue for it. Mistune isn't using Flit, so hopefully it's not me breaking stuff this time. ;-)
takluyver
on 6 Oct 2018
I have seen a similar error to that, with the METADATA file missing from six; it was a problem affecting CI builds, but it seemed to go away when I upgraded pip.
takluyver
on 6 Oct 2018
Maybe this one should be filled with conda? After all, isn't it their installer's failure to roll back that cripples pip?
Attempting to roll back.
Rolling back transaction: done
LinkError: post-link script failed for package conda-forge::jupyter_latex_envs-1.4.4-py36_0
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.jupyter_latex_envs-post-link.sh
And a simple solution exists now, because we know exactly which package's folder to remove:
rm -rf /opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/
root@9fa47a5e4577:/opt/conda/lib/python3.6/site-packages# pip install scipy --no-cache-dir
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
root@9fa47a5e4577:
And if we break pip again (with the usual "conda install -y -c conda-forge jupyter jupyter_contrib_nbextensions jupyter_nbextensions_configurator"):
root@9fa47a5e4577:/opt/conda/lib/python3.6/site-packages# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/ipywidgets-7.4.2.dist-info/METADATA'
..we (i.e. typically root) can remove the entire folder with the newly broken package: ipywidgets, and the removal repairs pip once again:
root@9fa47a5e4577:/opt/conda/lib/python3.6/site-packages# rm -rf /opt/conda/lib/python3.6/site-packages/ipywidgets-7.4.2.dist-info/
root@9fa47a5e4577:/opt/conda/lib/python3.6/site-packages# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
root@9fa47a5e4577:/opt/conda/lib/python3.6/site-packages#
It's working for me now (essentially using workaround by @mirekphd).
I was having the same problem upgrading pip packages in a conda environment.
base conda environment
~/miniconda3
This is a base conda environment with just the minimal packages to install conda, plus conda's pip (10.0.1) and pip's pip (18.0 now 18.1).
ai conda environment
~/miniconda3/envs/ai
This has a ton of packages and just a handful of pip packages. It also had it's own pip (This may have been a problem) also of version 18.0 now removed.
Solution:
- update pip to version 18.1 in the base conda environment.
source activate base
pip install --upgrade pip
source deactivate
- remove pip from the environment you want to update.
source activate ai
pip uninstall pip
source deactivate
double check that it is gone.
ls ~/miniconda3/envs/ai/bin | grep pip
pip install still wasn't working yet. I realized that my base conda environment was not active while in my ai conda environment and I was using my system /usr/bin/pip after step 2.
- activate _both_ your base and working conda environments before using pip install
source activate base
source activate ai
echo $PATH
Now your $PATH should begin with:
~/miniconda3/envs/ai/bin:~/miniconda3/bin
And use pip 18.1 from your base environment, ~/miniconda3/bin.
which pip
pip --version
- I also noticed that I now need sudo permissions for this pip (unclear why since it's not touching my system directories).
sudo pip install --upgrade <package>
OK. pip install should now work under these conditions.
P.S. The set of pip packages I have are different, so the problem is likely independent of jupyter notebook extensions, etc.
working update
################################
##### UPDATE CONDA #####
################################
>>>>> conda update --name=base --yes conda
Solving environment: done
# All requested packages already installed.
###########################
##### base ####
###########################
>>>>> source activate base
>>>>> conda update --name=base --yes --all
Solving environment: ...working... done
# All requested packages already installed.
>>>>> pip install --upgrade pip
Requirement already up-to-date: pip in ./miniconda3/lib/python3.6/site-packages (18.1)
conda list --name=ai | grep '<pip>' | cut --fields=1 --delimiter=' ' | xargs (sudo pip install --upgrade)
[sudo] password for roddie:
Requirement already up-to-date: configparser in ./miniconda3/lib/python3.6/site-packages (3.5.0)
Requirement already up-to-date: emoji in ./miniconda3/lib/python3.6/site-packages (0.5.1)
Requirement already up-to-date: music21 in ./miniconda3/lib/python3.6/site-packages (5.3.0)
Requirement already up-to-date: mypy in ./miniconda3/lib/python3.6/site-packages (0.630)
Requirement already up-to-date: mypy-extensions in ./miniconda3/lib/python3.6/site-packages (0.4.1)
Requirement already up-to-date: pydub in ./miniconda3/lib/python3.6/site-packages (0.23.0)
Requirement already up-to-date: pyls-isort in ./miniconda3/lib/python3.6/site-packages (0.1.1)
Requirement already up-to-date: pyls-mypy in ./miniconda3/lib/python3.6/site-packages (0.1.3)
Requirement already up-to-date: solargraph-utils.py in ./miniconda3/lib/python3.6/site-packages (1.1.0)
Requirement already up-to-date: typed-ast in ./miniconda3/lib/python3.6/site-packages (1.1.0)
Requirement already satisfied, skipping upgrade: python-language-server in ./miniconda3/lib/python3.6/site-packages (from pyls-isort) (0.21.2)
Requirement already satisfied, skipping upgrade: isort in ./miniconda3/lib/python3.6/site-packages (from pyls-isort) (4.3.4)
Requirement already satisfied, skipping upgrade: jedi>=0.12 in ./miniconda3/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.13.1)
Requirement already satisfied, skipping upgrade: future>=0.14.0 in ./miniconda3/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.16.0)
Requirement already satisfied, skipping upgrade: pluggy in ./miniconda3/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.7.1)
Requirement already satisfied, skipping upgrade: python-jsonrpc-server in ./miniconda3/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.0.2)
Requirement already satisfied, skipping upgrade: parso>=0.3.0 in ./miniconda3/lib/python3.6/site-packages (from jedi>=0.12->python-language-server->pyls-isort) (0.3.1)
>>>>> source deactivate
###########################
##### ai ####
###########################
>>>>> source activate ai
>>>>> conda update --name=ai --yes --all
Solving environment: ...working... done
# All requested packages already installed.
>>>>> pip install --upgrade pip
Requirement already up-to-date: pip in /usr/lib/python3.7/site-packages (18.1)
conda list --name=ai | grep '<pip>' | cut --fields=1 --delimiter=' ' | xargs (sudo pip install --upgrade)
Collecting configparser
Using cached https://files.pythonhosted.org/packages/7c/69/c2ce7e91c89dc073eb1aa74c0621c3eefbffe8216b3f9af9d3885265c01c/configparser-3.5.0.tar.gz
Collecting emoji
Using cached https://files.pythonhosted.org/packages/9c/fb/7586e11ff9205c9be9d11d376fcb6990ec4bdfae0a35663fb1ada7e3c10f/emoji-0.5.1-py3-none-any.whl
Collecting music21
Downloading https://files.pythonhosted.org/packages/4a/db/317c21f4b5b970c3bfb5ff321e333059faf775621ae6433abcd4c68c69db/music21-5.3.0.tar.gz (18.0MB)
100% |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 18.0MB 577kB/s
Collecting mypy
Downloading https://files.pythonhosted.org/packages/32/8c/a668527fe2ea69f3a03fc4d895e12b6cb2eb1bb62d61cfe8ff4142395d3a/mypy-0.630-py3-none-any.whl (1.3MB)
100% |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 1.3MB 807kB/s
Collecting mypy-extensions
Downloading https://files.pythonhosted.org/packages/4d/72/8d54e2b296631b9b14961d583e56e90d9d7fba8a240d5ce7f1113cc5e887/mypy_extensions-0.4.1-py2.py3-none-any.whl
Collecting pydub
Downloading https://files.pythonhosted.org/packages/2f/73/bb9c093882d647437a9e6e87c7e6592d2df852f83ffac6f348b878979be0/pydub-0.23.0-py2.py3-none-any.whl
Collecting pyls-isort
Downloading https://files.pythonhosted.org/packages/8c/bd/7466d13e2bbe77fd1ae7cdd8e3b892518fc0a34c169d4933fb474c815526/pyls_isort-0.1.1-py3-none-any.whl
Collecting pyls-mypy
Downloading https://files.pythonhosted.org/packages/7b/88/41e44206fcdaef8aa6fe139a1d86e7d11062173968cac8b77d03d8e11df1/pyls-mypy-0.1.3.tar.gz
Requirement already up-to-date: solargraph-utils.py in /usr/lib/python3.7/site-packages (1.1.0)
Collecting typed-ast
Downloading https://files.pythonhosted.org/packages/e5/97/0398bd62ee2536097ed4576053f389e09bd5ac88b0750b3588bd9e2d341d/typed_ast-1.1.0-cp37-cp37m-manylinux1_x86_64.whl (723kB)
100% |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 727kB 864kB/s
Requirement already satisfied, skipping upgrade: python-language-server in /usr/lib/python3.7/site-packages (from pyls-isort) (0.21.2)
Collecting isort (from pyls-isort)
Downloading https://files.pythonhosted.org/packages/1f/2c/22eee714d7199ae0464beda6ad5fedec8fee6a2f7ffd1e8f1840928fe318/isort-4.3.4-py3-none-any.whl (45kB)
100% |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 51kB 787kB/s
Requirement already satisfied, skipping upgrade: future>=0.14.0 in /usr/lib/python3.7/site-packages (from python-language-server->pyls-isort) (0.16.0)
Requirement already satisfied, skipping upgrade: jedi>=0.12 in /usr/lib/python3.7/site-packages (from python-language-server->pyls-isort) (0.13.0)
Requirement already satisfied, skipping upgrade: python-jsonrpc-server in /usr/lib/python3.7/site-packages (from python-language-server->pyls-isort) (0.0.1)
Requirement already satisfied, skipping upgrade: pluggy in /usr/lib/python3.7/site-packages (from python-language-server->pyls-isort) (0.7.1)
Requirement already satisfied, skipping upgrade: parso>=0.3.0 in /usr/lib/python3.7/site-packages (from jedi>=0.12->python-language-server->pyls-isort) (0.3.1)
Installing collected packages: configparser, emoji, music21, typed-ast, mypy-extensions, mypy, pydub, isort, pyls-isort, pyls-mypy
Running setup.py install for configparser ... done
Running setup.py install for music21 ... done
Running setup.py install for pyls-mypy ... done
Successfully installed configparser-3.5.0 emoji-0.5.1 isort-4.3.4 music21-5.3.0 mypy-0.630 mypy-extensions-0.4.1 pydub-0.23.0 pyls-isort-0.1.1 pyls-mypy-0.1.3 typed-ast-1.1.0
>>>>> source deactivate
original failed update
################################
##### UPDATE CONDA #####
################################
>>>>> conda update --name=base --yes conda
Solving environment: done
# All requested packages already installed.
###########################
##### base ####
###########################
>>>>> source activate base
>>>>> conda update --name=base --yes --all
setuptools-40.4.3 | 555 KB | #################################################################################################################################################### | 100%
wheel-0.32.0 | 35 KB | #################################################################################################################################################### | 100%
Solving environment: ...working... done
## Package Plan ##
environment location: /home/roddie/miniconda3
The following packages will be downloaded:
package | build
---------------------------|-----------------
setuptools-40.4.3 | py36_0 555 KB
wheel-0.32.0 | py36_0 35 KB
------------------------------------------------------------
Total: 590 KB
The following packages will be UPDATED:
setuptools: 40.2.0-py36_0 --> 40.4.3-py36_0
wheel: 0.31.1-py36_0 --> 0.32.0-py36_0
Downloading and Extracting Packages
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
>>>>> pip install --upgrade pip
Collecting pip
Downloading https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl (1.3MB)
100% |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 1.3MB 891kB/s
Installing collected packages: pip
Found existing installation: pip 18.0
Uninstalling pip-18.0:
Successfully uninstalled pip-18.0
Successfully installed pip-18.1
>>>>> conda list --name=base | grep '<pip>' | cut --fields=1 --delimiter=' ' | xargs pip install --upgrade
Requirement already up-to-date: pip in ./miniconda3/lib/python3.6/site-packages (18.1)
>>>>> source deactivate
###########################
##### ai ####
###########################
>>>>> source activate ai
>>>>> conda update --name=ai --yes --all
networkx-2.2 | 2.0 MB | #################################################################################################################################################### | 100%
prompt_toolkit-2.0.5 | 461 KB | #################################################################################################################################################### | 100%
ipykernel-5.0.0 | 156 KB | #################################################################################################################################################### | 100%
pywavelets-1.0.1 | 4.4 MB | #################################################################################################################################################### | 100%
numpy-base-1.15.2 | 4.2 MB | #################################################################################################################################################### | 100%
ipython-7.0.1 | 1.0 MB | #################################################################################################################################################### | 100%
notebook-5.7.0 | 7.4 MB | #################################################################################################################################################### | 100%
numpy-1.15.2 | 48 KB | #################################################################################################################################################### | 100%
packaging-18.0 | 33 KB | #################################################################################################################################################### | 100%
testpath-0.4.1 | 91 KB | #################################################################################################################################################### | 100%
jupyterlab-0.34.12 | 10.0 MB | #################################################################################################################################################### | 100%
pillow-5.3.0 | 599 KB | #################################################################################################################################################### | 100%
click-7.0 | 118 KB | #################################################################################################################################################### | 100%
pickleshare-0.7.5 | 13 KB | #################################################################################################################################################### | 100%
Solving environment: ...working... done
## Package Plan ##
environment location: /home/roddie/miniconda3/envs/ai
The following packages will be downloaded:
package | build
---------------------------|-----------------
networkx-2.2 | py36_1 2.0 MB
prompt_toolkit-2.0.5 | py36_0 461 KB
ipykernel-5.0.0 | py36h39e3cac_0 156 KB
pywavelets-1.0.1 | py36hdd07704_0 4.4 MB
numpy-base-1.15.2 | py36h81de0dd_1 4.2 MB
ipython-7.0.1 | py36h39e3cac_0 1.0 MB
notebook-5.7.0 | py36_0 7.4 MB
numpy-1.15.2 | py36h1d66e8a_1 48 KB
packaging-18.0 | py36_0 33 KB
testpath-0.4.1 | py36_0 91 KB
jupyterlab-0.34.12 | py36_0 10.0 MB
pillow-5.3.0 | py36h34e0f95_0 599 KB
click-7.0 | py36_0 118 KB
pickleshare-0.7.5 | py36_0 13 KB
------------------------------------------------------------
Total: 30.5 MB
The following packages will be UPDATED:
click: 6.7-py36_0 --> 7.0-py36_0
ipykernel: 4.10.0-py36_0 --> 5.0.0-py36h39e3cac_0
ipython: 6.5.0-py36_0 --> 7.0.1-py36h39e3cac_0
jupyterlab: 0.34.9-py36_0 --> 0.34.12-py36_0
networkx: 2.1-py36_0 --> 2.2-py36_1
notebook: 5.6.0-py36_0 --> 5.7.0-py36_0
numpy: 1.15.2-py36h1d66e8a_0 --> 1.15.2-py36h1d66e8a_1
numpy-base: 1.15.2-py36h81de0dd_0 --> 1.15.2-py36h81de0dd_1
packaging: 17.1-py36_0 --> 18.0-py36_0
pickleshare: 0.7.4-py36_0 --> 0.7.5-py36_0
pillow: 5.2.0-py36heded4f4_0 --> 5.3.0-py36h34e0f95_0
prompt_toolkit: 1.0.15-py36_0 --> 2.0.5-py36_0
pywavelets: 1.0.0-py36hdd07704_0 --> 1.0.1-py36hdd07704_0
setuptools: 40.2.0-py36_0 --> 40.4.3-py36_0
testpath: 0.3.1-py36_0 --> 0.4.1-py36_0
wheel: 0.31.1-py36_0 --> 0.32.0-py36_0
Downloading and Extracting Packages
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
>>>>> pip install --upgrade pip
Collecting pip
Using cached https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl
Exception:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
return self.__dep_map
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
raise exc
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
loc, tokens = self._parse( instring, 0 )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
super(Requirement, self).__init__(requirement_string)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
status = self.run(options, args)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
self._warn_about_conflicts(to_install)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
package_set, _dep_info = check_install_conflicts(to_install)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
package_set = create_package_set_from_installed()
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
package_set[name] = PackageDetails(dist.version, dist.requires())
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
dm = self._dep_map
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
self.__dep_map = self._compute_dependencies()
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
reqs.extend(parse_requirements(req))
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
yield Requirement(line)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"
You are using pip version 18.0, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
>>>>> conda list --name=ai | grep '<pip>' | cut --fields=1 --delimiter=' ' | xargs pip install --upgrade
Requirement already up-to-date: configparser in ./miniconda3/envs/ai/lib/python3.6/site-packages (3.5.0)
Requirement already up-to-date: emoji in ./miniconda3/envs/ai/lib/python3.6/site-packages (0.5.1)
Requirement already up-to-date: music21 in ./miniconda3/envs/ai/lib/python3.6/site-packages (5.3.0)
Requirement already up-to-date: mypy in ./miniconda3/envs/ai/lib/python3.6/site-packages (0.630)
Requirement already up-to-date: mypy-extensions in ./miniconda3/envs/ai/lib/python3.6/site-packages (0.4.1)
Collecting pip
Using cached https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl
Requirement already up-to-date: pydub in ./miniconda3/envs/ai/lib/python3.6/site-packages (0.23.0)
Requirement already up-to-date: pyls-isort in ./miniconda3/envs/ai/lib/python3.6/site-packages (0.1.1)
Requirement already up-to-date: pyls-mypy in ./miniconda3/envs/ai/lib/python3.6/site-packages (0.1.3)
Requirement already up-to-date: solargraph-utils.py in ./miniconda3/envs/ai/lib/python3.6/site-packages (1.1.0)
Requirement already up-to-date: typed-ast in ./miniconda3/envs/ai/lib/python3.6/site-packages (1.1.0)
Requirement already satisfied, skipping upgrade: isort in ./miniconda3/envs/ai/lib/python3.6/site-packages (from pyls-isort) (4.3.4)
Requirement already satisfied, skipping upgrade: python-language-server in ./miniconda3/envs/ai/lib/python3.6/site-packages (from pyls-isort) (0.21.2)
Requirement already satisfied, skipping upgrade: python-jsonrpc-server in ./miniconda3/envs/ai/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.0.2)
Requirement already satisfied, skipping upgrade: jedi>=0.12 in ./miniconda3/envs/ai/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.12.1)
Requirement already satisfied, skipping upgrade: future>=0.14.0 in ./miniconda3/envs/ai/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.16.0)
Requirement already satisfied, skipping upgrade: pluggy in ./miniconda3/envs/ai/lib/python3.6/site-packages (from python-language-server->pyls-isort) (0.7.1)
Requirement already satisfied, skipping upgrade: parso>=0.3.0 in ./miniconda3/envs/ai/lib/python3.6/site-packages (from jedi>=0.12->python-language-server->pyls-isort) (0.3.1)
Exception:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
return self.__dep_map
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
raise exc
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
loc, tokens = self._parse( instring, 0 )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
super(Requirement, self).__init__(requirement_string)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
status = self.run(options, args)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
self._warn_about_conflicts(to_install)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
package_set, _dep_info = check_install_conflicts(to_install)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
package_set = create_package_set_from_installed()
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
package_set[name] = PackageDetails(dist.version, dist.requires())
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
dm = self._dep_map
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
self.__dep_map = self._compute_dependencies()
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
reqs.extend(parse_requirements(req))
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
yield Requirement(line)
File "/home/roddie/miniconda3/envs/ai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"
You are using pip version 18.0, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
>>>>> source deactivate
rodrev
on 6 Oct 2018
Can you see if this issue is still occurring with the latest packages in the conda ecosystem?
The testpath 0.4.1 conda packages in both the defaults and conda-forge channels contained a METADATA file with an invalid Required-Dist line that pkg_resources was not able to parse. Both channels now have corrected packages.
jjhelmus
on 7 Oct 2018
That problem with pkg_resources seems to be gone indeed. But that does not stop Jupyter Extensions (conda installed) from breaking pip in my Kaggle-like environment, like this...
[root@mx01 ml-gpu-py36-cuda90]# docker run -it -u 0 -p 8888:8888 mirekphd/ml-gpu-py36-cuda8:latest bash
root@256c343bb1ef:~# conda list testpath
# packages in environment at /opt/conda:
#
# Name Version Build Channel
testpath 0.4.2 <pip>
root@256c343bb1ef:~# conda install -y -c conda-forge jupyter jupyter_contrib_nbextensions jupyter_nbextensions_configurator
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.10
latest version: 4.5.11
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: /opt/conda
added / updated specs:
- jupyter
- jupyter_contrib_nbextensions
- jupyter_nbextensions_configurator
The following packages will be downloaded:
package | build
---------------------------|-----------------
prometheus_client-0.4.0 | py_0 32 KB conda-forge
testpath-0.4.2 | py36_1000 91 KB conda-forge
jupyter_contrib_core-0.3.3 | py_2 15 KB conda-forge
jsonschema-2.6.0 | py36_1002 62 KB conda-forge
jupyter_contrib_nbextensions-0.5.0| py36_0 19.4 MB conda-forge
ipywidgets-7.4.2 | py_0 92 KB conda-forge
libsodium-1.0.16 | h470a237_1 536 KB conda-forge
nbformat-4.4.0 | py_1 91 KB conda-forge
entrypoints-0.2.3 | py36_1002 9 KB conda-forge
jupyter_console-5.2.0 | py36_1 34 KB conda-forge
pandoc-2.3.1 | 0 20.7 MB conda-forge
jupyter-1.0.0 | py_1 6 KB conda-forge
html5lib-1.0.1 | py_0 89 KB conda-forge
libxslt-1.1.32 | h88dbc4e_2 511 KB conda-forge
jupyter_core-4.4.0 | py_0 44 KB conda-forge
pandocfilters-1.4.2 | py_1 9 KB conda-forge
mistune-0.8.3 | py36h470a237_2 873 KB conda-forge
bleach-3.0.0 | py_0 108 KB conda-forge
notebook-5.7.0 | py36_1000 7.3 MB conda-forge
nbconvert-5.3.1 | py_1 320 KB conda-forge
pyzmq-17.1.2 | py36hae99301_0 1.0 MB conda-forge
terminado-0.8.1 | py36_1001 23 KB conda-forge
jinja2-2.10 | py_1 89 KB conda-forge
ipykernel-5.0.0 | pyh24bf2e0_1 76 KB conda-forge
send2trash-1.5.0 | py_0 12 KB conda-forge
widgetsnbextension-3.4.2 | py36_1000 1.7 MB conda-forge
jupyter_latex_envs-1.4.4 | py36_0 753 KB conda-forge
jupyter_highlight_selected_word-0.2.0| py36_1000 13 KB conda-forge
lxml-4.2.5 | py36hc9114bc_0 6.0 MB conda-forge
qtconsole-4.4.1 | py36_1 156 KB conda-forge
jupyter_nbextensions_configurator-0.4.0| py36_0 470 KB conda-forge
jupyter_client-5.2.3 | py_1 59 KB conda-forge
zeromq-4.2.5 | hfc679d8_6 939 KB conda-forge
webencodings-0.5.1 | py_1 12 KB conda-forge
------------------------------------------------------------
Total: 61.6 MB
The following NEW packages will be INSTALLED:
bleach: 3.0.0-py_0 conda-forge
entrypoints: 0.2.3-py36_1002 conda-forge
html5lib: 1.0.1-py_0 conda-forge
ipykernel: 5.0.0-pyh24bf2e0_1 conda-forge
ipywidgets: 7.4.2-py_0 conda-forge
jinja2: 2.10-py_1 conda-forge
jsonschema: 2.6.0-py36_1002 conda-forge
jupyter: 1.0.0-py_1 conda-forge
jupyter_client: 5.2.3-py_1 conda-forge
jupyter_console: 5.2.0-py36_1 conda-forge
jupyter_contrib_core: 0.3.3-py_2 conda-forge
jupyter_contrib_nbextensions: 0.5.0-py36_0 conda-forge
jupyter_core: 4.4.0-py_0 conda-forge
jupyter_highlight_selected_word: 0.2.0-py36_1000 conda-forge
jupyter_latex_envs: 1.4.4-py36_0 conda-forge
jupyter_nbextensions_configurator: 0.4.0-py36_0 conda-forge
libsodium: 1.0.16-h470a237_1 conda-forge
libxslt: 1.1.32-h88dbc4e_2 conda-forge
lxml: 4.2.5-py36hc9114bc_0 conda-forge
mistune: 0.8.3-py36h470a237_2 conda-forge
nbconvert: 5.3.1-py_1 conda-forge
nbformat: 4.4.0-py_1 conda-forge
notebook: 5.7.0-py36_1000 conda-forge
pandoc: 2.3.1-0 conda-forge
pandocfilters: 1.4.2-py_1 conda-forge
prometheus_client: 0.4.0-py_0 conda-forge
pyzmq: 17.1.2-py36hae99301_0 conda-forge
qtconsole: 4.4.1-py36_1 conda-forge
send2trash: 1.5.0-py_0 conda-forge
terminado: 0.8.1-py36_1001 conda-forge
testpath: 0.4.2-py36_1000 conda-forge
webencodings: 0.5.1-py_1 conda-forge
widgetsnbextension: 3.4.2-py36_1000 conda-forge
zeromq: 4.2.5-hfc679d8_6 conda-forge
Downloading and Extracting Packages
prometheus_client-0. | 32 KB | ########################################################################################################################################################### | 100%
testpath-0.4.2 | 91 KB | ########################################################################################################################################################### | 100%
jupyter_contrib_core | 15 KB | ########################################################################################################################################################### | 100%
jsonschema-2.6.0 | 62 KB | ########################################################################################################################################################### | 100%
jupyter_contrib_nbex | 19.4 MB | ########################################################################################################################################################### | 100%
ipywidgets-7.4.2 | 92 KB | ########################################################################################################################################################### | 100%
libsodium-1.0.16 | 536 KB | ########################################################################################################################################################### | 100%
nbformat-4.4.0 | 91 KB | ########################################################################################################################################################### | 100%
entrypoints-0.2.3 | 9 KB | ########################################################################################################################################################### | 100%
jupyter_console-5.2. | 34 KB | ########################################################################################################################################################### | 100%
pandoc-2.3.1 | 20.7 MB | ########################################################################################################################################################### | 100%
jupyter-1.0.0 | 6 KB | ########################################################################################################################################################### | 100%
html5lib-1.0.1 | 89 KB | ########################################################################################################################################################### | 100%
libxslt-1.1.32 | 511 KB | ########################################################################################################################################################### | 100%
jupyter_core-4.4.0 | 44 KB | ########################################################################################################################################################### | 100%
pandocfilters-1.4.2 | 9 KB | ########################################################################################################################################################### | 100%
mistune-0.8.3 | 873 KB | ########################################################################################################################################################### | 100%
bleach-3.0.0 | 108 KB | ########################################################################################################################################################### | 100%
notebook-5.7.0 | 7.3 MB | ########################################################################################################################################################### | 100%
nbconvert-5.3.1 | 320 KB | ########################################################################################################################################################### | 100%
pyzmq-17.1.2 | 1.0 MB | ########################################################################################################################################################### | 100%
terminado-0.8.1 | 23 KB | ########################################################################################################################################################### | 100%
jinja2-2.10 | 89 KB | ########################################################################################################################################################### | 100%
ipykernel-5.0.0 | 76 KB | ########################################################################################################################################################### | 100%
send2trash-1.5.0 | 12 KB | ########################################################################################################################################################### | 100%
widgetsnbextension-3 | 1.7 MB | ########################################################################################################################################################### | 100%
jupyter_latex_envs-1 | 753 KB | ########################################################################################################################################################### | 100%
jupyter_highlight_se | 13 KB | ########################################################################################################################################################### | 100%
lxml-4.2.5 | 6.0 MB | ########################################################################################################################################################### | 100%
qtconsole-4.4.1 | 156 KB | ########################################################################################################################################################### | 100%
jupyter_nbextensions | 470 KB | ########################################################################################################################################################### | 100%
jupyter_client-5.2.3 | 59 KB | ########################################################################################################################################################### | 100%
zeromq-4.2.5 | 939 KB | ########################################################################################################################################################### | 100%
webencodings-0.5.1 | 12 KB | ########################################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: \ Enabling notebook extension jupyter-js-widgets/extension...
- Validating: OK
- + /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''jupyter_highlight_selected_word'\'', sys_prefix=True, logger=logging.getLogger())'
- + /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''latex_envs'\'', sys_prefix=True, logger=logging.getLogger())'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
m, nbexts = _get_nbextension_metadata(module)
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
m = import_item(module)
File "/opt/conda/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
return __import__(parts[0])
File "/opt/conda/lib/python3.6/site-packages/latex_envs/__init__.py", line 3, in <module>
from . import latex_envs
File "/opt/conda/lib/python3.6/site-packages/latex_envs/latex_envs.py", line 19, in <module>
from IPython.display import HTML, display,JSON
File "/opt/conda/lib/python3.6/site-packages/IPython/__init__.py", line 55, in <module>
from .terminal.embed import embed
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/embed.py", line 16, in <module>
from IPython.terminal.interactiveshell import TerminalInteractiveShell
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 18, in <module>
from prompt_toolkit.document import Document
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
from .interface import CommandLineInterface
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/interface.py", line 19, in <module>
from .application import Application, AbortAction
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/__init__.py", line 2, in <module>
from .application import Application
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/application.py", line 3, in <module>
from prompt_toolkit.buffer import Buffer
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 9, in <module>
from .completion import Completer, Completion, CompleteEvent
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/__init__.py", line 2, in <module>
from .base import Completion, Completer, ThreadedCompleter, DummyCompleter, DynamicCompleter, CompleteEvent, merge_completers, get_common_complete_suffix
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/base.py", line 4, in <module>
from prompt_toolkit.eventloop import generator_to_async_generator, AsyncGeneratorItem
ImportError: cannot import name 'generator_to_async_generator'
failed
ERROR conda.core.link:_execute(502): An error occurred while installing package 'conda-forge::jupyter_latex_envs-1.4.4-py36_0'.
LinkError: post-link script failed for package conda-forge::jupyter_latex_envs-1.4.4-py36_0
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.jupyter_latex_envs-post-link.sh
==> script messages <==
+ /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''latex_envs'\'', sys_prefix=True, logger=logging.getLogger())'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
m, nbexts = _get_nbextension_metadata(module)
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
m = import_item(module)
File "/opt/conda/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
return __import__(parts[0])
File "/opt/conda/lib/python3.6/site-packages/latex_envs/__init__.py", line 3, in <module>
from . import latex_envs
File "/opt/conda/lib/python3.6/site-packages/latex_envs/latex_envs.py", line 19, in <module>
from IPython.display import HTML, display,JSON
File "/opt/conda/lib/python3.6/site-packages/IPython/__init__.py", line 55, in <module>
from .terminal.embed import embed
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/embed.py", line 16, in <module>
from IPython.terminal.interactiveshell import TerminalInteractiveShell
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 18, in <module>
from prompt_toolkit.document import Document
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
from .interface import CommandLineInterface
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/interface.py", line 19, in <module>
from .application import Application, AbortAction
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/__init__.py", line 2, in <module>
from .application import Application
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/application.py", line 3, in <module>
from prompt_toolkit.buffer import Buffer
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 9, in <module>
from .completion import Completer, Completion, CompleteEvent
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/__init__.py", line 2, in <module>
from .base import Completion, Completer, ThreadedCompleter, DummyCompleter, DynamicCompleter, CompleteEvent, merge_completers, get_common_complete_suffix
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/base.py", line 4, in <module>
from prompt_toolkit.eventloop import generator_to_async_generator, AsyncGeneratorItem
ImportError: cannot import name 'generator_to_async_generator'
Attempting to roll back.
Rolling back transaction: done
LinkError: post-link script failed for package conda-forge::jupyter_latex_envs-1.4.4-py36_0
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.jupyter_latex_envs-post-link.sh
==> script messages <==
+ /opt/conda/bin/python -c 'import logging; from jupyter_contrib_core.notebook_compat.nbextensions import install_nbextension_python; install_nbextension_python('\''latex_envs'\'', sys_prefix=True, logger=logging.getLogger())'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
m, nbexts = _get_nbextension_metadata(module)
File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
m = import_item(module)
File "/opt/conda/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
return __import__(parts[0])
File "/opt/conda/lib/python3.6/site-packages/latex_envs/__init__.py", line 3, in <module>
from . import latex_envs
File "/opt/conda/lib/python3.6/site-packages/latex_envs/latex_envs.py", line 19, in <module>
from IPython.display import HTML, display,JSON
File "/opt/conda/lib/python3.6/site-packages/IPython/__init__.py", line 55, in <module>
from .terminal.embed import embed
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/embed.py", line 16, in <module>
from IPython.terminal.interactiveshell import TerminalInteractiveShell
File "/opt/conda/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 18, in <module>
from prompt_toolkit.document import Document
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
from .interface import CommandLineInterface
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/interface.py", line 19, in <module>
from .application import Application, AbortAction
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/__init__.py", line 2, in <module>
from .application import Application
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/application/application.py", line 3, in <module>
from prompt_toolkit.buffer import Buffer
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 9, in <module>
from .completion import Completer, Completion, CompleteEvent
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/__init__.py", line 2, in <module>
from .base import Completion, Completer, ThreadedCompleter, DummyCompleter, DynamicCompleter, CompleteEvent, merge_completers, get_common_complete_suffix
File "/opt/conda/lib/python3.6/site-packages/prompt_toolkit/completion/base.py", line 4, in <module>
from prompt_toolkit.eventloop import generator_to_async_generator, AsyncGeneratorItem
ImportError: cannot import name 'generator_to_async_generator'
root@256c343bb1ef:~# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@256c343bb1ef:~# pip install scipy --no-cache-dir
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@256c343bb1ef:~#
But a workaround is to manually remove those folders reported in sequence by pip in its [Errno 2] error message (there are two such libs to purge: mistune 0.8.3 and ipywidgets 7.4.2, both from the conda-forge channel and installed together with jupyter_contrib_nbextensions and jupyter_nbextensions_configurator). Presumably pip should be immunized from being broken by such malformed packages as well?
root@256c343bb1ef:~# pip install scipy --no-cache-dir
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/METADATA'
root@256c343bb1ef:~# rm -rf /opt/conda/lib/python3.6/site-packages/mistune-0.8.3.dist-info/
root@256c343bb1ef:~# pip install scipy --no-cache-dir
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.6/site-packages/ipywidgets-7.4.2.dist-info/METADATA'
root@256c343bb1ef:~# rm -rf /opt/conda/lib/python3.6/site-packages/ipywidgets-7.4.2.dist-info/
root@256c343bb1ef:~# pip install scipy --no-cache-dir
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
root@256c343bb1ef:~#
Re: reporting it on conda - it's certainly an issue that the post-link script fails. But I'd say the bigger issue is that pip should handle the situation a bit more gracefully.
takluyver
on 7 Oct 2018
Maybe moving these folders to a temporary "quarantine" folder would be acceptable?
mirekphd
on 7 Oct 2018
Is this related to https://github.com/jupyter/notebook/issues/2605 in any way?
astrojuanlu
on 8 Oct 2018
Is this related to jupyter/notebook#2605 in any way?
I think so, only the solution offered was not general enough to work in more complex cases like this one. I cannot reproduce their case, but can quickly prove their solution does not work here. Docker has no equals when it comes to reproducing bugs...
[root@mx01 ml-gpu-py36-cuda90]# docker pull mirekphd/tst-ml-gpu-py36-cuda8-failing:001766c55ca8
001766c55ca8: Pulling from mirekphd/tst-ml-gpu-py36-cuda8-failing
3b37166ec614: Already exists
504facff238f: Already exists
ebbcacd28e10: Already exists
c7fb3351ecad: Already exists
2e3debadcbf7: Already exists
9d41bf765053: Already exists
b032b67ae8c0: Already exists
f0ca0f007bc1: Already exists
a3fbaa8d0330: Already exists
8d1ee91d6b9b: Already exists
07f5a0daf889: Already exists
844ed628e006: Pull complete
fedac359abb2: Pull complete
d3bbb12dcd7a: Pull complete
d51865d3bf17: Pull complete
f8268f466392: Pull complete
81458ef137c3: Pull complete
35b5befa520c: Pull complete
76e4e385dfc1: Pull complete
389f70da86d2: Pull complete
a2f9ec6d50f0: Pull complete
bdfbace07bda: Pull complete
f8aa731f37d7: Pull complete
6de061258af5: Pull complete
c936b1b29cae: Pull complete
a14b932d1362: Pull complete
2e870e219b9a: Pull complete
b421266e5bd7: Pull complete
e3f899f7462b: Pull complete
dead032ae4e1: Pull complete
38b5148bb95d: Pull complete
049707f48c29: Pull complete
61ba6fbe1f6a: Pull complete
bbec11a8049d: Pull complete
1b9130fdbc6e: Pull complete
d2cccfae9ad6: Pull complete
542f108dedcf: Pull complete
fbe2b7aa5d49: Pull complete
23c3b3f0afdb: Pull complete
6ca608281fdf: Pull complete
771ebe0dd4be: Pull complete
1f26185070ab: Pull complete
Digest: sha256:0bd083fca178923714050ed2578592e25d75a6b9e071904f987a47477ad9aa4e
Status: Downloaded newer image for mirekphd/tst-ml-gpu-py36-cuda8-failing:001766c55ca8
[root@mx01 ml-gpu-py36-cuda90]# docker run -it --rm --name test -u 0 mirekphd/tst-ml-gpu-py36-cuda8-failing:001766c55ca8 bash
pip install scipy
root@2f62b221038e:/# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Requirement already satisfied: numpy>=1.8.2 in /opt/conda/lib/python3.6/site-packages (from scipy) (1.15.2)
Exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
return self.__dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
raise exc
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
loc, tokens = self._parse( instring, 0 )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
super(Requirement, self).__init__(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
status = self.run(options, args)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
self._warn_about_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
package_set, _dep_info = check_install_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
package_set = create_package_set_from_installed()
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
package_set[name] = PackageDetails(dist.version, dist.requires())
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
dm = self._dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
self.__dep_map = self._compute_dependencies()
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
reqs.extend(parse_requirements(req))
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
yield Requirement(line)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"
root@2f62b221038e:/# sudo pip3 install --upgrade pip
sudo: pip3: command not found
root@2f62b221038e:/# sudo pip install --upgrade pip
sudo: pip: command not found
root@2f62b221038e:/# pip install --upgrade pip
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
Downloading https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl (1.3MB)
100% |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 1.3MB 12.9MB/s
Collecting first
Downloading https://files.pythonhosted.org/packages/71/12/1b24a85f543658804027de982f7e5cb80543ea33ec396c887c099c75274c/first-2.0.1-py2.py3-none-any.whl
Exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
return self.__dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
raise exc
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
loc, tokens = self._parse( instring, 0 )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
super(Requirement, self).__init__(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
status = self.run(options, args)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
self._warn_about_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
package_set, _dep_info = check_install_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
package_set = create_package_set_from_installed()
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
package_set[name] = PackageDetails(dist.version, dist.requires())
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
dm = self._dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
self.__dep_map = self._compute_dependencies()
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
reqs.extend(parse_requirements(req))
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
yield Requirement(line)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"
root@2f62b221038e:/# pip install scipy
The directory '/home/jovyan/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jovyan/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (1.1.0)
Requirement already satisfied: numpy>=1.8.2 in /opt/conda/lib/python3.6/site-packages (from scipy) (1.15.2)
Exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
return self.__dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
raise exc
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
loc, tokens = self._parse( instring, 0 )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
super(Requirement, self).__init__(requirement_string)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
status = self.run(options, args)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
self._warn_about_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
package_set, _dep_info = check_install_conflicts(to_install)
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
package_set = create_package_set_from_installed()
File "/opt/conda/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
package_set[name] = PackageDetails(dist.version, dist.requires())
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
dm = self._dep_map
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
self.__dep_map = self._compute_dependencies()
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
reqs.extend(parse_requirements(req))
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
yield Requirement(line)
File "/opt/conda/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"
root@2f62b221038e:/#
Apart from the situation with conda, we were hit by this issue when trying to use a PEP 508 URL requirement in an extras_require section (see https://github.com/pypa/pip/pull/5788#issuecomment-428146145). To debug this issue, we had to insert a pdb.set_trace() in pip/_vendor/packaging/requirements.py, so I totally agree with @takluyver here:
But I'd say the bigger issue is that pip should handle the situation a bit more gracefully.
Perhaps it should print the full problematic line and hide the useless _DistInfoDistribution__dep_map error?
astrojuanlu
on 9 Oct 2018
👍2
This was fixed in #5842.
pradyunsg
on 20 Jul 2019
Was this page helpful?
0 / 5 - 0 ratings
Related issues
JordanReiter
路
3Comments
yizhang-zen
路
3Comments
lululukas
路
3Comments
nathan130200
路
3Comments
GregBorrelly
路
3Comments
Most helpful comment
Apart from the situation with conda, we were hit by this issue when trying to use a PEP 508 URL requirement in an
extras_requiresection (see https://github.com/pypa/pip/pull/5788#issuecomment-428146145). To debug this issue, we had to insert apdb.set_trace()inpip/_vendor/packaging/requirements.py, so I totally agree with @takluyver here:Perhaps it should print the full problematic line and hide the useless
_DistInfoDistribution__dep_maperror?