poetry tries to install a version which is not present for the host OS

[w01fS]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Catalina 10.15.1
• Poetry version: 1.0.5
• Link of pyproject.toml

Issue

For a project, needed to install umap-learn which in turn needs _tbb_ package from pypi.

On running poetry add umap-learn, the installation failed with

[EnvCommandError] Command ['/Users/username/Downloads/Projects/env/bin/pip', 'install', '--no-deps', '-U', 'tbb==2020.0.133'] errored with the following return code 1, and output: ERROR: Could not find a version that satisfies the requirement tbb==2020.0.133 (from versions: 2018.0.4.1, 2019.0) ERROR: No matching distribution found for tbb==2020.0.133

It seems like the above package is not available for my host OS as when using pip install umap-learn, the version installed is 2019.0.

If I try to do _poetry add_ command again, it tries to still update the package and fails.

the verbose mode gives
`PyPI: 1 packages found for umap-learn >=0.4.1,<0.5.0
1: fact: umap-learn (0.4.1) depends on numpy (>=1.15)
1: fact: umap-learn (0.4.1) depends on scikit-learn (>=0.20)
1: fact: umap-learn (0.4.1) depends on scipy (>=1.0)
1: fact: umap-learn (0.4.1) depends on numba (>=0.46,<0.47 || >0.47)
1: fact: umap-learn (0.4.1) depends on tbb (>=2019.0)
1: selecting umap-learn (0.4.1)
1: derived: tbb (>=2019.0)
1: derived: numba (>=0.46,<0.47 || >0.47)
1: derived: scipy (>=1.0)
1: derived: scikit-learn (>=0.20)
1: derived: numpy (>=1.15)
PyPI: 2 packages found for tbb >=2019.0
1: selecting cython (0.29.15)
1: fact: celery (4.4.0) depends on pytz (>0.0-dev)
1: fact: celery (4.4.0) depends on billiard (>=3.6.1,<4.0)
1: fact: celery (4.4.0) depends on kombu (>=4.6.7,<4.7)
1: fact: celery (4.4.0) depends on vine (1.3.0)
1: selecting celery (4.4.0)
1: derived: vine (1.3.0)
1: derived: kombu (>=4.6.7,<4.7)
1: derived: billiard (>=3.6.1,<4.0)
1: derived: pytz (>0.0-dev)
1: fact: azure-storage (0.36.0) depends on azure-common (>=1.1.5)
1: fact: azure-storage (0.36.0) depends on azure-nspkg (>=2.0.0)
1: fact: azure-storage (0.36.0) depends on cryptography ()
1: fact: azure-storage (0.36.0) depends on python-dateutil ()
1: fact: azure-storage (0.36.0) depends on requests ()
1: selecting azure-storage (0.36.0)
1: derived: requests ()
1: derived: python-dateutil ()
1: derived: cryptography ()
1: derived: azure-nspkg (>=2.0.0)
1: derived: azure-common (>=1.1.5)
1: fact: sklearn (0.0) depends on scikit-learn ()
1: selecting sklearn (0.0)
1: selecting pymongo (3.8.0)
1: selecting configparser (3.7.0)
1: fact: tensorflow-hub (0.7.0) depends on numpy (>=1.12.0)
1: fact: tensorflow-hub (0.7.0) depends on six (>=1.10.0)
1: fact: tensorflow-hub (0.7.0) depends on protobuf (>=3.4.0)
1: selecting tensorflow-hub (0.7.0)
1: derived: protobuf (>=3.4.0)
1: derived: six (>=1.10.0)
1: fact: flask (1.1.1) depends on Werkzeug (>=0.15)
1: fact: flask (1.1.1) depends on Jinja2 (>=2.10.1)
1: fact: flask (1.1.1) depends on itsdangerous (>=0.24)
1: fact: flask (1.1.1) depends on click (>=5.1)
1: selecting flask (1.1.1)
1: derived: click (>=5.1)
1: derived: itsdangerous (>=0.24)
1: derived: Jinja2 (>=2.10.1)
1: derived: Werkzeug (>=0.15)
1: fact: numba (0.48.0) depends on llvmlite (>=0.31.0dev0,<0.32.0)
1: fact: numba (0.48.0) depends on numpy (>=1.15)
1: selecting numba (0.48.0)
1: derived: llvmlite (>=0.31.0dev0,<0.32.0)
1: fact: scipy (1.4.1) depends on numpy (>=1.13.3)
1: selecting scipy (1.4.1)
1: fact: scikit-learn (0.22.1) depends on numpy (>=1.11.0)
1: fact: scikit-learn (0.22.1) depends on scipy (>=0.17.0)
1: fact: scikit-learn (0.22.1) depends on joblib (>=0.11)
1: selecting scikit-learn (0.22.1)
1: derived: joblib (>=0.11)
1: selecting numpy (1.18.1)
1: selecting vine (1.3.0)
1: fact: kombu (4.6.7) depends on amqp (>=2.5.2,<2.6)
1: fact: kombu (4.6.7) depends on importlib-metadata (>=0.18)
1: selecting kombu (4.6.7)
1: derived: importlib-metadata (>=0.18)
1: derived: amqp (>=2.5.2,<2.6)
1: selecting billiard (3.6.2.0)
1: selecting pytz (2019.3)
1: fact: requests (2.22.0) depends on chardet (>=3.0.2,<3.1.0)
1: fact: requests (2.22.0) depends on idna (>=2.5,<2.9)
1: fact: requests (2.22.0) depends on urllib3 (>=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26)
1: fact: requests (2.22.0) depends on certifi (>=2017.4.17)
1: selecting requests (2.22.0)
1: derived: certifi (>=2017.4.17)
1: derived: urllib3 (>=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26)
1: derived: idna (>=2.5,<2.9)
1: derived: chardet (>=3.0.2,<3.1.0)
1: fact: python-dateutil (2.8.1) depends on six (>=1.5)
1: selecting python-dateutil (2.8.1)
1: fact: cryptography (2.8) depends on six (>=1.4.1)
1: fact: cryptography (2.8) depends on cffi (>=1.8,<1.11.3 || >1.11.3)
1: selecting cryptography (2.8)
1: derived: cffi (>=1.8,<1.11.3 || >1.11.3)
1: selecting azure-nspkg (3.0.2)
1: selecting azure-common (1.1.24)
1: fact: protobuf (3.11.3) depends on six (>=1.9)
1: selecting protobuf (3.11.3)
1: selecting six (1.14.0)
1: selecting click (7.0)
1: selecting itsdangerous (1.1.0)
1: fact: jinja2 (2.11.1) depends on MarkupSafe (>=0.23)
1: selecting jinja2 (2.11.1)
1: derived: MarkupSafe (>=0.23)
1: selecting werkzeug (1.0.0)
1: selecting llvmlite (0.31.0)
1: selecting joblib (0.14.1)
1: fact: importlib-metadata (1.5.0) depends on zipp (>=0.5)
1: selecting importlib-metadata (1.5.0)
1: derived: zipp (>=0.5)
1: fact: amqp (2.5.2) depends on vine (>=1.1.3,<5.0.0a1)
1: selecting amqp (2.5.2)
1: selecting certifi (2019.11.28)
1: selecting urllib3 (1.25.8)
1: selecting idna (2.8)
1: selecting chardet (3.0.4)
1: fact: cffi (1.14.0) depends on pycparser ()
1: selecting cffi (1.14.0)
1: derived: pycparser (*)
1: selecting markupsafe (1.1.1)
1: selecting zipp (2.2.0)
1: selecting pycparser (2.19)
1: selecting tbb (2020.0.133)
1: Version solving took 0.279 seconds.
1: Tried 1 solutions.

Writing lock file

Package operations: 2 installs, 0 updates, 0 removals

• Installing tbb (2020.0.133)

[EnvCommandError]
Command ['/Users/username/Downloads/Projects/env/bin/pip', 'install', '--no-deps', 'tbb==2020.0.133'] errored with the following return code 1, and output:
ERROR: Could not find a version that satisfies the requirement tbb==2020.0.133 (from versions: 2018.0.4.1, 2019.0)
ERROR: No matching distribution found for tbb==2020.0.133
`

Any help would be appreciated.

[niclas-thomas]

Same issue for me. Catalina 10.15.4, Poetry 1.0.5, Python 3.7

[roniemartinez]

Same issue here:

% sw_vers
ProductName:    Mac OS X
ProductVersion: 10.15.4
BuildVersion:   19E287
% python -V
Python 3.7.7
% poetry -V
Poetry version 1.0.5

[ekinrf]

Same issue here, any update?

Mac OS X 10.14.6
python 3.7.6
poetry 1.0.5

[solzard]

Same for me:

  - Installing tbb (2020.0.133)

[EnvCommandError]
Command ['/path_to_project/bin/pip', 'install', '--no-deps', 'tbb==2020.0.133'] errored with the following return code 1, and output:
Collecting tbb==2020.0.133
  ERROR: Could not find a version that satisfies the requirement tbb==2020.0.133 (from versions: 2018.0.4.1, 2019.0)
ERROR: No matching distribution found for tbb==2020.0.133
WARNING: You are using pip version 19.2.3, however version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

❯ pip --version
pip 20.1.1 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)

❯ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.6
BuildVersion:   18G4032

❯ python -V
Python 3.8.3

❯ poetry -V
Poetry version 1.0.5

[kdiri]

Same here:

• Mac OS X: 10.15.5
• Python: 3.7.3
• Poetry: 1.0.5

 $ poetry add pyramid-arima

  - Updating statsmodels (0.10.2 -> 0.11.1)
  - Installing pyramid-arima (0.9.0)

[EnvCommandError]
Command ['/path_to_project/bin/pip', 'install', '--no-deps', 'pyramid-arima==0.9.0'] errored with the following return code 1, and output: 
Collecting pyramid-arima==0.9.0
  Could not find a version that satisfies the requirement pyramid-arima==0.9.0 (from versions: 0.5, 0.5.1, 0.6.2, 0.6.5, 0.7.1, 0.8.1)
No matching distribution found for pyramid-arima==0.9.0
You are using pip version 19.0.3, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

[ShamrockLee]

Same issue here on NixOS using python37Packages.poetry.

It seems that the version given from poetry search [package-name], pip3 search [package-name], or PyPI web page is somehow not available for pip to install.
E. g. Both the search result and the PyPI web page shows that the latest version of opencv-python on PyPI is 4.3.0.36, but the latest _installable_ version is actually 4.2.0.34.

When running poetry run pip3 install [package-name], however, pip simply pick up the latest available version.

$ poetry search opencv-python

opencv-python (4.3.0.36)
 Wrapper package for OpenCV python bindings.

opencv-python-headless (4.3.0.36)
 Wrapper package for OpenCV python bindings.

opencv-contrib-python (4.3.0.36)
 Wrapper package for OpenCV python bindings.

opencv-python-aarch64 (3.3.0.1)
 Wrapper package for OpenCV python bindings.

opencv-cffi (0.2.2)
 A random subset of OpenCV's functionality, wrapped via CFFI

opencv-proto (0.3.0)
 Allows fast prototyping in Python for OpenCV

opencv-camera (0.10.4)
 An OpenCV camera library

opencv-mat (0.1.4)
 A cython interface for paper: A global sampling method for alpha matting

opencv-utils (0.0.2)
 OpenCV Utilities

opencv-log (1.4.0)
 OpenCV based visual logger for debugging,logging and testing image processing code

opencv-wrapper (0.2.3)
 A Python wrapper for OpenCV.

opencv-iav (0.0.3)
 Take a picture or record a video using OpenCV.

dajngo-opencv (0.3)
 Django Opencv integratio

opencv-samjones (0.0.1)
 Perform Some Opencv face detection functions.

opencv-engine (1.0.1)
 OpenCV imaging engine for thumbor.

opencv-helpers (1.1)
 Helper functions for opencv

opencv-transforms (0.0.3.post2)
 A drop-in replacement for Torchvision Transforms using OpenCV

opencv-contrib-python-headless (4.3.0.36)
 Wrapper package for OpenCV python bindings.

opencv-openvino-contrib-python (4.1.1.26)
 Wrapper package for OpenCV python bindings.

opencv-python-inference-engine (4.3.0.2)
 Wrapper package for OpenCV 4.3.0 with Inference Engine 2020.2 python bindings

$ poetry add opencv-python
Creating virtualenv opencv-exercises-oNK0uR6U-py3.7 in /home/shamrock/.cache/pypoetry/virtualenvs
Using version ^4.3.0 for opencv-python

Updating dependencies
Resolving dependencies... (16.3s)

Writing lock file


Package operations: 13 installs, 0 updates, 0 removals

  - Installing zipp (3.1.0)
  - Installing importlib-metadata (1.7.0)
  - Installing pyparsing (2.4.7)
  - Installing six (1.15.0)
  - Installing attrs (19.3.0)
  - Installing more-itertools (8.4.0)
  - Installing numpy (1.19.0)
  - Installing packaging (20.4)
  - Installing pluggy (0.13.1)
  - Installing py (1.9.0)
  - Installing wcwidth (0.2.5)
  - Installing opencv-python (4.3.0.36)

  EnvCommandError

  Command ['/home/shamrock/.cache/pypoetry/virtualenvs/opencv-exercises-oNK0uR6U-py3.7/bin/pip', 'install', '--no-deps', 'opencv-python==4.3.0.36'] errored with the following return code 1, and output: 
  Collecting opencv-python==4.3.0.36
    ERROR: Could not find a version that satisfies the requirement opencv-python==4.3.0.36 (from versions: 3.4.2.17, 3.4.3.18, 3.4.4.19, 3.4.5.20, 3.4.6.27, 3.4.7.28, 3.4.8.29, 3.4.9.31, 3.4.9.33, 4.0.0.21, 4.0.1.23, 4.0.1.24, 4.1.0.25, 4.1.1.26, 4.1.2.30, 4.2.0.32, 4.2.0.34)
  ERROR: No matching distribution found for opencv-python==4.3.0.36
  WARNING: You are using pip version 19.2.3, however version 20.1.1 is available.
  You should consider upgrading via the 'pip install --upgrade pip' command.


  at /nix/store/3gvhkkqandb76yaqrsi822yj69l0807q-python3.7-poetry-1.0.9/lib/python3.7/site-packages/poetry/utils/env.py:893 in _run
       889│                 output = subprocess.check_output(
       890│                     cmd, stderr=subprocess.STDOUT, **kwargs
       891│                 )
       892│         except CalledProcessError as e:
    →  893│             raise EnvCommandError(e, input=input_)
       894│ 
       895│         return decode(output)
       896│ 
       897│     def execute(self, bin, *args, **kwargs):

[ShamrockLee]

Probable workaround:

1. Manually edit pyproject.toml and set the version of the affected package to "<=[available version]".
2. Set the version of other package to "*" or some wider range.
3. Type poetry update.
4. Set those packages with version "*" to the installed version.

[lig]

Can confirm the issue on Ubuntu 18.04.
Doesn't reproduce on Fedora 32.

It looks like it has something to do with this:

Starting from 4.3.0 and 3.4.10 builds the Linux build environment was updated from manylinux1 to manylinux2014. This dropped support for old Linux distributions.

https://pypi.org/project/opencv-contrib-python/4.3.0.36/ (at the bottom of the page)

Ubuntu 18.04
Poetry version 1.0.9

# poetry run pip --version
pip 9.0.1 from /root/.cache/pypoetry/virtualenvs/ocv-n9HisIcJ-py3.6/lib/python3.6/site-packages (python 3.6)

# poetry add opencv-python-headless
Using version ^4.3.0 for opencv-python-headless

Updating dependencies
Resolving dependencies... (0.1s)


Package operations: 1 install, 0 updates, 0 removals

  - Installing opencv-python-headless (4.3.0.36)

[EnvCommandError]
Command ['/root/.cache/pypoetry/virtualenvs/ocv-n9HisIcJ-py3.6/bin/pip', 'install', '--no-deps', 'opencv-python-headless==4.3.0.36'] errored with the following return code 1, and output: 
Collecting opencv-python-headless==4.3.0.36
  Could not find a version that satisfies the requirement opencv-python-headless==4.3.0.36 (from versions: 3.4.0.14, 3.4.1.15, 3.4.2.16, 3.4.2.17, 3.4.3.18, 3.4.4.19, 3.4.5.20, 3.4.6.27, 3.4.7.28, 3.4.8.29, 3.4.9.31, 3.4.9.33, 4.0.0.21, 4.0.1.23, 4.0.1.24, 4.1.0.25, 4.1.1.26, 4.1.2.30, 4.2.0.32, 4.2.0.34)
No matching distribution found for opencv-python-headless==4.3.0.36

Workaround

Run

poetry run pip install --upgrade pip

Check

# poetry run pip --version
pip 20.1.1 from /root/.cache/pypoetry/virtualenvs/ocv-n9HisIcJ-py3.6/lib/python3.6/site-packages/pip (python 3.6)

Now it works!

[ShamrockLee]

@lig Thanks for the solution, but it didn't work for me (with poetry installed from the repo on NixOS) and one of my friend (with poetry installed from get-poetry.py and python3 from Anaconda).

[lig]

@ShamrockLee maybe, you have another issue of some kind. Installing the latest pip inside an environment created with poetry does help with resolving the issue with opencv-python 4.3.0.36. I've tested it on Fedora, Ubuntu and Centos with Python 3.6.