Poetry: Do not install Ansible dependencies

[nikaro]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Fedora 29
• Poetry version: 0.12.5
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/nikaro/f192e0c0a4a53f9d89fc500db8a69154

Issue

It does not install Ansible dependencies, only Ansible itself, alone:

$ poetry install -vvv
Creating virtualenv envsetup3-py3.7 in /home/nicolas/Projets/ubicast/envsetup3/.venv
Using virtualenv: /home/nicolas/Projets/ubicast/envsetup3/.venv
Updating dependencies
Resolving dependencies...
   1: fact: envsetup3 is 0.1.0
   1: derived: envsetup3
   1: fact: envsetup3 depends on ansible (^2.6)
   1: selecting envsetup3 (0.1.0)
   1: derived: ansible (^2.6)
PyPI: No release information found for ansible-1.9.0, skipping
PyPI: No release information found for ansible-2.0.0, skipping
PyPI: 16 packages found for ansible >=2.6,<3.0
   1: selecting ansible (2.7.1)
   1: Version solving took 0.013 seconds.
   1: Tried 1 solutions.


Package operations: 1 install, 0 updates, 0 removals

Writing lock file

  - Installing ansible (2.7.1)

$ poetry run pip freeze
ansible==2.7.1

What it should install:

$ python3 -m venv /tmp/ansible

$ /tmp/ansible/bin/pip install ansible
Collecting ansible
  Using cached https://files.pythonhosted.org/packages/ec/ee/1494474b59c6e9cccdfde32da1364b94cdb280ff96b1493deaf4f3ae55f8/ansible-2.7.1.tar.gz
Collecting jinja2 (from ansible)
  Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting PyYAML (from ansible)
  Using cached https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz
Collecting paramiko (from ansible)
  Using cached https://files.pythonhosted.org/packages/cf/ae/94e70d49044ccc234bfdba20114fa947d7ba6eb68a2e452d89b920e62227/paramiko-2.4.2-py2.py3-none-any.whl
Collecting cryptography (from ansible)
  Using cached https://files.pythonhosted.org/packages/59/32/92cade62c645756a83598edf56289e9b19aae5370642a7ce690cd06bc72f/cryptography-2.3.1-cp34-abi3-manylinux1_x86_64.whl
Requirement already satisfied: setuptools in /tmp/ansible/lib/python3.7/site-packages (from ansible) (40.4.3)
Collecting MarkupSafe>=0.23 (from jinja2->ansible)
  Using cached https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz
Collecting pyasn1>=0.1.7 (from paramiko->ansible)
  Using cached https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl
Collecting pynacl>=1.0.1 (from paramiko->ansible)
  Using cached https://files.pythonhosted.org/packages/27/15/2cd0a203f318c2240b42cd9dd13c931ddd61067809fee3479f44f086103e/PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
Collecting bcrypt>=3.1.3 (from paramiko->ansible)
  Using cached https://files.pythonhosted.org/packages/a8/ce/1b9fe8001f95191a6254736d502fa51a2ff6bd0ffa9e290640f909b3adcb/bcrypt-3.1.4-cp34-abi3-manylinux1_x86_64.whl
Collecting asn1crypto>=0.21.0 (from cryptography->ansible)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting six>=1.4.1 (from cryptography->ansible)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.7 (from cryptography->ansible)
  Using cached https://files.pythonhosted.org/packages/51/7b/d1014289d0578c3522b2798b9cb87c65e5b36798bd3ae68a75fa1fe09e78/cffi-1.11.5-cp37-cp37m-manylinux1_x86_64.whl
Collecting idna>=2.1 (from cryptography->ansible)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography->ansible)
  Using cached https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz
Installing collected packages: MarkupSafe, jinja2, PyYAML, pyasn1, asn1crypto, six, pycparser, cffi, idna, cryptography, pynacl, bcrypt, paramiko, ansible
  Running setup.py install for MarkupSafe ... done
  Running setup.py install for PyYAML ... done
  Running setup.py install for pycparser ... done
  Running setup.py install for ansible ... done
Successfully installed MarkupSafe-1.0 PyYAML-3.13 ansible-2.7.1 asn1crypto-0.24.0 bcrypt-3.1.4 cffi-1.11.5 cryptography-2.3.1 idna-2.7 jinja2-2.10 paramiko-2.4.2 pyasn1-0.4.4 pycparser-2.19 pynacl-1.3.0 six-1.11.0

$ /tmp/ansible/bin/pip freeze
ansible==2.7.1
asn1crypto==0.24.0
bcrypt==3.1.4
cffi==1.11.5
cryptography==2.3.1
idna==2.7
Jinja2==2.10
MarkupSafe==1.0
paramiko==2.4.2
pyasn1==0.4.4
pycparser==2.19
PyNaCl==1.3.0
PyYAML==3.13
six==1.11.0

[sdispater]

Unfortunately, this is on ansible's end. They should make their dependencies available in a declarative way since Poetry does not execute the setup.py file to get the dependencies for security reasons.

They also don't have a *.egg-info directory in their sdist which sometimes can be used to get the dependencies information.

And finally, they don't publish a wheel on PyPI which have a metadata file that can hold this information.

So, as it is, Poetry has no way to get the dependencies. I am trying to figure out a way to extract the dependency information from the setup.py file without executing it but for the time being it would be preferable to notify the ansible's maintainer to improve the way they package their project.

[nikaro]

That's what i was afraid of…

I know that they are already working on a way to provide wheel package, but do not know if they have progressed on it. Cf. https://github.com/ansible/ansible/issues/12610

[lig]

@sdispater Well, this is no way on Ansible end. As bad as it is executing setup.py or running pip to do the same and build the wheel is essential to install some packages.

If poetry has no way to do it this means the only thing: poetry lacks some setup.py support. There is no doubt that the default way poetry behaves is better but it's clear that this way is not applicable for all cases.

So, it looks like poetry needs a way to execute setup.py or pip for some packages. Surely, this should be defined per package and doesn't need to become the default behavior.