Poetry: Command-line entry points are not created when installing Jupyter

[jacebrowning]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 10.13.5
• Poetry version: 0.11.2
• Link of a Gist with the contents of your pyproject.toml file: N/A

Issue

I assume that Jupyter might be doing something strange with their "meta" installer, but I think this should still work with Poetry:

$ poetry new .
Created package jupyter-test in .

$ poetry add jupyter
Creating virtualenv jupyter-test-py3.6 in /private/tmp/jupyter-test/.venv
Using version ^1.0 for jupyter

Updating dependencies
Resolving dependencies... (0.3s)


Package operations: 8 installs, 0 updates, 0 removals

Writing lock file

  - Installing six (1.11.0)
  - Installing atomicwrites (1.1.5)
  - Installing attrs (18.1.0)
  - Installing more-itertools (4.2.0)
  - Installing pluggy (0.6.0)
  - Installing py (1.5.4)
  - Installing jupyter (1.0.0)
  - Installing pytest (3.6.3)

$ poetry run jupyter notebook --version

[FileNotFoundError]
[Errno 2] No such file or directory: 'jupyter': 'jupyter'

run <args> (<args>)...

But if I use pip instead:

$ .venv/bin/pip install jupyter
Requirement already satisfied: jupyter in ./.venv/lib/python3.6/site-packages
Collecting notebook (from jupyter)
...
Successfully installed MarkupSafe-1.0 Send2Trash-1.5.0 appnope-0.1.0 backcall-0.1.0 bleach-2.1.3 decorator-4.3.0 entrypoints-0.2.3 html5lib-1.0.1 ipykernel-4.8.2 ipython-6.4.0 ipython-genutils-0.2.0 ipywidgets-7.2.1 jedi-0.12.1 jinja2-2.10 jsonschema-2.6.0 jupyter-client-5.2.3 jupyter-console-5.2.0 jupyter-core-4.4.0 mistune-0.8.3 nbconvert-5.3.1 nbformat-4.4.0 notebook-5.5.0 pandocfilters-1.4.2 parso-0.3.1 pexpect-4.6.0 pickleshare-0.7.4 prompt-toolkit-1.0.15 ptyprocess-0.6.0 pygments-2.2.0 python-dateutil-2.7.3 pyzmq-17.0.0 qtconsole-4.3.1 simplegeneric-0.8.1 terminado-0.8.1 testpath-0.3.1 tornado-5.1 traitlets-4.3.2 wcwidth-0.1.7 webencodings-0.5.1 widgetsnbextension-3.2.1

$ poetry run jupyter notebook --version
5.5.0

[cauebs]

The problem isn't even with the scripts: the dependencies for jupyter are not being installed. From its setup.py:

install_requires = [
    'notebook',
    'qtconsole',
    'jupyter-console',
    'nbconvert',
    'ipykernel',
    'ipywidgets',
]

I could reproduce it here, so there is definitely something wrong with the dependency resolver.

[simonpercivall]

Because poetry uses the JSON API, perhaps? It lacks correct dependency information sometimes, I've read.

[cauebs]

@simonpercivall I think you may be right: there are no requires_dist on https://pypi.org/pypi/jupyter/json.
I believe this is a known "bug": https://github.com/pypa/warehouse/issues/474#issuecomment-370974607

[sdispater]

Poetry does multiple checks to get the dependencies of a package:

• First it checks the PyPI JSON API for the requires_dist field. If it's non-empty it uses this information and doesn't go further.
• If it is empty, Poetry will download either a wheel or an sdist and inspect them.
• For wheels, it will check the *.dist-info/METADATA file for information.
• For sdists, it will check the **/*.egg-info/requires.txt file for information.
• If all this has failed, Poetry assumes no dependencies. It could get the information by executing the setup.py file but it was a decision made at the start of the project to not do it, for security reasons and also to enforce good practices in packaging.

Now, for this particular case, Poetry chooses the sdist, which doesn't hold the dependency information, and not the wheel, which has the appropriate information, because the wheel has been declared has compatible only with Python 3.4.

That being said, I think we could do something similar as to what is currently done when using private repositories, i.e. if after using the compatible distributions without success we select wheels that are not necessarily compatible. I think it will work most of the time since wheels hold this information regardless of its specified Python version.

[sdispater]

This is fixed in commit ad55beb and it will land in the next bugfix release.

[sdispater]

Release 0.11.3 is out!

[jacebrowning]

Hmmm, I can still reproduce this with 0.11.3.

[sdispater]

Well, at least you should now have the proper dependencies when installing jupyter.

Regarding entrypoints I must admit I am lost since Poetry uses pip under the hood, I am not sure why they are not properly installed.

[sdispater]

Strange, I just created a new project and did poetry add jupyter and it works fine on my end. So there must be something wrong happening when you execute poetry run

[jacebrowning]

I think many dependencies are still missing:

$ poetry --version
Poetry 0.11.3

$ poetry add jupyter
Creating virtualenv jupyter-test-py3.6 in /private/tmp/jupyter-test/.venv
Using version ^1.0 for jupyter

Updating dependencies
Resolving dependencies... (0.2s)


Package operations: 8 installs, 0 updates, 0 removals

Writing lock file

  - Installing six (1.11.0)
  - Installing atomicwrites (1.1.5)
  - Installing attrs (18.1.0)
  - Installing more-itertools (4.2.0)
  - Installing pluggy (0.6.0)
  - Installing py (1.5.4)
  - Installing jupyter (1.0.0)
  - Installing pytest (3.6.3)

Followed by:

$ poetry run pip install jupyter
...
Successfully installed MarkupSafe-1.0 Send2Trash-1.5.0 appnope-0.1.0 backcall-0.1.0
bleach-2.1.3 decorator-4.3.0 entrypoints-0.2.3 html5lib-1.0.1 ipykernel-4.8.2
ipython-6.4.0 ipython-genutils-0.2.0 ipywidgets-7.3.0 jedi-0.12.1 jinja2-2.10
jsonschema-2.6.0 jupyter-client-5.2.3 jupyter-console-5.2.0 jupyter-core-4.4.0
mistune-0.8.3 nbconvert-5.3.1 nbformat-4.4.0 notebook-5.6.0 pandocfilters-1.4.2
parso-0.3.1 pexpect-4.6.0 pickleshare-0.7.4 prometheus-client-0.3.0
prompt-toolkit-1.0.15 ptyprocess-0.6.0 pygments-2.2.0 python-dateutil-2.7.3
pyzmq-17.1.0 qtconsole-4.3.1 simplegeneric-0.8.1 terminado-0.8.1 testpath-0.3.1
tornado-5.1 traitlets-4.3.2 wcwidth-0.1.7 webencodings-0.5.1 widgetsnbextension-3.3.0

[sdispater]

@jacebrowning Oh, this is a cache problem. You need to clear the jupyter-1.0.0 cache:

poetry cache:clear pypi:jupyter:1.0.0

It should work after this.

[pmav99]

I think that this is still an issue...

Steps to reproduce:

1. I installed poetry from PyPi using pipx. The python version that runs poetry is 3.7
2. I created a python 2.7 virtualenv
3. I activated the virtualenv
4. I run poetry init -n
5. I run poetry add -D jupyter. Everything seemed fine until this point.
6. I tried to run jupyter notebook but I got ImportErrors. E.g.

Traceback (most recent call last):
  File "/home/feanor/Prog/git/grass_object/.venv/bin/jupyter-notebook", line 7, in <module>
    from notebook.notebookapp import main
  File "/home/feanor/Prog/git/grass_object/.venv/lib/python2.7/site-packages/notebook/notebookapp.py", line 62, in <module>
    from tornado import httpserver
  File "/home/feanor/Prog/git/grass_object/.venv/lib/python2.7/site-packages/tornado/httpserver.py", line 33, in <module>
    from tornado.http1connection import HTTP1ServerConnection, HTTP1ConnectionParameters
  File "/home/feanor/Prog/git/grass_object/.venv/lib/python2.7/site-packages/tornado/http1connection.py", line 29, in <module>
    from tornado import gen
  File "/home/feanor/Prog/git/grass_object/.venv/lib/python2.7/site-packages/tornado/gen.py", line 113, in <module>
    from singledispatch import singledispatch  # backport
ImportError: No module named singledispatch

After installing the missing packages, jupyter notebook runs just fine:

• poetry add -D singledispatch
• poetry add -D futures
• poetry add -D backports_abc

All three packages are tornado dependencies on python versions prior to 3.4 (source). tornado is a jupyter dependency.

python 3.7 + poetry

Repeating the procedure with a python 3.7 virtualenv works just fine (which, of course, is pretty much obvious since the packages are not needed, duh...).

Python 2.7 + directly using pip

I created a new virtualenv with python 2.7 and I run pip install jupyter. These are the installed packages:

Successfully installed MarkupSafe-1.1.0 Send2Trash-1.5.0 backports-abc-0.5 backports.shutil-get-terminal-size-1.0.0 bleach-3.0.2 configparser-3.5.0 decorator-4.3.0 defusedxml-0.5.0 entrypoints-0.2.3 enum34-1.1.6 functools32-3.2.3.post2 futures-3.2.0 ipaddress-1.0.22 ipykernel-4.10.0 ipython-5.8.0 ipython-genutils-0.2.0 ipywidgets-7.4.2 jinja2-2.10 jsonschema-2.6.0 jupyter-1.0.0 jupyter-client-5.2.4 jupyter-console-5.2.0 jupyter-core-4.4.0 mistune-0.8.4 nbconvert-5.4.0 nbformat-4.4.0 notebook-5.7.4 pandocfilters-1.4.2 pathlib2-2.3.3 pexpect-4.6.0 pickleshare-0.7.5 prometheus-client-0.5.0 prompt-toolkit-1.0.15 ptyprocess-0.6.0 pygments-2.3.1 python-dateutil-2.7.5 pyzmq-17.1.2 qtconsole-4.4.3 scandir-1.9.0 simplegeneric-0.8.1 singledispatch-3.4.0.3 six-1.12.0 terminado-0.8.1 testpath-0.4.2 tornado-5.1.1 traitlets-4.3.2 wcwidth-0.1.7 webencodings-0.5.1 widgetsnbextension-3.4.2

As you can see, all the dependencies are being pulled. I can't understand how though, since setup.py does not mention them... The full list of dependencies can probably be found here

#

All in all, it seems that poetry is not able to detect version specific dependencies (e.g. backports to python 2.7). Of course, perhaps this is just an upstream bug and tornado should be using install_requires or something, but somehow pip seems to be working just fine.

Anyway, this is not critical, since it is relatively easy to resolve by adding the packages to pyproject.toml, but newcomers to python might be puzzled by this.