poetry install failed for {path=.../git=...} dependency if the package does not have setup.py

[lambdalisue]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• macOS Catalina 10.15.3
• 1.0.5 and 1.1.0a1
• https://github.com/lambdalisue/poetry-test

Issue

I'm not sure if it's a bug on poetry but I could not complete poetry install by the following exceptions.
The minimum repository to reproduce this behavior has linked above.

With Poetry 1.0.5

Installing dependencies from lock file


Package operations: 1 install, 0 updates, 0 removals

  - Installing package-a (0.1.0 package-a)

[EnvCommandError]
Command ['/Users/alisue/Library/Caches/pypoetry/virtualenvs/poetry-test-MDzUKMz
t-py3.7/bin/pip', 'install', '--no-deps', '-U', '-e', '/Users/alisue/.ghq/githu
b.com/lambdalisue/poetry-test/package-a'] errored with the following return cod
e 1, and output:
Obtaining file:///Users/alisue/.ghq/github.com/lambdalisue/poetry-test/package-
a
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Installing collected packages: package-a
  Running setup.py develop for package-a
    ERROR: Command errored out with exit status 1:
     command: /Users/alisue/Library/Caches/pypoetry/virtualenvs/poetry-test-MDz
UKMzt-py3.7/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"
'/Users/alisue/.ghq/github.com/lambdalisue/poetry-test/package-a/setup.py'"'"';
 __file__='"'"'/Users/alisue/.ghq/github.com/lambdalisue/poetry-test/package-a/
setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read()
.replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '
"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/alisue/.ghq/github.com/lambdalisue/poetry-test/package-a/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/alisue/Library/Caches/pyp
oetry/virtualenvs/poetry-test-MDzUKMzt-py3.7/bin/python -c 'import sys, setupto
ols, tokenize; sys.argv[0] = '"'"'/Users/alisue/.ghq/github.com/lambdalisue/poe
try-test/package-a/setup.py'"'"'; __file__='"'"'/Users/alisue/.ghq/github.com/l
ambdalisue/poetry-test/package-a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'
"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close
();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the l
ogs for full command output.
WARNING: You are using pip version 19.2.3, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

I have no idea but executing /Users/alisue/.../bin/python -c 'import setuptools' after above exception does NOT fail. Thus it seems setuptools is not available only during pip installation???

Poetry 1.1.0a1

Installing dependencies from lock file


Package operations: 1 install, 0 updates, 0 removals

  - Installing package-a (0.1.0 package-a)

  TypeError

  __init__() takes from 2 to 3 positional arguments but 4 were given

  at ~/.poetry/lib/poetry/installation/pip_installer.py:214 in install_director
y
      210|             # file since pip, as of this comment, does not support
      211|             # build-system for editable packages
      212|             # We also need it for non-PEP-517 packages
      213|             builder = SdistBuilder(
    > 214|                 Factory().create_poetry(pyproject.parent), NullEnv()
, NullIO()
      215|             )
      216|
      217|             with open(setup, "w", encoding="utf-8") as f:
      218|                 f.write(decode(builder.build_setup()))

I've followed SdistBuilder -> Builder on my environment and found the following on ~/.../python/3.7.7/lib/python3.7/site-packages/poetry/core/masonry/builders/builder.py (not in ~/.poetry but where python exists)

class Builder(object):
    AVAILABLE_PYTHONS = {"2", "2.7", "3", "3.4", "3.5", "3.6", "3.7"}

    format = None

    def __init__(
        self, poetry, ignore_packages_formats=False
    ):  # type: ("Poetry", "Env", "IO", bool) -> None
        self._poetry = poetry
        self._package = poetry.package

It seems the poetry-core version installed is wrong?

[lambdalisue]

I've found that the poetry-core package bundled in poetry 1.1.0a1 is not correct.
At least poetry-1.1.0a1-darwin.tar.gz in https://github.com/python-poetry/poetry/releases/tag/1.1.0a1 has the following.

poetry/_vendor/py3.7/poetry/core/masonry/builders/builder.py

# ...
class Builder(object):
    AVAILABLE_PYTHONS = {"2", "2.7", "3", "3.4", "3.5", "3.6", "3.7"}

    format = None

    def __init__(
        self, poetry, ignore_packages_formats=False
    ):  # type: ("Poetry", "Env", "IO", bool) -> None
        self._poetry = poetry
        self._package = poetry.package
        self._path = poetry.file.parent
        self._original_path = self._path

        packages = []

# ...

See __init__ which does NOT have 4 arguments but 3.

[lambdalisue]

I've found that in 1.0.5, installing a package WITHOUT setup.py caused that issue

$ ~/Library/Caches/pypoetry/virtualenvs/poetry-test-MDzUKMzt-py3.7/bin/pip install --no-deps -U -e ./package-a/
ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /Users/alisue/.ghq/github.com/lambdalisue/poetry-test/package-a
(A "pyproject.toml" file was found, but editable mode currently requires a setup.py based build.)
WARNING: You are using pip version 19.2.3, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Not sure yet why poetry tried to install a package as editable...

[lambdalisue]

~Finally I've found a workaround. Use Poetry 1.0.5 and develop = false like~ (It works only when you use path = ... but git = ....)

package-a = { path = "./package-a", develop = false }

I think some note about this behavior is required in a documentation at least. It would be better if there is warning message or so on.

---

Unfortunately, above workaround does not work on git = ...

[RuntimeError]
The Poetry configuration is invalid:
  - [dependencies.xxx] {'git': '...', 'develop': False} is not valid under any of the given schemas

I've found that https://github.com/python-poetry/poetry/pull/1986 fixes above issue. And it seems this entire issue (path = ... or git = ... does not work) appears from pip 19.1 according to the issue note of https://github.com/python-poetry/poetry/issues/1080

[lambdalisue]

It seems using pip 18.1 solves the issue for now. I'll use the following as a workaround for this issue.

$ poetry env remove 3.7
$ poetry run pip install 'pip<19.0'
$ poetry install
$ poetry run python -c 'import package_a'
This is package A

Note that pip 19.0 implements PEP 517 and that cause 'setuptools' exception because no setuptools exists under the isolated environment for building.

[albireox]

Any news on this? It breaks all workflows that require using path dependencies for development.

[abn]

This should be fixed in the next preview release.

[exhuma]

Not sure if this is wholly related, but I have the same problem (albeit with another error).

I work on two projects mainproject and library for example. mainproject uses library and library is used in other applications as well, serving as an abstraction/sharing layer between several projects. mainproject requires some additions to library and for that reason, I normally install library into the environment of mainproject using pip install -e /path/to/library. This allows me to update the library while I also directly do the bulk of the work in the main project.

I am now in the process of evaluating a build workflow based around pyproject.toml and am testing out both flit and poetry. As a POC I'm using flit in the "library" project and poetry in the "main" project.

This means, that the source tree of the "library" only contains a pyproject.toml file without a setup.py file. What makes things even trickier, is that flit does not write the version number into pyproject.toml but picks it up from library.__version__.

In the example below, the main project is called "ppproj" and the library is a submodule inside the folder libs/mylib in that project. With that structure, the following fails:

$ cd /path/to/ppproj
$ poetry add -D /libs/mylib

Using virtualenv: /home/exhuma/.cache/pypoetry/virtualenvs/ppproj-Li4ARlUq-py3.8

[RuntimeError]
Unable to retrieve the package version for /home/exhuma/tmp/ppproj/libs/mylib

Traceback (most recent call last):
  File "/home/exhuma/.local/lib/python3.8/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/home/exhuma/.local/lib/python3.8/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/home/exhuma/.local/lib/python3.8/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/home/exhuma/.local/lib/python3.8/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/home/exhuma/.local/lib/python3.8/site-packages/poetry/console/commands/add.py", line 88, in handle
    requirements = self._determine_requirements(
  File "/home/exhuma/.local/lib/python3.8/site-packages/poetry/console/commands/init.py", line 293, in _determine_requirements
    requires = self._parse_requirements(requires)
  File "/home/exhuma/.local/lib/python3.8/site-packages/poetry/console/commands/init.py", line 405, in _parse_requirements
    package = Provider.get_package_from_directory(path)
  File "/home/exhuma/.local/lib/python3.8/site-packages/poetry/puzzle/provider.py", line 372, in get_package_from_directory
    raise RuntimeError(

[abn]

This issue should be resolved via #2302 #2810.

@exhuma your issue might be fixed via #2957.