Poetry: Relative path imports are resolved relative to CWD instead of pyproject.toml

[swist]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: circleci/python3.7.4-stretch, osx
• Poetry version:1.0.0
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/swist/89660bf70dbed6d0622bba7db381ae9c

Issue

I have 3 projects in a monorepo situation
One has a pyproject.toml in kafkaesque/pyproject.toml
It depends on another poetry package defined in config/pyproject.toml
Then there's another project that installs from kafkaesque and config into pyproject.toml like so:

config/pyproject.toml
kafkaesque/pyproject.toml
pyproject.toml

I get the following error:

[EnvCommandError]
Command ['/Users/swistak/Library/Caches/pypoetry/virtualenvs/marmot-api-SV6HPF22-py3.7/bin/pip', 'install', '--no-deps', '-U', '/Users/swistak/code/marmot/api/kafkaesque'] errored with the following return code 1, and output: 
Processing ./kafkaesque
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  Complete output from command /Users/swistak/Library/Caches/pypoetry/virtualenvs/marmot-api-SV6HPF22-py3.7/bin/python /Users/swistak/Library/Caches/pypoetry/virtualenvs/marmot-api-SV6HPF22-py3.7/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/93/78_vw68j75j3xf_z25p29_b80000gn/T/tmpmcyd92s5:
  Traceback (most recent call last):
    File "/Users/swistak/Library/Caches/pypoetry/virtualenvs/marmot-api-SV6HPF22-py3.7/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
      main()
    File "/Users/swistak/Library/Caches/pypoetry/virtualenvs/marmot-api-SV6HPF22-py3.7/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/swistak/Library/Caches/pypoetry/virtualenvs/marmot-api-SV6HPF22-py3.7/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/private/var/folders/93/78_vw68j75j3xf_z25p29_b80000gn/T/pip-build-env-go6yfl9w/overlay/lib/python3.7/site-packages/poetry/masonry/api.py", line 25, in get_requires_for_build_wheel
      poetry = Factory().create_poetry(Path("."))
    File "/private/var/folders/93/78_vw68j75j3xf_z25p29_b80000gn/T/pip-build-env-go6yfl9w/overlay/lib/python3.7/site-packages/poetry/factory.py", line 99, in create_poetry
      package.add_dependency(name, constraint)
    File "/private/var/folders/93/78_vw68j75j3xf_z25p29_b80000gn/T/pip-build-env-go6yfl9w/overlay/lib/python3.7/site-packages/poetry/packages/package.py", line 331, in add_dependency
      develop=constraint.get("develop", True),
    File "/private/var/folders/93/78_vw68j75j3xf_z25p29_b80000gn/T/pip-build-env-go6yfl9w/overlay/lib/python3.7/site-packages/poetry/packages/directory_dependency.py", line 36, in __init__
      raise ValueError("Directory {} does not exist".format(self._path))
  ValueError: Directory ../config does not exist

Note that both kafkaesque/pyproject.toml and pyproject.toml both depend on config/pyproject.toml albeit with a different path in their respective project files

[finswimmer]

Hello @swist ,

could be that this is similar to https://github.com/python-poetry/poetry/issues/1689.

What's the pip version within the venv? If it's greater >18, could you please try downgrading pip to a version <19 within the venv?

fin swimmer

[swist]

Hey, I have managed to carve some more time to debug this. Removing develop = false from all of the dependencies that I install via relative paths seems to have fixed it. Could it perhaps be related to the fact that poetry quite eagerly cleans setup.py when develop = false ?

I think it might have been broken all along, I'll update the initial post (just tried with 1.0.0b8 and the same happened)

[swist]

Another update. Even with the change outlined above I am still getting pathlib value error when I try to run poetry build

[teknico]

I'm hitting this too. I have two projects in one repo, the second one depends on the first, I'm using a relative path, and I get the same error in the same place.

[finswimmer]

Should be fixed in the latest release by #1875.

[teknico]

Regrettably I'm still getting the same error with v.1.0.3 (which includes #1875).

[finswimmer]

Hello @teknico ,

can you please show how your folder structure looks like and provide the pyproject.toml's?

fin swimmer

[teknico]

@finswimmer, sure, the code is in the tinydecred repo. The tinywallet project depends on the decred project's code by means of a relative path. Thanks.

[finswimmer]

Ok, I've got it. Will have a closer look at it the next days.

[finswimmer]

I tried to take a closer look at this problem. It arises if a path dependency is more then one time in the dependency tree and has different relative paths. Then something went wrong during the version solving. And that's the part in the poetry which is still hard for me to understand.

I've uploaded a small example I used for testing: https://github.com/finswimmer/poetry-issue1757

I tried to follow the code. The dependency that is passed to search_for here:

https://github.com/python-poetry/poetry/blob/12db4a5e189ea8514e70f22279ae540274a0e5af/poetry/puzzle/provider.py#L97

has a wrong path value, while base and full_path are ok:

depency.name: packageb
dependency.base: /home/finswimmer/tmp/poetry/issue_1757/package
dependency.path: packagea/../packageb
dependency.full_path: /home/finswimmer/tmp/poetry/issue_1757/packagea/../packageb

The correct value for depency.path would be ../packageb. This is correct identified in an earlier step, when poetry finds packagea and collect it's dependencies.

So it looks like, that poetry is mixing some information when it recognize that the same package name is referenced twice and it has to decide which one it should take.

@sdispater could you please take a look at this?

[sdispater]

@finswimmer I'll try to take a look at this. I thought I had fixed but I missed the case where, like you mentioned, the same package is references multiple times with various relative paths.

I'll assigned myself to the issue.

[Timost]

I'm also hitting this issue. Not sure if this will help but I work in a monorepo context with the following structure:

.
├── acquisition
│   ├── customclient
│   │   ├── poetry.lock                                                                                                                                                                                                                                                                   
│   │   ├── pyproject.toml                                                                                                                                                                                                                                                                
│   │   ├── README.md     
│   ├── ...
├── commons
│   ├── logutils
│   │   ├── poetry.lock                                                                                                                                                                                                                                                                   
│   │   ├── pyproject.toml                                                                                                                                                                                                                                                                
│   │   ├── ...
│   ├── geoutils
│   │   ├── poetry.lock                                                                                                                                                                                                                                                                   
│   │   ├── pyproject.toml                                                                                                                                                                                                                                                                
│   │   ├── ...
├── top_project
│   ├── .....
│   ├── poetry.lock
│   ├── pyproject.toml

geoutils depends on logutils. (../logutils)
top_project depends on geoutils (../commons/geoutils) and on logutils (../commons/logutils)
customclient depends on logutils (../../commons/logutils) and top_project (../../top_project)

I know this might look messed up in terms of layout, but the path resolved for (logutils) is wrong when I do a poetry install in acquisition/customclient

[yanbussieres]

Python 3.9, pip 20.2.3

Is this solving every corner case ?
Please have a look at this example :
Example here

• main requires dep2,
• dep2 requires dep1

Trying to run poetry install from the main folder, I still have this issue :

 File "/private/var/folders/5d/yl0b7h4x00s0jt_zjqy58r040000gn/T/pip-build-env-19rc_6el/overlay/lib/python3.9/site-packages/poetry/core/packages/directory_dependency.py", line 36, in __init__
          raise ValueError("Directory {} does not exist".format(self._path))
      ValueError: Directory ../dep1 does not exist

Solution _(kind of)_

1. from lib/dep2, python 3.9, pip 20.2.3, runpoetry install
2. from main/ , python 3.9, Downgrading to pip 18.0, run poetry install

Or
Everything is ok with python 3.8, pip 20.2.3, poetry 1.0.10 (all installed from brew)

[finswimmer]

Hello @yanbussieres,

since poetry 1.1 path dependencies are no longer installed in editable mode by default. You will need to add a develop=true to your dependency definitions in the pyproject.toml, e.g. dep1 = {path = "../dep1", develop=true}.

fin swimmer