Poetry: Export fails after upgrading poetry to version 1.1.0

[iksteen]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Linux (archlinux and CentOS 7).
• Poetry version: 1.1.0
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/iksteen/2464d767a57ed034af25956264be2694

Issue

After upgrading poetry from 1.0.0 to 1.1.0, poetry export -frequirements.txt results in the following exception when using a poetry.lock file created by an earlier version: https://gist.github.com/iksteen/bd4fd24e2b79e0f0644461fe501297d4

Running poetry lock using poetry 1.1.0 fixes the problem.

While not a huge issue, this currently frustrates our CI strategy as we have an assumption there that installing poetry>=1.0,<2.0 would be sufficient to guard against incompatibilities of poetry updates. Currently we have quite a few PRs on several projects that suddenly won't build in the CI anymore.

While this seems similar to https://github.com/python-poetry/poetry/issues/3018, the conditions under which this occurs differ significantly (#3018 is also triggered after running poetry lock).

[craigjbass]

We've also noticed the same issue in all our builds :sob:

[remram44]

Same. Also the error gets printed to stdout, which is piped to the requirements.txt output. I guess I'll open an issue about that (edit: #3025)

[finswimmer]

Fascinating that many people seem to use poetry export :) Can someone enlighten me please what are the use cases for that? Can be over at discord, so that we don't get off topic in this issue.

[remram44]

Personally I use this as part of CI, to install a subset of the packages (export requirements.txt, then use it with pip install --constraint

I also use it for Docker builds. To use the cache effectively, you want to install dependencies first, then put your code in and install it. I believe the use cases are already tracked as #936

[finswimmer]

Personally I use this as part of CI, to install a subset of the packages (export requirements.txt, then use it with pip install --constraint

Sounds like the planned feature "dependency groups", correct?

[remram44]

Groups would work, thanks for bringing up that issue. Although in my case that's a single package, I just want to install flake8 and its dependencies, but using the locked package versions (CI config).

The current way works fine (the script I wrote to replace poetry install is way more contrived).

[mikenerone]

"Me too." My use case for export is that I need to pull dependencies for distribution for offline installs. So export combines with some pip wheel, and I have what I need. Basically #2873.

[abn]

@iksteen any chance you can verify that #3024 fixes your issue as well? I could not test your pyproject file due to the repo being private obviously. And since export uses lock file ... :) Appreciate the report.

[abn]

I have raised a feature request (#3033) to handle installing specific packages.

As for caching, docker build, I recommend you consider doing the following.

```Dockerfile
FROM docker.io/python:3.8

RUN python -m pip install poetry
RUN python -m poetry config virtualenvs.in-project true

RUN install -d /opt/src

WORKDIR /opt/src

COPY poetry.lock pyproject.toml /opt/src/

RUN python -m poetry install --no-dev --no-root

COPY . /opt/src/

RUN python -m poetry install --no-dev

ENTRYPOINT ["python", "-m", "poetry", "run"]
CMD ["bash"]

[abn]

While not a huge issue, this currently frustrates our CI strategy as we have an assumption there that installing poetry>=1.0,<2.0 would be sufficient to guard against incompatibilities of poetry updates. Currently we have quite a few PRs on several projects that suddenly won't build in the CI anymore.

We really do appreciate that this is an issue. However, please do bear with us. The various variables to consider in how people are using poetry is sometimes hard to keep track of. Especially when we need to make critical changes to to the resolver to expand both reliability and supported scenarios. Again, thank you for using poetry, and providing detailed bug reports.

[iksteen]

@iksteen any chance you can verify that #3024 fixes your issue as well? I could not test your pyproject file due to the repo being private obviously. And since export uses lock file ... :) Appreciate the report.

Yes, this fixes the issue. Thank you, this saves me (and my team) quite a bit of manual intervention once released.

We really do appreciate that this is an issue. However, please do bear with us. The various variables to consider in how people are using poetry is sometimes hard to keep track of. Especially when we need to make critical changes to to the resolver to expand both reliability and supported scenarios. Again, thank you for using poetry, and providing detailed bug reports.

I absolutely understand this and sympathize. Also, I would've been fine with backwards compatibility breaking if poetry's new version number had semantically represented that (eg. 2.0.0). Things don't have to be backwards compatible forever.

[abn]

@iksteen this wasn't meant to be a breaking change. The export behavior should have remained as is, but the changes in the lockfile meant that we had to resolve all dependencies without evironment markers available. This forced the change in this logic, however we weren't able to test all cases thoroughly since this was patched after rc.

So, if all had gone well this would have just worked with an old lock file. Hence why we didn't bump the major for this one.

[remram44]

The fact that the new lock files can't be read by the old Poetry is the breaking change.

[klintan]

I run poetry in Github actions, and run poetry export -f requirements.txt > requirements.txt for python-serverless-requirements. Since I run curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 I got the latest version, which made it fail.

Tried to run poetry lock, poetry update and a range of other stuff before that command. Tried 1.1 branch as well after the referenced PR in this issue, but issue remained (even so that same change looked like it got into master as well so shouldn't really have made any difference from the master branch).

If anyone else hits this issue for poetry 1.1: only thing that helped was to revert back to 1.0.10.

[abn]

@klintan can ypu provide a sample pyproject.toml that is failing please? If you could, please raise a new issue with the sample pyproject.toml along with the output of the command when run with -vvv.

[remram44]

Is there documentation on how one might test #3024? It seems that, to set up Poetry for development, I need a working Poetry already? Am I reading the contributing guide correctly?

[remram44]

I can confirm that it does work for me (poetry 1.1.0-8-g32b01be). I did not have to update my lock file (#3034).