Poetry: Upload to PyPI silently fail

[ghost]

• [X] I am on the latest Poetry version.
• [X] I have searched the issues of this repo and believe that this is not a duplicate.

• [X] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: MacOS Mojave

• Poetry version: 0.12.10
• pyproject.toml

Issue

poetry publish fails silently if the Pypi server returns an error:

1. If I use invalid credentials (it even shows the "Uploading foobar.tar.gz 100%" mesage)
2. If the Pypi server returns an error.

I was trying to upload a project with an invalid name ("pipes", invalid because Python standard library has a module named the same) and nothing indicated me that the publishing was failing.

If I try with twine this is the output instead:

$ twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Uploading Pipes-0.1.0-py3-none-any.whl
100%|████████████████████| 4.26k/4.26k [00:00<00:00, 7.88kB/s]
NOTE: Try --verbose to see response content.
HTTPError: 403 Client Error: The user 'jpscaletti' isn't allowed to upload to
project 'pipes'. See https://pypi.org/help/#project-name for more information.
for url: https://upload.pypi.org/legacy/

I think #742 is a specific case of this issue.

[carlosperate]

I had a similar issue using v0.12.16 where using https://test.pypi.org/simple/ failed silently, but https://test.pypi.org/legacy/ worked.

[sztomi]

@sdispater I have a fairly good idea why this is happening and a proposal for a fix.

In uploader.py

            resp = session.post(
                url,
                data=monitor,
                allow_redirects=False,
                headers={"Content-Type": monitor.content_type},
            )

            if resp.ok:
                bar.finish()

                self._io.writeln("")

The bug is triggered when the URL is not exactly https://test.pypi.org/legacy/ (note the trailing /). If the URL is missing the trailing /, the response will be a 301 redirect to /legacy/. resp.ok is still True in this case, but the upload did not take place (nor was the redirect followed).
Setting allow_redirects=True does make requests follow the redirect, but it doesn't seem to perform the upload (BUT the status code becomes 200). The same thing happens for /simple (gets redirected to /simple/). If the URL is configured to /simple/, a HTTP 405 error is raised.

In summary:

/legacy -> silently fails because of 301 status, resp.ok is True
/legacy/ -> works :sparkles:
/simple -> silently fails because of 301 status, resp.ok is True
/simple/ -> fails with 405 status

My proposal for the fix is:

• Test resp.status_code == 200 instead of resp.ok
• Follow the redirect "manually" (i.e. perform another request to that URL with the POST data, because requests doesn't seem to do it that way)
• Maybe check the uploaded versions afterwards to verify that the upload worked

This will make /simple fail as 405, too, and /legacy will work. I think it would be worthwhile to highlight in the docs that the legacy API is expected by poetry.

Also, it might be a good idea to add a default testpypi repository to the configuration?

If this proposal is liked, I'm happy to implement the first two points and open a PR. Point 3 is an improvement.

[geckon]

Can confirm this bug still exists. The fix is really easy, just go to your configuration file (~/.config/pypoetry/config.toml on my system) and replace url = "https://test.pypi.org/legacy" with url = "https://test.pypi.org/legacy/".

[zyxue]

I'm seeing this as well with poetry-1.0.9 with a private pypi server

[claeyzre]

I am also seeing this with poetry 1.0.9 with a private pypi server

[abn]

@zyxue @claeyzre can you try 1.0.10 or 1.1.0b2 please?

[claeyzre]

Thanks for the reply @abn
The issue is still the same with 1.0.10
I get a "HTTP Error 405: Method Not Allowed" when pushing to https://X.X.com/simple/

[sztomi]

@abn if you look at my analysis above (and compare the state of the code today: https://github.com/python-poetry/poetry/blob/master/poetry/publishing/uploader.py#L262) you will see that the root cause has not been addressed.

[juyoung-yoo]

Thanks for the reply @abn
The issue is still the same with 1.0.10
I get a "HTTP Error 405: Method Not Allowed" when pushing to https://X.X.com/simple/

1.1.0b2 has the same problem

[abn]

@claeyzre @juyoung-yoo you cannot push to the simple/ endpoint. The expected upload api is typically legacy/. Obviously this can be different for different flavours of indices (artifactory, nexus, azure, bintray etc.). This is not poetry specific, but rather what warehouse specifies. This also means some index implementations play loose and fast with their APIs.

@sztomi appreciate the analysis. As for the redirects on trailing slashes, we should simply error out gracefully with a message saying what status was returned. So simply ensuring that it is okay and not in the 300 range is what we need. Since different index implentations return different 20X codes for upload unfortunately. We should not try to be smart about recovery since that could end up causing more issues than it solves. We can also revisit this later. I am happy to review a PR on the 300 checks. (#3069)

The issue reported here was resolved with #2285.

[davidbfrogman]

I just wanted to add it seems my private repo seemed to have 405 problems with either legacy or simple. I corrected by chopping both off.

final url:

http://myserver.com

(note the lack of "legacy" or "simple"

Comment that pointed me in that direction:
https://github.com/pypiserver/pypiserver/issues/212#issuecomment-454661922