Twine: upload to untrusted hosts with --trusted option

[glenfant]

Hi,

Twine denies uploading to untrusted https PyPI clones. This triggers an SSL error caused by an unknown CA certificate. I need the ability to upload packages to untrusted private PyPI servers since

Backgroung : I'm in process to add PyPI and NPM support to a Nexus server that's published in production with regular CA signed certificates. But can't obtain such certificates for the Nexus test site.

I made a small quick'n'dirty patch setting verify=False option when posting a tarball or wheel file here.
https://github.com/pypa/twine/blob/master/twine/repository.py#L152

I know that the requests lib issues a warning in such situations but I don't care.

Thanks

[glenfant]

Would you accept a PR with a new option "trusted-host" option that does the same job as for pip ?

$ pip help install
...
  --trusted-host <hostname>   Mark this host as trusted, even though it does
                              not have valid or any HTTPS.
...

[sigmavirus24]

That seems like a good way to name it and go about it, @glenfant .

[piersf]

Hello there,

Was this ever implemented?
We need this badly as setup.py upload command does not seem to be working anymore with Python 3.7.x versions. when uploading to private PyPi repositories(gives 400 Bad Request)

[bhrutledge]

Thanks for the ping, @piersf. There's currently an open PR #463 that's waiting for the submitter to make some changes. I'll ping him to see if/when he's able to do that.

[di]

Sorry I didn't comment earlier, but I wanted to say that I think I'm -1 on making this change.

I'm a bit concerned that if we add this flag, it will proliferate through comments, stack overflow answers, etc as an "easy" way to fix SSL errors that should actually be fixed by producing and obtaining valid certificates for third-party indices. Basically, this shouldn't be necessary and feels like an easy way out.

I'm especially concerned about the possibility of this being used with --trusted-host pypi.org which should basically never happen. At the very least, I'd like to see this being implemented in a way that prevents this flag from ever being used with that host.

(cc @ewdurbin for thoughts here as well)

[glenfant]

I agree, it's preferable and technically easy to add a real CA approved certificate to the private registry server (Nexus) of my customer's company, but it's a political nightmare. I can understand your opinion, but the only workaround I found is to add to every workstation, CI container (etc.) a script that add the private CA cert to the standard keyring :/

[glenfant]

@di If I understand correctly, could you accept a change that ignores --trusted on *.pypi.org target registry?

[itaym120]

Sorry I didn't comment earlier, but I wanted to say that I think I'm -1 on making this change.

I'm a bit concerned that if we add this flag, it will proliferate through comments, stack overflow answers, etc as an "easy" way to fix SSL errors that should actually be fixed by producing and obtaining valid certificates for third-party indices. Basically, this shouldn't be necessary and feels like an easy way out.

I'm especially concerned about the possibility of this being used with --trusted-host pypi.org which should basically never happen. At the very least, I'd like to see this being implemented in a way that prevents this flag from ever being used with that host.

(cc @ewdurbin for thoughts here as well)

It's not always possible to work the "right" way, that's why pip, conda install and npm gives the option to add trusted-host\verify-ssl flags. Not as the recommended way, but as a possiblity for users. The default should obviously be to verify whenever its a possibility.

In my case, I just can't demand from the Artifactory repository providers to add a real CA approved certificate and I have no choise but use verify=False when uploading and downloading packages from there. and there are no alternatives in the lan...

[jaraco]

Is there a better option here? Is it possible without disabling SSL to still trust/whitelist it? That is, is it possible to just download the certificate and whitelist it? Something like:

import ssl
import sys
import urllib.parse

url, = sys.argv[1:]
if '/' not in url:
    url = '//' + url
parsed = urllib.parse.urlparse(url)

with open('cert.pem', 'w') as f:
    f.write(ssl.get_server_certificate((parsed.hostname, parsed.port or 443)))

to download the certificate and then

REQUESTS_CA_BUNDLE=cert.pem twine upload ...

According to this answer, I'd expect that to work, but it didn't work for me when I tested against self-signed.badssl.com.

[sigmavirus24]

@jaraco this is what --cert is supposed to provide. If you have the certificates as a PEM file which can be generated trivially by searching StackOverflow, then they should be able to pass that to --cert iirc instead of having --trusted. That solves only the one case that @glenfant described though.

There's still the issue of "what about an implementation without TLS?" I think the answer there is "maybe that works today already." I don't think we validate for https:// as a scheme or change that forcefully. If we do, then we'd need an option along the lines of --yes-i-know-i-should-use-tls but that wouldn't be fixed by --trusted because we'd otherwise still try connecting over :443 and the server may not even listen on that port.

[jaraco]

I can say with confidence that upload requests over http work, because twine now has integration tests against a local devpi instance over plaintext HTTP.

[okainov]

+1 for the issue, I'd really like to have this option in

[jaraco]

The solution to this problem is threefold:

• insecure requests over plain HTTP are allowed
• where SSL is used with an otherwise untrusted certificate, the --cert parameter should be used to pass the trusted certificate
• if the above solutions aren't available, implement a hack (such as this) at your own risk to disable certificate checking.

This last option is intentionally ugly and inconvenient to limit the proliferation of that discouraged approach.

If none of these approaches are suitable, please feel free to elaborate on why.