Twine: GPG signing - how does that really work with PyPI?

[techtonik]

Although twine supports signing, I don't see how PyPI validates the signature or how can users do this manually. Is there some quick rtfm/intro that I've missed?

[dstufft]

GPG signing is not validated in any way by anything Python tools (including PyPI). It's an older, less thought out feature that currently is only generally useful for tools like Debian's uscan which can be configured to verify a GPG signature. It is also possible to manually verify it, but I doubt many people ever do that.

[techtonik]

Thanks for the fast reply. Is this signature is at least stored by PyPI so that Debian tools can grab and check it? What is the manual process, by the way?

[dstufft]

If the signature was uploaded to PyPI (as is the case with Twine, and distutils/setuptools) it is stored at the same URL as the file with a .asc appended to it. For example, using pip 7.1.2 the wheel file is located at https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl and the signature is located at https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl.asc. This is also linked from the UI if it's available.

To verify, you'd download both the file and the signature and do gpg --verify pip-7.1.2-py2.py3-none-any.whl.asc pip-7.1.2-py2.py3-none-any.whl. Sadly, there isn't a good way to know what key _should_ be signing releases. The best you can do is just a trust-on-first-use style mechanism where you require manual intervention if the key changes (which is what Debian typically does in their uscan program).

[LucidOne]

keybase's approach seems interesting for key validation and Github's "stars" and their support of signed commits and releases seems to provide additional points of consensus.

Hardware devices such as the Yubikey and Nitrokey support more or less secure storage of private keys and are making GPG easier to use.

Is there a roadmap for TUF deployment on PyPI somewhere?

[sigmavirus24]

@LucidOne this is not the appropriate place to ask

[LucidOne]

Thanks! I'll try and figure out the right place to ask.
Hopefully on topic, will twine maintain .asc upload support until TUF is actually deployed?

[sigmavirus24]

There's little reason not to. Twine supports more than PyPI and other archives may use the signatures so we're not going to remove that functionality yet.

[maltfield]

If the signature was uploaded to PyPI (as is the case with Twine, and distutils/setuptools) it is stored at the same URL as the file with a .asc appended to it. For example, using pip 7.1.2 the wheel file is located at https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl and the signature is located at https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl.asc. This is also linked from the UI if it's available.

To verify, you'd download both the file and the signature and do gpg --verify pip-7.1.2-py2.py3-none-any.whl.asc pip-7.1.2-py2.py3-none-any.whl. Sadly, there isn't a good way to know what key _should_ be signing releases. The best you can do is just a trust-on-first-use style mechanism where you require manual intervention if the key changes (which is what Debian typically does in their uscan program).

@dstufft how did you get that URL for the pip wheel? When I browse PyPI's website, the download links point to an object in s3. Appending .asc to this s3 URL does not work, and I cannot remove the file part of the URL to browse the directory structure.

How can I find which packages have .asc signatures and where I can download them?

[di]

@maltfield See https://warehouse.pypa.io/api-reference/integration-guide/#querying-pypi-for-package-urls