Conan: verify .sig for tarballs

[SSE4]

many tarballs come with .sig files (especially, for GNU), it might be worth to improve tools.download helper to optionally verify downloaded tarballs, if possible

[memsharded]

I don't know much about those files, PGP signatures? how to verify them?
Sounds like it makes sense.

[SSE4]

correct, they are GNU GPG signatures.
the following gist demonstrate how to do it (although, it requires additional modules)
https://gist.github.com/lkdocs/6519372

[hrantzsch]

I would like to help advance development on this issue, as validating the authenticity of package sources is crucial for my company's use case. Let me try to summarize what we think needs to be done here.

As an example, consider we want to build the crypto library Botan. On the Botan release page a .tar.xz and a .tar.xz.asc are provided for each version. Manually, we could check the authenticity of our downloaded archive using:

# download version archive and signature file
wget https://botan.randombit.net/releases/Botan-2.12.1.tar.xz{.asc,}
# import Botan Distribution Key public key
# note that the key or its fingerprint will need to be provided in the recipe
gpg --receive-keys 621DAF6411E1851C4CF9A2E16211EBF1EFBADFBC
# verify the signature
gpg --verify Botan-2.12.1.tar.xz{.asc,}

This is basically what we would like Conan to do.

What do we get from this?

Validating the signature provided allows us to be sure that the archive we downloaded is authentic, before we extract and build it. This assumes that the key is not compromised.

We still need to trust the recipe, in which both the download URLs and the key (or its fingerprint) are specified. We also don't check the authenticity of downloaded binaries (see #480) with this.

How to implement this?

AFAIK, two ways of implementing this have been discussed in other issues and PRs already:

(A) Implement this functionality in tools.get()

Package sources are downloaded and extracted in a conanfile's source() method, which usually makes use of tools.get() for this. get is already able to verify the integrity of the downloaded source archive by checking user-provided hashes. I think that checking the archive's signature is conceptually very similar.

This is the approach that is already implemented to a large extend in the PR referenced above (#2356).
It has been discussed which new dependencies this would introduce in Conan. The python-gnupg package used by @weatherhead99 is MIT compatible and does not introduce further dependencies.
Obviously, gpg must also be available.

The PR also already took into consideration how the public key should be retrieved and stored. In the implementation, a dedicated GnuPG home directory is used in order not to interfere with the user's system keyring. The public key is provided either as a hardcoded string, as a file, or as a fingerprint which can be used to gpg --recv-keys.

(B) Implement as a Conan-Hook

@jgsogo suggested the functionality could be implemented as a hook. I like this idea very much, as I see the advantage of not introducing new dependencies and complexity to Conan. It would also be quite easy to reuse the code in the existing PR to implement the hook.

So I went ahead and tried to implement the hook. Unfortunately I didn't get very far, as there is no appropriate hook function available. There's post_source, which sounds nice, but cannot be used because source will not only download the archive, it will also extract it and delete it.

We don't want to extract the archive before we have validated that we can trust the archive we downloaded. And of course, if the archive is deleted we cannot check the signature anymore.

In addition, packages could require more than one resource to be downloaded and verified in the source step.

I looked into implementing a new hook function in tools.get. But tools don't know anything about hooks or the hook_manager and I guess we want to keep it like that.

---

In my opinion this functionality should be part of the get, which already validates hashes of downloaded archives. The hook implementation sounds really nice, but as I've described, it's not a good fit here.
Furthermore, I believe that validating source signatures should be a "native" functionality of Conan. Most, if not all, major libraries provide signature files along with their releases. We should use them.

Please let me know what you think and how we should continue.

[Tymolc]

What is the state of this? Our company would also very much welcome this feature. Making sure that an archive is actually trustworthy does feel like a vital functionality of a package manager.