git-secret-hide fails when encrypting with a revoked and active key for the same email, so help user avoid this situation

[Cantido]

What are the steps to reproduce this issue?

1. Create a new git repository, with a file you'd like to encrypt
2. git secret tell a key with multiple identities
3. git secret hide

What happens?

git-secret aborts.

What were you expecting to happen?

The hide to succeed.

Any logs, error output, etc?

$ git secret hide
gpg: error retrieving '[email protected]' via WKD: No data
gpg: [email protected]: skipped: No data
gpg: /path/to/example/secret.txt: encryption failed: No data
git-secret: abort: problem encrypting file with gpg: exit code 2: secret.txt

Any other comments?

I verified that another key works just fine.

This is they key I'm trying to git secret tell to: DD2C 4F19 5E70 A0D9 2F86 2D6F 60BA 2A8B FD58 3641.

There are several revoked identities on that key, and the error message says the error is in retrieving one of the revoked identities. I used one of the valid identities when I added the key via git secret tell. All identities, including the revoked ones, are listed by git secret whoknows.

What versions of software are you using?

Operating system: (uname -a)
Darwin hostname 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

git-secret path: (which git-secret)
/usr/local/bin/git-secret

git-secret version: (git secret --version)
0.3.1

git version: (git --version)
git version 2.23.0

Shell type and version: ($SHELL --version)
zsh 5.3 (x86_64-apple-darwin18.0)

gpg version: (gpg --version)

gpg (GnuPG) 2.2.17
libgcrypt 1.8.5
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: /Users/user/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

[joshrabinowitz]

Hello, @Cantido ,

This is a very interesting bug report.

Is it the case that all of the identities on that key except one has been revoked?

This seems related to #508 and https://github.com/sobolevn/git-secret/issues/491#issuecomment-505492667

[Cantido]

Hi @joshrabinowitz!

There are four active identities, and I made sure the primary ID is one of the active ones. git secret tell was run on one of the active IDs.

[joshrabinowitz]

So if a key has a primary ID, non-primary IDs, and revoked IDs, you'd like git-secret to prefer the primary, non revoked identity?

If not, can you please detail how you think this should work?

[Cantido]

I was assuming that it would use the ID with the email I gave to git secret tell. Using the primary ID also sounds reasonable.

[joshrabinowitz]

I looked at the data you exported and I think I see what's going on: your keyring has at least one case of having a revoked key and an active key for the same email address.

Since git-secret does not currently consider if keys are revoked or not (see #508 again) it can try to use a revoked key, which I think is causing your issue.

The solution, I believe is to re-implement #492, which was reverted when we previously decided to close #508 as 'wontfix'. (Edit: this was performed in PR #553)

I'm reopening #508 because I think it will resolve this issue.

[joshrabinowitz]

@Cantido , can you please test this issue using PR #553 and let us know what error you encounter, or if it fixes your problem?

[Cantido]

This is the error I run into:

./git-secret/git-secret tell <current email>
./git-secret/git-secret hide
gpg: <current email>: skipped: public key already present
gpg: error retrieving '<revoked email>' via WKD: No data
gpg: <revoked email>: skipped: No data
gpg: /Users/rosa/workspace/example/secret.txt: encryption failed: No data
git-secret: abort: problem encrypting file with gpg: exit code 2: secret.txt

[joshrabinowitz]

@cantido thanks for the reply.

I cannot replicate your issue without more information, but I _think_ what is happening now is that your git-secret keychain in repo/.gitsecret/keys still contains the revoked key.

To fix this current situation, use git secret killperson <revoked email>, make sure the revoked email doesn't exist in git secret whoknows.

Then see what you get with git secret hide -v and git secret reveal -v and let us know what you get.

Also I should have mentioned this before, but if you could please use -v with git-secret commands that support it (or export SECRETS_VERBOSE=1), it would give us more data on how to debug this.

Again, can you let us know if the above steps (removing the revoked key from your repo's keychain, and adding the correct key) fixes the issue, and if not, please re-post more data so we can help debug this?

[joshrabinowitz]

This line is weird, too:
gpg: <current email>: skipped: public key already present
Can you please also start by killperson-ing that key too?

[Cantido]

I created an empty directory, initialized it for git and git-secret, and created a text file. Adding myself and running hide resulted in that error message.

Something interesting I noticed was after telling myself, my primary email address appears twice. Then after killperson-ing myself, one instance of my primary email was still in the whoknows list, but all the other identities were removed. That happened even though I ran killperson on my primary email.

[joshrabinowitz]

git-secret is not really equipped to deal with user's keychains which have multiple keys for the same email address, as it only uses email addresses to describe keys. This is closely related to #268 , and requires the same fix - the ability to address keys by fingerprint or ID, and not just email.

To resolve this issue with git-secret as it is, you have to ensure that there is only one, valid, key in your keyring for each email you want to use with git-secret, and then make sure you only have the right, valid keys in your repo's keychain.

[Cantido]

I understand! The email address I'm using for all my operations isn't duplicated, but it's the other duplicate one that's causing all the trouble, since I'm technically telling all of my IDs.

I appreciate the work you put into this, thank you!

[joshrabinowitz]

duplicate of #268

[joshrabinowitz]

@cantido, does the current code in #553 give better error messages for your case?

[Cantido]

I got the same error message:

$ ./git-secret/git-secret hide
gpg: <current email>: skipped: public key already present
gpg: error retrieving '<revoked email>' via WKD: No data
gpg: <revoked email>: skipped: No data
gpg: /Users/rosa/workspace/git-secret-test/secret.txt: encryption failed: No data
git-secret: abort: problem encrypting file with gpg: exit code 2: secret.txt

[joshrabinowitz]

@Cantido yes, sorry I wasn't clearer-- if you replicate these steps again:

I created an empty directory, initialized it for git and git-secret, and created a text file. Adding myself and running hide resulted in that error message.

Then I think you'll get different warnings and errors during the tell phase.

Edit: We can't really automatically fix the issue where you're hiding files and your repo's keychain _already_ contains duplicate/revoked/non-working keys, but we can stop it from occuring in tell.

[Cantido]

I gotcha, I forgot to git checkout the branch that had your change. It looks like I got the message you were expecting:

$ ./git-secret/git-secret tell <current email>
git-secret: warning: at least one key for email(s) is revoked, expired, or otherwise invalid: <old revoked email>

But I'm a little confused. My key was never revoked, only the identity was revoked.

[joshrabinowitz]

Hm, git-secret doesn't understand such concepts yet.

To git-secret, an 'identity' is an email address like [email protected] and the 'key' is what we get from gpg --export --armor [email protected]. See: https://github.com/sobolevn/git-secret/blob/master/src/commands/git_secret_tell.sh#L82

In the context of the keys you uploaded at https://keybase.io/cantido/pgp_keys.asc, can you detail what gnupg commands git-secret should be using to export the correct keys for git-secret to use?

[Cantido]

That's how I understand identities vs keys as well, but your last sentence is where I'm lost. That key is a master key with three subkeys, but none of those keys were revoked, so I'm not sure where that error message is coming from. Identity revocation is a separate process from key revocation, unless I'm misunderstanding something.

[joshrabinowitz]

Gotcha, we're getting close... Take a look at this line of code
https://github.com/sobolevn/git-secret/blob/c5b815c7db1a18fa20699260aa1fa0cf9126188e/src/commands/git_secret_tell.sh#L82
where we use the _email address_ to export key data into $keyfile , and then do the equiv of this:

gpg --homedir "$secrets_dir_keys" --no-permission-warning --import "$keyfile"

Here's the question:

• What should gpg --export -a [email protected] put in place of '[email protected]' (and what else might be needed) to make sure it exports the correct _public key data_ for use in the repo's git-secret keychain, and which won't cause gnupg encryption (which hide uses) to complain about the revocation?

One might need to specify the correct gnupg 'user id' using gnupg 'fingerprints' or 'key ids' instead of using an email address, or do something different than that. (To be clear, git-secret itself does not yet support use of fingerprints or key ids, I'm just figuring out what the right thing to do here is.)

[Cantido]

I think gpg --export -a "$email" is returning a correct public key, unless I'm misunderstanding what "correct" means here; it still contains the revoked identities. When I normally encrypt things using this key, I don't get any error messages or even warnings about the revocation.

[joshrabinowitz]

Re: what is a 'key' and an 'identity':

• My understanding is that each public/private keypair constitutes what we are calling a "key" (technically a pair of pub/priv keys).
• I can't find a clear definition of an 'identity' in gnupg, but I think it's basically your email address (the thing you used to access the key(s)). The closest I could find to a discussion of 'identity' in the gnupg docs is this: https://www.gnupg.org/documentation/manuals/gnupg/Specify-a-User-ID.html (let me know if you see better docs for gnupg identities).

Therefore if a user has a single keypair associated with their email, revokes it, and adds a new keypair for that email to their keychain (without removing the revoked pair), I would consider that "two key pairs" associated with one email address (identity).

What would make this much easier to debug is if we had a publicly available set of gnupg keys (both public and private parts, revoked and unrevoked, and each for the same email address, with the relevant parts recorded on https://keybase.io) that matched this issue's use case (including the revoked and unrevoked keys). Otherwise it's difficult to replicate the scenario.

@cantido could you possibly create a set of test keys that match the above criteria (including relevant parts on https://keybase.io) and make all portions available to us ? Then we could add a test case that exercises this exact issue and we could talk about the issue more exactly.

[joshrabinowitz]

Closed by #553