Git-secret: Git secret ignoring gpg issues with expired public key

[JulianStier]

Hey,
it makes sense if git-secret would provide a warning if gpg can not be used for encryption. This happens e.g. if the public key expired. Git secret then creates an entry in the mapping configuration for the file intended for encryption, but is not able to create a file with the _.secret_ extension. This leads to an error when performing _git secret reveal_ on another repository instance.

What are the steps to reproduce this issue?

1. Create a gpg key pair with short expiration (outdated expiration) for the public key; initialize git rep with git secret; add the identity with expired public key to git secret (tell)
2. Add a file with _git secret add_, commit, push
3. In a cloned repository, try to reveal the encrypted file

What happens?

_git secret_ gives no warning about problems with gpg

What were you expecting to happen?

A warning (at least the warning given by gpg) and no entry in the mapping file.

What versions of software are you using?

Operating system: (uname -a) Linux - 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

git-secret path: (which git-secret) /usr/bin/git-secret

git-secret version: (git secret --version) 0.2.4

git version: (git --version) 2.7.4

Shell type and version: ($SHELL --version) GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)

gpg version: (gpg --version) gpg (GnuPG) 1.4.20

[joshrabinowitz]

Hm, I looked at this briefly and it doesn't seem easy to even detect the expiration date/time of a particular key, which seems like it'd be a pre-requisite for this feature.

@JulianStier any tips on how we'd detect _when_ keys expire?

I was looking at keys that had no expiration date. I see we can get expiration dates from gpg --list-keys. If you wanted to submit a PR towards implementing this feature that would be great.

[joshrabinowitz]

The code should use gpg --list-keys --with-colons for this feature.

[adamgotterer]

Ran into a related problem today where someones key in the ring was expired. Spent over an hour trying to debug what happened before realizing it was an expired key. Basically hide was just exiting without any output. It would be great to know that a key was expired and that it couldn't en/decrypt.

[joshrabinowitz]

@adamgotterer @JulianStier @sobolevn @simbo1905 I started working up a fix better messaging for this here:
https://github.com/sobolevn/git-secret/compare/master...joshrabinowitz:expired-keys
https://github.com/sobolevn/git-secret/compare/master...joshrabinowitz:expired-keys2
Feel free to comment or contribute

[joshrabinowitz]

Also @JulianStier and @adamgotterer , I realize this issue is fixed in the current development version (aka master branch) of git-secret. git-secret will issue an error to let you know that the file could not be decrypted.

[joshrabinowitz]

@sobolevn @simbo1905 @adamgotterer @JulianStier

I think we should close this issue as the current version of git-secret gives an error message in this case like:

git-secret: abort: problem encrypting file with gpg: exit code 2: secret.txt

It would be nice to directly detect the expired key, but that's not particularly trivial (See currently unfinished PR #262, "code to check expiration of keys")

Let me know if anyone has any additional input about this issue. If not, I'll plan to close this issue in a week or so.
See also: #281

[joshrabinowitz]

git-secret no longer ignores errors caused by expired keys.
Closing as per comment above.

[joshrabinowitz]

See also: #290

[torbjornk]

I just spent a couple of hours due to this type of error. The error message that was considered "good enough" in this issue is basically not helpful at all. I had to dig quite deep into the source code to try to figure out the command git-secret uses to encrypt, which turned out to work just fine with myself as the only recipient. Then, I had to check out the code, modify it so that it actually outputs the gpg-command it tries to run, copy this gpg-command and run it, and then see the actual error message.

We have about 100 users in our git-secret repo, each with a gpg key that is valid for two years. This will happen quite often, and if we are supposed to use hours trying to figure this out at regular intervals, we might as well stop using git-secret.

Also, I'm a bit worried if the mentioned error message is what you consider "good enough". It is a worthless error message that provides nothing when it comes to usability. You are basically leaving your users to figuring this out themselves.

The very least you should do is implement a way for the user to see the actual error message from gpg. That would have saved me a couple of hours of digging. The way things are now, even the verbose-flag won't output anything useful.

[sobolevn]

@torbjornk sorry, that this happened to you. What error message would be helpful to you?

[joshrabinowitz]

@torbjornk thanks for your note about this, and I agree, apologies for the difficulty figuring this out.

Also note that in recent versions of git-secret, activating the SECRETS_VERBOSE env variable (or -v switch) should make git-secret output error messages from gnupg (by not passing --quiet to gnupg which will help to debug this). Did this not work for you in the master branch of git-secret?

I agree it would be nice to message about expired keys more clearly, though

[torbjornk]

Hi

Thank you for the quick reply :)

Seems I got a bit confused with versions and their output, but I managed to backtrack.

First, I had v0.2.4 installed, and got the following output:

~/UKE/project$ git secret hide
~/UKE/project$ git secret hide -v

So no output, no files encrypted.

Then, I upgraded to 0.2.5 (latest available with apt), and got the following output:

~/UKE/project$ git secret hide
git-secret: abort: problem encrypting file with gpg: exit code 2: secrets.yaml
~/UKE/project$ git secret hide -v
git-secret: abort: problem encrypting file with gpg: exit code 2: secrets.yaml

It gave an error message, but a general one, with nothing added by the -v-flag. After digging out the gpg-command being run, I got this error message:

~/UKE/project$ gpg [...]
[...]
gpg: error retrieving '[email protected]' via WKD: No data
gpg: [email protected]: skipped: No data
gpg: /home/torbjornk/UKE/project/secrets.yaml: encryption failed: No data

Still not a very good error message from gpg, but enough investigate further. But, I tried running gpg with -v, and the reponse was good enough to pinpoint the actual error:

~/UKE/project$ gpg -v [...]
[...]
gpg: Note: signature key SOMETHINGSOMETHING expired to. 30. mai 2019 kl. 11.11 +0200 CEST
gpg: error retrieving '[email protected]' via Local: No public key
gpg: error retrieving '[email protected]' via WKD: No data
gpg: [email protected]: skipped: No data
gpg: /home/torbjornk/UKE/project/secrets.yaml: encryption failed: No data

When retesting with the master branch (or by building v0.2.6), there seems to be an improvement:

~/UKE/project$ git secret hide
git-secret: abort: problem encrypting file with gpg: exit code 2: secrets.yaml
~/UKE/project$ git secret hide -v
gpg: error retrieving '[email protected]' via WKD: No data
gpg: [email protected]: skipped: No data
gpg: /home/torbjornk/UKE/project/secrets.yaml: encryption failed: No data
git-secret: abort: problem encrypting file with gpg: exit code 2: secrets.yaml

So with the latest version it seems that -v will output the same as the gpg-command (without -v).

So it seems to be fixed, as it outputs more than the error message mentioned higher up. It should be enough to at least start investigating what was wrong with the mentioned key. Hopefully the new version will become available in apt soon, so the rest of the project teams can get a better error message if it should happen again.

As a small suggestion, I hope you would consider something like this:

git secret hide: outputs the output from gpg (the same that git secret hide -v yields in v0.2.6). If there are no errors or warnings from gpg, there should be no output. If there are warnings (duplicate emails/keys for instance) you get a message about that. I seem to remember that this was the way it worked in the past, as I have seen this type of message before, without using the -v flag: gpg: [email protected]: skipped: public key already present.

git secret hide -v: outputs the output from gpg, with the -v flag set. Then, you can see what gpg is actually doing behind the scenes, and get useful debugging info in case of errors. Alternatively provide -vv flag that enables verbose output from gpg.

[joshrabinowitz]

Hello, I like your ideas @torbjornk . Please feel free to submit a PR!

[joshrabinowitz]

Closed by #553 and better error messaging in git-secret when gnupg returns error.