Git-secret: Feature request: "git secret hide" should only touch modified files

[kingpong]

It would be nice if git secret hide would only re-encrypt files if the underlying plaintext file has changed.

What are the steps to reproduce this issue?

1. git secret hide
2. git commit -am "save secrets"
3. git secret hide
4. git status

What happens?

git status shows that git-secret-hide has rewritten all of my secret files, plus the random_seed file.

What were you expecting to happen?

Ideally, the git secret hide in step 3 would have been a no-op since none of the underlying plaintext has changed.

What versions of software are you using?

Operating system: Mac OS X 10.11.6 (El Capitan)

git-secret version: 0.2.2

Shell type and version: zsh 5.0.8 (x86_64-apple-darwin15.0)

[sobolevn]

Thanks! That's a good idea indeed.

Do you have any free time to make a PR for this?

[kingpong]

@sobolevn Yes, I might have time. I'll need to get acquainted with the code first, though. I'll try to give it a whirl this week.

[kingpong]

(sorry for the long message -- just want to get this right)

There are a couple of tricky parts to this:

Problem 1
Due to the async nature of gpg, the person running git secret hide may not have access to any of the private keys capable of decrypting the original ciphertext (to know if it has changed). When this is the case, I think we can fall back to the existing behavior of just overwriting all the ciphertext files. Unfortunately, since we need access to the private key, this means the hide command would need to accept the passphrase and homedir parameters like reveal does.

Problem 2
The user may want to rewrite the ciphertext files even if the plaintext hasn't changed. One example of this is when adding (tell-ing) a new public key. I believe we can solve this by comparing the current pub key list to the previous list. We can generate the list using gpg --list-packets like so:

$ gpg --homedir=$PWD/.gitsecret/keys --list-packets file_to_hide.secret 2>/dev/null | grep ^:pubkey
:pubkey enc packet: version 3, algo 1, keyid 644EBD411BFB8CE9
:pubkey enc packet: version 3, algo 1, keyid 4D60BA6295AB8A33

So, the approach will be:

1. Encrypt the plaintext and store the ciphertext in a temp file.
2. Decrypt the original ciphertext (if it exists).
3. Extract the list of public keys from the original ciphertext (if it exists).
4. If old(plaintext,pubkeys) == new(plaintext,pubkeys), delete the temp file and continue.
5. Replace the ciphertext with the temp file.

If we run into trouble decrypting the ciphertext or listing keys, we fall back to the current behavior, to replace all ciphertext.

@sobolevn Please let me know if you're ok with my plans for solving these problems.

[sobolevn]

I think that this is a nice way to solve it.

Just one thing to mention. This change is quite big. And it can break stuff people are already using. I'm thinking about releasing it with 0.3.0, not the 0.2.3

[skywinder]

Looks like git secret hide touches .secret files. So my pre-commit hook initiates reencryption of .sectret files each time.

[skywinder]

can you please check it and probably reopen this issue?
I can give you examples of the files if you need them to reproduce this bug.

It should be solved in #107 by @hurricanehrndz

[joshrabinowitz]

Hello @skywinder,

If you use the -m flag with git-secret you will get the behavior you're talking about (where only changed files get re-encrypted). However, as the git-secret hide man pages say:

   It is recommended to encrypt (or re-encrypt) all the files in a git-secret repo each time  git
   secret hide is run.

   Otherwise  the  keychain (the one stored in .gitsecret/keys/*.gpg), may have changed since the
   last time the files were encrypted, and it's possible to create a state where the users in the
   output of git secret whoknows may not be able to decrypt only some files in the repo, or may be
   able decrypt files they're not supposed to be able to.

   In other words, unless you re-encrypt all the files in a repo each time you 'hide'  any,  it's
   possible to make it so some files can no longer be decrypted by users who should be (and would
   appear) able to decrypt them, and vice-versa.

   If you know what you are doing and wish to encrypt or re-encrypt only a subset  of  the  files
   even  after  reading the above paragraphs, you can use the -F or -m option to only encrypted a
   subset of files. The -F option forces git secret hide to skip any hidden files where the unen-
   crypted  versions  aren't present. The -m option skips any hidden files that have not be modi-
   fied since the last time they were encrypted.

If there's a discrepancy between the docs of git-secret and what it does, or if you'd like it to behave differently than it does, please open a new ticket for the issue.

[skywinder]

Gotcha, thanks. In this case, I just want to add this to the manual to the section, there you talking about pre-commit hook.

So mine looks like this now:

> .git/hooks/pre-commit

git secret hide -m
git add .gitsecret/paths/mapping.cfg
git add *.secret

I can add this example to https://git-secret.io/ , but can't find is there open source? gh-pages looks different.

Otherwise I can add this section to readme if you want to.

[Mobe91]

This should definitely be added as pre-commit hook example to the docs!

[joshrabinowitz]

@Mobe91 if you submit a PR we will consider!

[Mobe91]

@joshrabinowitz is it sufficient to add the changes to man/man7/*?

[joshrabinowitz]

@Mobe91 , sure, you can add them to man/man7/git-secret.7.ronn

the man page is auto-generated from that file