Sops: Git diff does not work with path_regex

[benjefferies]

Description

When using git diff with path_regex it fails to match any paths. This appears to be because git diff will create a temp file (for textconv?). Sops will then try to decrypt that file using the .sops.yaml configuration. The temp file does not match the path_regex so it fails.

I've hacked the sops source to get some debugging to see what's going on and this shows the events

[CMD]    time="2020-04-27T13:21:53+01:00" level=info msg="CLI args are: [/var/folders/2k/6q86kgm12sq9rxxvqp7d0c5c0000gn/T//PeMph9_etcd-browser-aws.yaml]"
[CMD]    time="2020-04-27T13:21:53+01:00" level=info msg="Loading file /var/folders/2k/6q86kgm12sq9rxxvqp7d0c5c0000gn/T/PeMph9_etcd-browser-aws.yaml"
[CMD]    time="2020-04-27T13:21:53+01:00" level=info msg="Loading config .sops.yaml"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Loaded config file"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Parsing rules!!!"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Parsing rule - secrets/dev/.*"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Does secrets/dev/.* match /var/folders/2k/6q86kgm12sq9rxxvqp7d0c5c0000gn/T/PeMph9_etcd-browser-aws.yaml"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Parsing rule - secrets/test/.*"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Does secrets/test/.* match /var/folders/2k/6q86kgm12sq9rxxvqp7d0c5c0000gn/T/PeMph9_etcd-browser-aws.yaml"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Parsing rule - secrets/prod/.*"
[CONFIG]         time="2020-04-27T13:21:53+01:00" level=info msg="Does secrets/prod/.* match /var/folders/2k/6q86kgm12sq9rxxvqp7d0c5c0000gn/T/PeMph9_etcd-browser-aws.yaml"
[CMD]    time="2020-04-27T13:21:53+01:00" level=info msg="parsed config"
[CMD]    time="2020-04-27T13:21:53+01:00" level=info msg="Got error parsing config"
error loading config: no matching creation rules found
fatal: unable to read files to diff

[autrilla]

That file (PeMph9_etcd-browser-aws.yaml) is encrypted, right? If so, SOPS should not care about creation rules at all. I have a feeling this might have been fixed by #662.

[benjefferies]

Good point it's encrypted so it should have all it's metadata. I'll build master and test the fix you suggest

[benjefferies]

I've rebuilt the binary on develop branch and it's still an issue

[autrilla]

I managed to reproduce, and this is definitely a bug. #667 is a very naive attempt at fixing it, and indeed it should for your case. But I imagine we made not finding a matching creation rule an error for a reason.

[mlushpenko]

Greetings, any progress on this? If not, do you know if I could put .sops file in subfolders per environment vs root folder and then no path_regex would be needed?

[autrilla]

Greetings, any progress on this? If not, do you know if I could put .sops file in subfolders per environment vs root folder and then no path_regex would be needed?

No progress. SOPS will look for the .sops.yaml file in the current working directory, so you'd have to run git diff for each directory, but that sould work.