sops/3.6.0 encrypted yaml file is broken (even to sops)

[icy]

I have the following yaml file:

## Configuration for prometheus-node-exporter subchart
##
prometheus-node-exporter:
  podLabels:
    ## Add the 'node-exporter' label to be used by serviceMonitor to match standard common usage in rules and grafana dashboards
    ##

    jobLabel: node-exporter
  extraArgs:
    - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
    - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$

When it's encrypted with sops, I get the output as below

prometheus-node-exporter:
## Add the 'node-exporter' label to be used by serviceMonitor to match standard common usage in rules and grafana dashboards
##  podLabels:
        jobLabel: node-exporter
    extraArgs:
    - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
    - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$
sops:
    kms:
    -   arn: arn:aws:kms:eu-central-1:0000000:key/75f9aedd-ce9e-4b85-b112-00000000
        created_at: '2020-07-16T17:43:07Z'
        enc: AQICAHgquprv1l5xoZPFQCgP7aJyPm2SG/VQw5mKqJ1aWMOo1wGMTOvYqCZquWh0DzogTo7uAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMgkcK1GZYgX5WXfFJAgEQgDsRmTsCzr7vNGSLvP0sZ7zGQcXqg+5pEHNOp3H7jjVdyUAFY/LowjnEmWzWGdQCku4hljPHv/N3MXrjMw==
        aws_profile: ""
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    lastmodified: '2020-07-16T17:43:08Z'
    mac: ENC[AES256_GCM,data:N5EwksB9NZFd3Z1hLREp6lgOIc3K00XO6K54xQ7dDnYiywzozXvXFV1/5adZW4a3XaTfP3UkyzCFkWkfPesIJ1roLwjcYmwBzDke5wsl4MW2jNy8/M9B8u8sCmj6MpZY149gS6RV+8gETBKa9P9phGNAW4GLT+4DjXpBUIuXDiY=,iv:aCQQ7kjZBosxQipF68UiymJtvlsGWz6kCZX7Q/WiPEU=,tag:cMuYOyQajwV3kEMWtahzzA==,type:str]
    pgp: []
    encrypted_regex: (newrelic|certificate|private|pass|key|value)|(api_url)
    version: 3.6.0

This output is a broken yaml file. sops can't even decrypt it:

$ sops -d foobar.yaml
Error unmarshaling input YAML: yaml: line 4: did not find expected key

$ ryaml < foobar.yaml
Traceback (most recent call last):
        3: from -e:2:in `<main>'
        2: from /usr/lib/ruby/2.7.0/psych.rb:565:in `load_stream'
        1: from /usr/lib/ruby/2.7.0/psych.rb:459:in `parse_stream'
/usr/lib/ruby/2.7.0/psych.rb:459:in `parse': (<unknown>): did not find expected key while parsing a block mapping at line 1 column 1 (Psych::SyntaxError)

The first comment in my original file is gone. I noticed these lines caused issue.

    ##

    jobLabel: node-exporter

[ajvb]

@icy How did you get that output? As that does not look properly encrypted. If I put your original yaml into a file and run sops -i -e file.yaml and then decrypt it, it's working alright.

There does seem to be a different problem I'm seeing where the comments are not being unencrypted.

[icy]

@ajvb I actually used sops -i -e file.yaml, or sops --output tmp.yaml file.yaml. Same problem. You got it right: sops generated wrong output: that's the whole point of this report ;)

(I made the kms key arn obfuscate btw)

[icy]

Due to this issue, I have to develop a small script https://gist.github.com/icy/d567dcf6e767d22e3683eb075a442967 , which tries to encrypt my file, decrypt the output, and if the decryption step fails, the script reports the issue.

This script works with most of my files, except the file that I reported in this thread.

[ajvb]

@icy I'm sorry but I'm having trouble reproducing your issue.

Got this yaml file as test.yaml

$ cat test.yaml 
## Configuration for prometheus-node-exporter subchart
##
prometheus-node-exporter:
  podLabels:
    ## Add the 'node-exporter' label to be used by serviceMonitor to match standard common usage in rules and grafana dashboards
    ##

    jobLabel: node-exporter
  extraArgs:
    - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
    - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$

I encrypt it in-place:

$ sops --pgp F69E4901EDBAD2D1753F8C67A64535C4163FB307 -e -i test.yaml     
$ cat test.yaml 
prometheus-node-exporter:
#ENC[AES256_GCM,data:dhEylAf1LKZu3HRYi5CM7vs/DVKmMVPh1aoJbmrXZ62wRAgnVrzQD3WtL2o5tcg5+pv5b6RHrd5/3fzSVv+GTGxYRZd3ppnv1lYvwsRy9jMQYCoWCpqf6q4AIQqDOF8qTDPc8vQOdwzdm6Rz6dLxB8WOLuvghaJWvwPT,iv:KKA5n7bHqJLtlvfw2/LKAgvKZeofcWKuK+7QZhREC80=,tag:DlKy3ESiCQ4/9KwdJggi6Q==,type:comment]
#ENC[AES256_GCM,data:PA==,iv:mpzTzmzLaYgihjOiHY0J8fOI1VAe7HrdruynAgiRSMk=,tag:4dV1gejqFBhGJT/2yy35OQ==,type:comment]
    podLabels:
        jobLabel: ENC[AES256_GCM,data:Wm2Su1kun78h5nmhTQ==,iv:AEXBKXWhg3EQXphoOtXdzxkjq1joih6LGmpi9FCzNus=,tag:UgsVcTY7W2L+2cz17LCMEA==,type:str]
    extraArgs:
    - ENC[AES256_GCM,data:TxEHX7lWSby8JWna2SZFiDYiiKsWAe7gl/Sfv0khNiHsShoTRKFLthwaaP4fs4U7b/RCgCF9HtMELTYxE22Pmgpz37t9LCRl/4NeaqNH+CLfkJ4=,iv:SiNlJLvSgkiKmxksuquG+QgTVn0oyth8WbHiDsP1hV4=,tag:cF2cf1uDqgP5D8Ym/vP84w==,type:str]
    - ENC[AES256_GCM,data:ppmtC1+a1+BRR3T6cGh+a8rzGjpE2wPSzqRL7GpWta1fQMN3mGYzAPUNhr709qtx7VPs0cxvEP/+FUmvck7vpUJ/dD6R+BdWrJfxQX4tTEG9JcDnYLMSzP8OBN6jEgKlCeYFgLP0v6Z1p2s0FfxzLLBskp7qBFBbBli0gX+3wa6T0VUOsY3OQqVfuXGdLpod8CWqTuMMl/A9pmNXt7ZlWUMtGxySj7aCh0RUQZ599hQKljYwYl+QwUpgV1B/Dw==,iv:mXoC5Cw0lFA3Uu95hPaJ/j7Rzxyc6Q3ujOt+EPqkRWY=,tag:kiC+n0YmmpZcLrzbRnLdzw==,type:str]
sops:
    kms: []
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    lastmodified: '2020-07-16T19:04:31Z'
    mac: ENC[AES256_GCM,data:cleEXM6LJ7d6L/UJAPSXNQ+kV0nwUbTmD92GCrCTF4x0/090l5dQ7BcDnbk/3+xyNKkFWV6+HMOSOh873eeq0VyhEuyT/UM5qw1blIddQWA3xUevZ6D3VL0qmANjHDFCIuWNnB48KJwlYhwpxykb3qSwRW+2tpXDTgtJ165mR7s=,iv:kHv5qeEW+IbVJsbAFvg+2x/E+ICo17GNPLZoUqRJroo=,tag:dU+3ifU98F0pbDLeOALfjg==,type:str]
    pgp:
    -   created_at: '2020-07-16T19:04:31Z'
        enc: |-
            -----BEGIN PGP MESSAGE-----

            wcBMAze5s52/VnsCAQgAsifO6J5+w9OulJ3ocu3WKdEFl5//0X0dVHtTeW5YkzZ3
            ncy3TJHIi2hATMrzMe9IcplS7FQI2iioOoaTesQaOZh31NWa4ZnlDMtONzO16iz/
            JJAn0qYR81RqWEvYyOmktq+TO0htgm5etqaxO/myHl2IfaWlTvtIZ5x6n3Aw8x37
            S6DDWpIl5/E5oJDx8bQK+eYQe5qHiuD6rhmIUVAikptO+OwO/1PoUoUJCcYkHpGV
            s555HRJY1Vm3Qd3GE0C7aP1oWKqbzDto3RwIUIbqMFr181pdD2S7kZPdTePwdrNg
            r/apsrCV3PSSLWl24JiKfETzP1DJl0ZkhPseu/XCzdLgAeT2Dv8/NAJ9JkxvF+5y
            k9iH4Qwl4Irg++EqceAx4p1NGHjgS+UEEWtNGl6zTgNxmUvfGq4ILoqiUOuR73Uv
            wEdPgFZhmOAl5M60D5rzaJw/Glk6+ttcBpTiWpMi7eG4mAA=
            =6LTI
            -----END PGP MESSAGE-----
        fp: F69E4901EDBAD2D1753F8C67A64535C4163FB307
    unencrypted_suffix: _unencrypted
    version: 3.6.0

Then I can decrypt it just fine:

$ sops -d -i test.yaml 
[SOPS]   WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS.  comment="ENC[AES256_GCM,data:dhEylAf1LKZu3HRYi5CM7vs/DVKmMVPh1aoJbmrXZ62wRAgnVrzQD3WtL2o5tcg5+pv5b6RHrd5/3fzSVv+GTGxYRZd3ppnv1lYvwsRy9jMQYCoWCpqf6q4AIQqDOF8qTDPc8vQOdwzdm6Rz6dLxB8WOLuvghaJWvwPT,iv:KKA5n7bHqJLtlvfw2/LKAgvKZeofcWKuK+7QZhREC80=,tag:DlKy3ESiCQ4/9KwdJggi6Q==,type:comment]"
[SOPS]   WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS.  comment="ENC[AES256_GCM,data:PA==,iv:mpzTzmzLaYgihjOiHY0J8fOI1VAe7HrdruynAgiRSMk=,tag:4dV1gejqFBhGJT/2yy35OQ==,type:comment]"

$ cat test.yaml 
#ENC[AES256_GCM,data:dhEylAf1LKZu3HRYi5CM7vs/DVKmMVPh1aoJbmrXZ62wRAgnVrzQD3WtL2o5tcg5+pv5b6RHrd5/3fzSVv+GTGxYRZd3ppnv1lYvwsRy9jMQYCoWCpqf6q4AIQqDOF8qTDPc8vQOdwzdm6Rz6dLxB8WOLuvghaJWvwPT,iv:KKA5n7bHqJLtlvfw2/LKAgvKZeofcWKuK+7QZhREC80=,tag:DlKy3ESiCQ4/9KwdJggi6Q==,type:comment]
#ENC[AES256_GCM,data:PA==,iv:mpzTzmzLaYgihjOiHY0J8fOI1VAe7HrdruynAgiRSMk=,tag:4dV1gejqFBhGJT/2yy35OQ==,type:comment]
prometheus-node-exporter:
    podLabels:
        jobLabel: node-exporter
    extraArgs:
    - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
    - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$

The "problem" I mentioned is that the comments are not currently being decrypted, but that is valid yaml being outputted.

[icy]

I'm sorry but I'm having trouble reproducing your issue.

Thanks for your support. That's interesting. I'm using sops downloaded here https://github.com/mozilla/sops/releases/download/v3.6.0/sops-v3.6.0.linux . Are you using the same file?

How can I provide more details of the problem on my laptop?

[ajvb]

@icy I just tested it again with https://github.com/mozilla/sops/releases/download/v3.6.0/sops-v3.6.0.linux and got the same results.

My first thoughts are:

1. Does anything different happen if you remove the comments entirely from the original plaintext file?
2. Have you double-checked that there isn't any encoding problem in the original file? If you copy/paste the yaml contents you posted in the original comment into a file and encrypt that, does it work?

[icy]

@ajvb I've pushed my files here https://github.com/icy/sops-695/tree/master (Somehow I couldn't push to my gists).

diff bad/fooobar.yaml good/fooobar.yaml 
6d5
<     ##

[icy]

@icy I just tested it again with https://github.com/mozilla/sops/releases/download/v3.6.0/sops-v3.6.0.linux and got the same results.

1. Does anything different happen if you remove the comments entirely from the original plaintext file?

See the diff above. I provide both bad and good yaml file in my repo.

2. Have you double-checked that there isn't any encoding problem in the original file? If you copy/paste the yaml contents you posted in the original comment into a file and encrypt that, does it work?

The yaml files in my repo are ASCII text (output from file command.) So I don't think that is an encoding issue.

[icy]

@ajvb I'd go to bed now. If you need any further information I can provide tomorrow. Thanks for your time.

[autrilla]

My guess is it's the double # that might be confusing SOPS.

Works for me with 3.5.0:

/tmp/sops-695 master
❯ cat bad/fooobar.yaml
## Configuration for prometheus-node-exporter subchart
##
prometheus-node-exporter:
  podLabels:
    ## Add the 'node-exporter' label to be used by serviceMonitor to match standard common usage in rules and grafana dashboards
    ##

    jobLabel: node-exporter
  extraArgs:
    - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
    - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$

/tmp/sops-695 master
❯ sops --pgp FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4 -e bad/fooobar.yaml > test.yaml

/tmp/sops-695 master*
❯ cat test.yaml
prometheus-node-exporter:
#ENC[AES256_GCM,data:ZqzU5N3/amKUfCoPXbbM2JJ4Lpi/zJ+1UmrCurZ8hJAFGCf1QZUU93I0IPI3JwIXvwDZkrZF9KBbRhBtQT+3eVv1o7TCkY/rq90nkFflzo7AGeTf8L0YpuP5Ryv5Ha/R46Zr5X4hf3chjlONUdxaioX+ao7Chhn/Vth7,iv:JwkbA4HlGflHkQILpYgODN8zjFDoNHISuXyUycQ7aLg=,tag:NEjFWZ3frOh85z8vV7THUQ==,type:comment]
#ENC[AES256_GCM,data:XA==,iv:PDDdDuyRzk/TeMPuy88bD4MIB6RFiTD9t1VSA7PWLFM=,tag:KULnokdDNGvGoBecZNbmvA==,type:comment]
    podLabels:
        jobLabel: ENC[AES256_GCM,data:AJ/VvSBK0tBda96fyA==,iv:LHOn9QesPTRb2zyW4mXwITgXHfq5hadcnAv2mTkZCGA=,tag:+wNPHoEVAnFSRNuZVDWxHw==,type:str]
    extraArgs:
    - ENC[AES256_GCM,data:ZHFNHtuQVP5kPjD5miGqBymsTAfk/IVz36eLuYNg2FWCRB8KgmtBbx4UCaec9us6Dr27HW+JV6TMs+rJFDawHNrsagdHcm6Ruf079xgTjwDaDg8=,iv:k7dPw0LXkV3ebMcQrZV0No+TlE/caPlaHvPitiIqewk=,tag:wSclNXQVw0e+vvBU2Cz34g==,type:str]
    - ENC[AES256_GCM,data:meAFAXt94Sd/lPCI2JBi+j6P6oheEbbPjlYd/jzXDp9Ao1ZIo21VKuw5yOS6dkrWWv1BTuUaW2GROBjA6G7QZ3Rp7Ity4v+KIKt5i6YxtZfTwoLpqYx5ANdJMAoGW2dODUm/ZhgCJOweGcp9WBx5krLe+KdxmJnMLOlmD0SUrVTBiv+DX50aTd98mHIYbi4kqgAXH8xek3/MeescfU1bGOLDkpQLrXqNUH776LQ1dZG+cfv3+fglKtZb2lfGQg==,iv:PDM3VBVMILGX0SLAa5mwTzrC59FjSh45GUgUOlO6udI=,tag:/f6cDI/bHv1RFika1xDLtg==,type:str]
sops:
    kms: []
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    lastmodified: '2020-07-16T20:45:59Z'
    mac: ENC[AES256_GCM,data:Osy+3yi8qbhIAxxcCrKobON9K57iJhu8PL3hQFRAGVtGCvWjExQC/dNYnfoqcDlzjvXxOPQPVUeeSmh14b4PaSLuHtHcsqzbKTuHAoD/+nNO8H6XGPTK1sBoXSYV2h6IC1aXJ2lj2MkIW2JYLZZX4HkSKkJKv8nVwEUXMugau2w=,iv:MNt3Zuql9v0DvZ0LMcw4Gopcb0w7aAepLYIwgGHdS7I=,tag:NYK+FSu3KojvF25IkUCvsQ==,type:str]
    pgp:
    -   created_at: '2020-07-16T20:45:58Z'
        enc: |-
            -----BEGIN PGP MESSAGE-----

            wcBMAyUpShfNkFB/AQgAB4945xTWOtNoyXxWnqiAjJZvNGfKOBlK6eXpteoIjpsi
            MlC1230if8dPJLe5EatygCvC3MMuO0soCp7d3DTNRMKpFIghMzY3L78NQ4FFZ+18
            meN58DXnGzYrdfnIZuhsRWislBQE1hYnZQY5YRMdNSLdTM2DxtE8UdXFCZl5Fx62
            K/R193g8gJzefsqPdnrrb3SZ9VankH8koJ3GdoK3ZiGrGue3SlbwmjpXfapJhoeI
            pEbA0e411fBFp82sdngw8NkrhPdRKXt3tGdinv+iT5Pgra/n5/p4m0JYSZTeImIt
            b8Rz7RgZoxnKjJK0dxdpOn8C9y0iOz/p2iIs9wh++tLgAeREfSka2fcCWL8Otn3W
            o54q4X8R4IvgjuHX9uDF4hhal7Hg+OWrRGcaunXz5dZPiKgw/+8LH8iWsvz4uoIv
            Tj4WwrzBueAN5JDPt3oy0ZL6vsXiFbWp2xTi4ewpWeE7sAA=
            =8yIh
            -----END PGP MESSAGE-----
        fp: FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4
    unencrypted_suffix: _unencrypted
    version: 3.5.0

/tmp/sops-695 master*
❯ sops -d test.yaml
[SOPS]   WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS.  comment="ENC[AES256_GCM,data:ZqzU5N3/amKUfCoPXbbM2JJ4Lpi/zJ+1UmrCurZ8hJAFGCf1QZUU93I0IPI3JwIXvwDZkrZF9KBbRhBtQT+3eVv1o7TCkY/rq90nkFflzo7AGeTf8L0YpuP5Ryv5Ha/R46Zr5X4hf3chjlONUdxaioX+ao7Chhn/Vth7,iv:JwkbA4HlGflHkQILpYgODN8zjFDoNHISuXyUycQ7aLg=,tag:NEjFWZ3frOh85z8vV7THUQ==,type:comment]"
[SOPS]   WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS.  comment="ENC[AES256_GCM,data:XA==,iv:PDDdDuyRzk/TeMPuy88bD4MIB6RFiTD9t1VSA7PWLFM=,tag:KULnokdDNGvGoBecZNbmvA==,type:comment]"
#ENC[AES256_GCM,data:ZqzU5N3/amKUfCoPXbbM2JJ4Lpi/zJ+1UmrCurZ8hJAFGCf1QZUU93I0IPI3JwIXvwDZkrZF9KBbRhBtQT+3eVv1o7TCkY/rq90nkFflzo7AGeTf8L0YpuP5Ryv5Ha/R46Zr5X4hf3chjlONUdxaioX+ao7Chhn/Vth7,iv:JwkbA4HlGflHkQILpYgODN8zjFDoNHISuXyUycQ7aLg=,tag:NEjFWZ3frOh85z8vV7THUQ==,type:comment]
#ENC[AES256_GCM,data:XA==,iv:PDDdDuyRzk/TeMPuy88bD4MIB6RFiTD9t1VSA7PWLFM=,tag:KULnokdDNGvGoBecZNbmvA==,type:comment]
prometheus-node-exporter:
    podLabels:
        jobLabel: node-exporter
    extraArgs:
    - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
    - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$

/tmp/sops-695 master*
❯ sops --version
sops 3.5.0
[info] sops 3.6.0 is available, update with `go get -u go.mozilla.org/sops/v3/cmd/sops`

And also with 3.6.0:

/tmp/sops-695 master
❯ sops --pgp FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4 -e bad/fooobar.yaml > test.yaml

/tmp/sops-695 master*
❯ sops -d test.yaml
[SOPS]   WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS.  comment="ENC[AES256_GCM,data:caFfgs1qNuJPCDiSIj+q55vubOFDVF8Tez5J6HVGMFiNbgVXBRzZRcTVSVlW2otdLjPA0v/8sVQ6t9OKYAN3GyEAqFhJSlOXM2DM1soFS68c/l0n3XSpAPAlenFUlySau9vfzrRuSvaKOEr6rnpfKoQHFbm9on6NhTlD,iv:WxYka+Xqd1WRXE65qRPc1sPPIsC3S1G7885zRh82+bs=,tag:41r3tTrMlfCs/4P6UcRunw==,type:comment]"
[SOPS]   WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS.  comment="ENC[AES256_GCM,data:Dg==,iv:rLPXIR3T9635N829VQhVzSyojKdPZsM3TQ7jc+3BnWI=,tag:A8H8Lxwglzw8GiOwNa3NFQ==,type:comment]"
#ENC[AES256_GCM,data:caFfgs1qNuJPCDiSIj+q55vubOFDVF8Tez5J6HVGMFiNbgVXBRzZRcTVSVlW2otdLjPA0v/8sVQ6t9OKYAN3GyEAqFhJSlOXM2DM1soFS68c/l0n3XSpAPAlenFUlySau9vfzrRuSvaKOEr6rnpfKoQHFbm9on6NhTlD,iv:WxYka+Xqd1WRXE65qRPc1sPPIsC3S1G7885zRh82+bs=,tag:41r3tTrMlfCs/4P6UcRunw==,type:comment]
#ENC[AES256_GCM,data:Dg==,iv:rLPXIR3T9635N829VQhVzSyojKdPZsM3TQ7jc+3BnWI=,tag:A8H8Lxwglzw8GiOwNa3NFQ==,type:comment]
prometheus-node-exporter:
    podLabels:
        jobLabel: node-exporter
    extraArgs:
    - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
    - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$

/tmp/sops-695 master*
❯ sops --version
sops 3.6.0 (latest)

[autrilla]

Note that the comments are left encrypted -- that's definitely broken and we should fix it.

[icy]

@ajvb @autrilla I've found why you couldn't reproduce my issue. It's because I use a different .sops.yaml that is also provided now in https://github.com/icy/sops-695/blob/master/.sops.yaml#L3

creation_rules:
- path_regex: ./
  encrypted_regex: '(newrelic|certificate|private|pass|key|value)|(api_url)'

with my encrypted_regex enabled as above, I got the broken output file, with my comments left decrypted:

If I remove the encrypted_regex from that file, I get the pretty good output,with my comments encrypted:

prometheus-node-exporter:
#ENC[AES256_GCM,data:1e+kZFvCxVNPv/JnWF+17xCrfkqMh7mL2TMRKFE71veST6lkFXGAVYj8fPSwGd10tJgpipqQODUs4EfEvX+sIMXGSlsOYeLiy3/wQ12YpTxrszPKoyZFnonqq+TLyHHxy71RpXk1wJEDJ0z+gx+cNROPYZZMAR1Sja16,iv:7/3HzbqySsqdz2RmhcLrJImNZhyrHNpx7BtTx0IqDEQ=,tag:lJ9LadZ/GSgun/xg5onXEg==,type:comment]
#ENC[AES256_GCM,data:nw==,iv:L1RHCkgfLZQZ8w9n58k6BstinJkSsbm+dlIHWMugPNs=,tag:Ah3WphlATwZtQGGk7uJ8Ow==,type:comment]
    podLabels:
        jobLabel: ENC[AES256_GCM,data:2X3ryKkya44OPK0q1A==,iv:dWUTWH+XNbogeSRZZ1P7qn+NakJALQVfeDn94mzMvZE=,tag:sq5x1kdxPt+fBJrzP+HlMA==,type:str]
    extraArgs:
    - ENC[AES256_GCM,data:Kw6v7lD6hmEYwfHH8PKpb3i4PQCeUPZffgRbMlgVV/CMyxiD7sXsJa/5BaOtwy1zq54avhEhCGmZEgwzvjIFToq57oFAMSOi8FJ4IEcXUpXcF+U=,iv:c5/H1vHoBt97PCWEeSDwifXork/pe53e1vqs5FrIv38=,tag:NZT9pwVwP1gOYxNkAKwq1Q==,type:str]
<snip>

It's very long topics, please accept my apologies that I forgot to mention my encrypted_regex from the beginning (The output of sops has information about sops version and encrypted_regex so I thought that's fine to ignore.)

Here's my short summary: Looked like there are 3 problems:

• Without encrypted_regex: sops -e generates parse-able yaml output. The sops -d could decrypt the output, however it couldn't decrypt some comments (so I can reproduce your problem https://github.com/mozilla/sops/issues/695#issuecomment-659663843)
• With my encrypted_regex: sops -e generates a broken yaml output.

In both cases, the two comments in the original file are not included in the yaml output.

[icy]

Three problems are

• Broken yaml output (with encrypted_regex)
• Some comments are not included in the output
• Some encrypted comments are not decrypted

[autrilla]

This is the smallest I could get the example and keep it broken:

a:
  b:
    #
    c: {}
  d:

The "Some comments are not included in the output" is known. The top-of-file comments are always going to get removed. This is documented and unfortunately hard to fix.

[icy]

The "Some comments are not included in the output" is known. The top-of-file comments are always going to get removed. This is documented and unfortunately hard to fix.

Thanks a lot, and I'm sorry that I didn't read the doc carefully. It's fine for me if that's a known problem.

[autrilla]

No worries.

Pretty sure this is a bug in the YAML library:

~/Code/yaml_test
❯ cat main.go
package main

import (
    "fmt"
    "github.com/mozilla-services/yaml"
)

func main() {
    var data []yaml.MapSlice
    in := []byte(`
a:
  b:
    #
    c: {}
  d:`)

    if err := (yaml.CommentUnmarshaler{}).UnmarshalDocuments(in, &data); err != nil {
        fmt.Printf("%v\n", err)
    }

    fmt.Printf("read: %#v\n", data)

    tout, err := (&yaml.YAMLMarshaler{Indent: 4}).Marshal(data)
    if err != nil {
        fmt.Printf("Error marshaling to yaml: %s", err)
    }

    fmt.Printf("output: \n%s\n", tout)
}

~/Code/yaml_test
❯ go run ./main.go
read: []yaml.MapSlice{yaml.MapSlice{yaml.MapItem{Key:"a", Value:yaml.MapSlice{yaml.MapItem{Key:yaml.Comment{Value:""}, Value:interface {}(nil)}, yaml.MapItem{Key:"b", Value:yaml.MapSlice{yaml.MapItem{Key:"c", Value:yaml.MapSlice(nil)}}}, yaml.MapItem{Key:"d", Value:interface {}(nil)}}}}}
output:
-   a:
    #   b:
            c: {}
        d: null

Well, in the code I wrote for it ;). I'll have a look at fixing it.

[autrilla]

Opened https://github.com/mozilla-services/yaml/pull/7 with a fix.

[icy]

Opened mozilla-services/yaml#7 with a fix.

Thanks for the quick fix. I'm looking forward to the new release.

I think there may be a problem that, sops -e doesn't verify its output. As in discussed in this issue, sops -e reported 0 however the yaml output is broken. I actually need to verify the output manually (https://github.com/icy/sops-695/blob/b6cbbcd473663dee2ac69a75a8e0eb0ed0b08f84/sops_secure_encryption.sh#L46) and this is slow.

Do you think that we can add some simple check by default to ensure any future issue of the yaml parser?

[autrilla]

That PR has landed, but we need to write a test on the SOPS side to make sure there's no regressions, and also we need to upgrade the yaml dependency.

[felixfontein]

I'm getting a lot of similar errors with some YAML files. I've built a following minimal example:
```.yaml
a:
b:
# foo

When cycled through `sops -e` / `sops -d`, I get warnings:

[SOPS] WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS. comment="ENC[AES256_GCM,data:i0gx/Q==,iv:/FOmU45WS/1e6Zv80MIGsxSpXtQFl6DaIGGGhEJGTB8=,tag:c5Bp8X3M8eu2i1Ahny+V4Q==,type:comment]"

The resulting decrypted file:
```.yaml
#ENC[AES256_GCM,data:i0gx/Q==,iv:/FOmU45WS/1e6Zv80MIGsxSpXtQFl6DaIGGGhEJGTB8=,tag:c5Bp8X3M8eu2i1Ahny+V4Q==,type:comment]
a:
    b: null

I.e. not only did the comment end up encrypted, it also moved the comment around (and when cycling again, it will vanish because it is now at the top of the file).

Is this the same bug, or should I create a new issue for this?

[felixfontein]

(I'm using sops 3.6.1)

[consideRatio]

Reproduction

Assuming there is a creation rule for my.yaml configured in a .sops.yaml already, the output of this script, which should be a decrypted file, still contains encrypted comments.

cat << EOF > my.yaml
a:
 b:
 # foo
EOF

sops --in-place -e my.yaml
sops -d my.yaml

Reproduction output - comments still encrypted

[SOPS]   WARN[0000] Found possibly unencrypted comment in file. This is to be expected if the file being decrypted was created with an older version of SOPS.  comment="ENC[AES256_GCM,data:W9cP9Q==,iv:cbarhfRocqVdBmQq9CHMo6nvNQ9PQpSCOWf1oqr/1f4=,tag:/EhN4oE7f52cGyJp/ndPmA==,type:comment]"
#ENC[AES256_GCM,data:W9cP9Q==,iv:cbarhfRocqVdBmQq9CHMo6nvNQ9PQpSCOWf1oqr/1f4=,tag:/EhN4oE7f52cGyJp/ndPmA==,type:comment]
a:
    b: null

[felixfontein]

@autrilla what is blocking a fix for this bug?

[felixfontein]

(Assuming mozilla-services/yaml#7 fixes this.)

[autrilla]

Still the same thing as in https://github.com/mozilla/sops/issues/695#issuecomment-661269798

[consideRatio]

@autrilla to summarize, are these sensible action points? My main question is if you want unit tests or e2e tests to be developed.

1. Write additional _unit tests_ of SOPS to catch potential regressions caused by https://github.com/mozilla-services/yaml/pull/7 within sops_test.go that can be run with make as done in the .travis.tml also.
2. Bump SOPS dependency of https://github.com/mozilla-services/yaml in go.mod

[autrilla]

Either works for me. This seems testable with a unit test, so I'd prefer that to keep things simple. So yeah, those look fine. The test should probably be put in https://github.com/mozilla/sops/blob/master/stores/yaml/store_test.go though.

[felixfontein]

I started trying to write a test, by simply deserializing and then re-serializing the YAML file with the store code (i.e. cycle it through the internal TreeBranches representation). I started with

```.yaml

a:
b:
# foo

Before bumping the YAML dependency, cycling it turns it into
```.yaml
a:
# foo
    b: null

After bumping the YAML dependency, the first cycle turns it into

```.yaml

a:
# foo
b: null

Cycling another time gives
```.yaml
---
# foo
a:
    b: null

So the comment crawl is still happening.

[felixfontein]

The problem is the YAML parser, the tree it returns already has the comment at the wrong position.

[autrilla]

@felixfontein actually, your issue seems unrelated to the original one in this thread. Really, we should fix these issues by migrating to the v3 YAML parser, which natively supports comments, and does it properly, instead of the way I hacked it into the v2 parser.

[felixfontein]

@autrilla yep, looks like that. In any case, I played around a bit with the v2 parser, resp. your comment hack, and got it to work better (mozilla-services/yaml#8). For example, https://github.com/mozilla-services/yaml/blob/d0788957849384a9eb9507eea2a9c8d321375598/yaml_test.go#L9-L28 cycled through the YAML parser and writer yields https://github.com/mozilla-services/yaml/blob/d0788957849384a9eb9507eea2a9c8d321375598/yaml_test.go#L110-L128, which looks a lot better.

[felixfontein]

Forgot to add: in the long term using the v3 parser will definitely be better, but no idea how hard it is to switch to it, so for now improving the v2 parser might be better :)

[felixfontein]

I created a PR to bump the yaml library version and add tests for all the problems mentioned in this issue in #754.

[kim0]

Hello .. I've been facing a lot of trouble getting comments to get in/out safely. I use https://get-ytt.io/ which depends on comments for a lot of its functionality. Ideally, after ecryption | decryption I'd get the exact same input. I saw a lot of fixes already were merged. So I built from head today HEAD-38b25bd but I think I still see issues (top level comments stripped, comments not decrypting ..etc).

Can someone please update us on the state. What is fixed, and what yet isn't?
Thanks a lot

[felixfontein]

@kim0 I think the worst bugs are fixed, at least the ones I know about / which annoyed me. What's mainly missing is a new release.

@autrilla is there a chance of a new bugfix release anytime soon? Would be really appreciated.

[felixfontein]

@kim0 I think you built from the wrong branch, the current development state is in develop, see https://github.com/mozilla/sops/commits/develop - the HEAD there is 886c1ee78144be6328f546d3aae09445562aefff, not 38b25bd (which is the HEAD of master).

[kim0]

Thanks @felixfontein that's exactly what happened. I sent a fix to homebrew https://github.com/Homebrew/homebrew-core/pull/64939/files to point to develop branch.

[autrilla]

I’m unfortunately not able to create releases ever since I left Mozilla.
@ajvb is currently the only person able to do it to my knowledge, and I
imagine they’re quite busy so they don’t like to do releases super often.

On Mon, 16 Nov 2020 at 14:48, Felix Fontein [e-mail]
wrote:

@kim0 https://github.com/kim0 I think the worst bugs are fixed, at
least the ones I know about / which annoyed me. What's mainly missing is a
new release.

@autrilla https://github.com/autrilla is there a chance of a new bugfix
release anytime soon? Would be really appreciated.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/mozilla/sops/issues/695#issuecomment-728044958, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AARH4V4IKZXEPIXBKTDBHILSQEURXANCNFSM4O4VESBQ
.

[kim0]

I built from develop branch and indeed it looks much improved! Thanks! Alas, one tiny issue still remains. The yaml file ytt needs looks like

#@data/values
---
key: value

So yeah, that first yaml document is empty. When I try to encrypt that file, I get

Error unmarshalling file: Error unmarshaling input YAML: yaml: line 1: did not find expected node content

Can I hope to get a fix so that after encryption/decryption, the resulting file is exactly equal to the input? Thanks a lot for your help!

[felixfontein]

@kim0 the error you're experiencing is already reported in #748. I just took a look, it's a problem with the YAML parser used, it doesn't like empty content.

What you can do is using the trick from #748: change your YAML file to
```.yaml

@data/values

{}

key: value

and encode it. If you decode, you'll get:
```.yaml
#@data/values
---
key: value

So every time you edit it, you have to remember to put the {} in. This is definitely annoying, but at least a workaround :)

[apeschel]

I can also confirm the devel branch works with YAML comments.

There's still a big issue though: it only works if you reencrypt the original file. The dev build won't decrypt comments encrypted with either the 3.6.0 or 3.6.1 release.