Yq: `yq read` crash on particular input

[mdubinko]

The following YAML (which I’m not sure is valid or not) crashes the latest (according to mac homebrew) version of yq

?: ?

In hex, that’s
3f 3a 20 3f 0a
In a file named wtf.yaml

FWIW online validators are fairly evenly split on whether the above is valid YAML or not.
$ yq read ./wtf.yaml
=>
panic: attempted to parse unknown event: none [recovered]

            panic: attempted to parse unknown event: none

goroutine 1 [running]:

github.com/mikefarah/yq/vendor/gopkg.in/mikefarah/yaml%2ev2.handleErr(0xc42006fa58)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/vendor/gopkg.in/mikefarah/yaml.v2/yaml.go:249 +0x99

panic(0x11c4800, 0xc420060930)

            /usr/local/Cellar/go/1.10.3/libexec/src/runtime/panic.go:502 +0x229

github.com/mikefarah/yq/vendor/gopkg.in/mikefarah/yaml%2ev2.(*parser).parse(0xc4200e2000, 0x121f220)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/vendor/gopkg.in/mikefarah/yaml.v2/decode.go:158 +0x1af

github.com/mikefarah/yq/vendor/gopkg.in/mikefarah/yaml%2ev2.(*Decoder).Decode(0xc4200608d0, 0x11ba0a0, 0xc4200608c0, 0x0, 0x0)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/vendor/gopkg.in/mikefarah/yaml.v2/yaml.go:122 +0xe5

main.readProperty.func1(0xc4200608d0, 0xc42000e0b0, 0xc4200608d0)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/yq.go:231 +0xac

main.readStream(0x7ffeefbff757, 0xa, 0xc420028400, 0x0, 0x0)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/yq.go:616 +0xe0

main.readProperty(0xc420012f00, 0xc420060860, 0x1, 0x1, 0x0, 0x0)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/yq.go:229 +0x184

github.com/mikefarah/yq/vendor/gopkg.in/spf13/cobra%2ev0.(*Command).execute(0xc420012f00, 0xc420060800, 0x1, 0x1, 0xc420012f00, 0xc420060800)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/vendor/gopkg.in/spf13/cobra.v0/command.go:762 +0x468

github.com/mikefarah/yq/vendor/gopkg.in/spf13/cobra%2ev0.(*Command).ExecuteC(0xc420012c80, 0xc420013400, 0xc420013680, 0xc420013900)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/vendor/gopkg.in/spf13/cobra.v0/command.go:852 +0x30a

github.com/mikefarah/yq/vendor/gopkg.in/spf13/cobra%2ev0.(*Command).Execute(0xc420012c80, 0x0, 0x0)

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/vendor/gopkg.in/spf13/cobra.v0/command.go:800 +0x2b

main.main()

            /private/tmp/yq-20180711-47555-1xaef2k/yq-2.1.0/src/github.com/mikefarah/yq/yq.go:33 +0x27

[mikefarah]

Yq uses the gopkg.in/yaml library to parse the yaml, so I'm guessing it's an issue in that - that said that is a pretty obscure input ;) I'm not sure if it'd be worthwhile raising the issue there as apparently they're in the midst of a rewrite.

[deitch]

Just stumbled across this one as well. Nothing special in the file AFAIK (if large-ish), but still panics.

[grahamwhaley]

I just ran into this as well - but mine is a more 'normal' string I think. It is a regexp url matching. Here is the input:

---
assets:
    badstring: "https://mirrors.edge.kernel.org/pub/linux/kernel/v4.x/linux-(.+)\.tar\.gz"

And the result:

```bash+ /snap/bin/yq r bad.yaml
panic: attempted to parse unknown event: none [recovered]
panic: attempted to parse unknown event: none

goroutine 1 [running]:
gopkg.in/mikefarah/yaml%2ev2.handleErr(0xc420051a20)
/build/yq/parts/yq/go/src/gopkg.in/mikefarah/yaml.v2/yaml.go:249 +0x9f
panic(0x5df080, 0xc420042860)
/build/yq/parts/go/build/src/runtime/panic.go:491 +0x283
gopkg.in/mikefarah/yaml%2ev2.(parser).parse(0xc4200b8000, 0x639c90)
/build/yq/parts/yq/go/src/gopkg.in/mikefarah/yaml.v2/decode.go:158 +0x1af
gopkg.in/mikefarah/yaml%2ev2.(Decoder).Decode(0xc4200427f0, 0x5d4260, 0xc4200427e0, 0x0, 0x0)
/build/yq/parts/yq/go/src/gopkg.in/mikefarah/yaml.v2/yaml.go:122 +0x119
main.readProperty.func1(0xc4200427f0, 0xc42000e0c0, 0xc4200427f0)
/build/yq/parts/yq/go/src/github.com/mikefarah/yq/yq.go:254 +0xb2
main.readStream(0x7ffc814fa51f, 0x8, 0xc42005c400, 0x0, 0x0)
/build/yq/parts/yq/go/src/github.com/mikefarah/yq/yq.go:680 +0xed
main.readProperty(0xc420084a00, 0xc420042780, 0x1, 0x1, 0x0, 0x0)
/build/yq/parts/yq/go/src/github.com/mikefarah/yq/yq.go:252 +0x191
gopkg.in/spf13/cobra%2ev0.(Command).execute(0xc420084a00, 0xc420042720, 0x1, 0x1, 0xc420084a00, 0xc420042720)
/build/yq/parts/yq/go/src/gopkg.in/spf13/cobra.v0/command.go:762 +0x475
gopkg.in/spf13/cobra%2ev0.(Command).ExecuteC(0xc420084780, 0xc420085180, 0xc420085400, 0xc420085680)
/build/yq/parts/yq/go/src/gopkg.in/spf13/cobra.v0/command.go:852 +0x334
gopkg.in/spf13/cobra%2ev0.(*Command).Execute(0xc420084780, 0xc420051f70, 0x0)
/build/yq/parts/yq/go/src/gopkg.in/spf13/cobra.v0/command.go:800 +0x2b
main.main()
/build/yq/parts/yq/go/src/github.com/mikefarah/yq/yq.go:33 +0x27
```

yq is the snap version 2.2.0

I see above that github is highlighting those backslashed .'s as 'strange yaml', and I'll look into that a bit harder. Still, you shouldn't be able to blow things up from an odd value string :-)

[grahamwhaley]

For completeness, updating the string to have \\'s fixed this for me:

    badstring: "https://mirrors.edge.kernel.org/pub/linux/kernel/v4.x/linux-(.+)\\.tar\\.gz"

and happens to be OK for my use case.
My suspicion is that the yaml parser is trying to interpret \. as a special character expansion, like it would \t, but fails spectacularly, as \. is not an expansion it recognises.

[esnible]

Just ran into this with the version I got from brew install yq.

Reproduce with yq r <(curl --silent https://raw.githubusercontent.com/istio/istio/master/tests/helm/templates/app-a.yaml) metadata.annotations

[Nuru]

Here's much simpler way to crash yq version 2.4.0 (and earlier)

$ echo '{}{}' | ./yq_linux_amd64 r -
panic: attempted to parse unknown event: none [recovered]
    panic: attempted to parse unknown event: none

@mikefarah Do you think you could fix that case more easily?

[javadba]

The following crashes it this way:

myVar=myVarVal

Simply changing to

myVar: myVarVal

resolves. This is a misleading/confusing error message and should probably be corrected.

[mikefarah]

This should be fixed in the latest 3.0.1 release https://github.com/mikefarah/yq/releases/tag/3.0.1