Yq: Error while merge files with Russian letters in comment

Created on 25 Feb 2020  路  3Comments  路  Source: mikefarah/yq

Describe the bug
Error while merging yaml files

Input Yaml
data1.yml:

value: newValue  

data2.yml:

value: newValue #锌褉芯胁械褉懈褌褜

Command
The command you ran:

yq m -x data1.yaml data2.yaml

Actual behavior

panic: runtime error: index out of range [10] with length 10 [recovered]
        panic: runtime error: index out of range [10] with length 10

goroutine 1 [running]:
gopkg.in/yaml%2ev3.handleErr(0xc000071878)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/yaml.go:276 +0xa1
panic(0x65b080, 0xc0000be660)
        /usr/local/Cellar/go/1.13.3/libexec/src/runtime/panic.go:679 +0x1c0
gopkg.in/yaml%2ev3.write(0xc000040380, 0xc0000c0730, 0xa, 0x10, 0xc000070528, 0x525f01)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/emitterc.go:93 +0x210
gopkg.in/yaml%2ev3.yaml_emitter_write_comment(0xc000040380, 0xc0000c0730, 0xa, 0x10, 0x1)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/emitterc.go:1978 +0xd2
gopkg.in/yaml%2ev3.yaml_emitter_process_line_comment(0xc000040380, 0xc00011c440)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/emitterc.go:1128 +0x71
gopkg.in/yaml%2ev3.yaml_emitter_emit_block_mapping_value(0xc000040380, 0xc00011c440, 0xc000070601, 0x1)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/emitterc.go:817 +0xeb
gopkg.in/yaml%2ev3.yaml_emitter_state_machine(0xc000040380, 0xc00011c440, 0xc0000be401)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/emitterc.go:303 +0x424
gopkg.in/yaml%2ev3.yaml_emitter_emit(0xc000040380, 0xc0000405c0, 0x85d020)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/emitterc.go:153 +0x14e
gopkg.in/yaml%2ev3.(*encoder).emit(0xc000040380)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/encode.go:81 +0x40
gopkg.in/yaml%2ev3.(*encoder).node(0xc000040380, 0xc0000f3a40, 0x0, 0x0)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/encode.go:504 +0xd32
gopkg.in/yaml%2ev3.(*encoder).node(0xc000040380, 0xc0000f3900, 0x0, 0x0)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/encode.go:448 +0x224
gopkg.in/yaml%2ev3.(*encoder).nodev(0xc000040380, 0x64a640, 0xc0000f3900, 0x16)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/encode.go:406 +0x80
gopkg.in/yaml%2ev3.(*encoder).marshalDoc(0xc000040380, 0x0, 0x0, 0x64a640, 0xc0000f3900, 0x16)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/encode.go:101 +0x22f
gopkg.in/yaml%2ev3.(*Encoder).Encode(0xc0001060a0, 0x64a640, 0xc0000f3900, 0x0, 0x0)
        /Users/mikefarah/dev/go/pkg/mod/gopkg.in/[email protected]/yaml.go:251 +0x105
github.com/mikefarah/yq/v3/pkg/yqlib.(*yamlEncoder).Encode(0xc0001060a8, 0xc0000f3900, 0x0, 0x0)
        /Users/mikefarah/dev/go/src/github.com/mikefarah/yq/pkg/yqlib/encoder.go:28 +0x4b
github.com/mikefarah/yq/v3/cmd.mapYamlDecoder.func1(0xc0000d08e0, 0xc0000d08e0, 0x600)
        /Users/mikefarah/dev/go/src/github.com/mikefarah/yq/cmd/utils.go:249 +0x280
github.com/mikefarah/yq/v3/cmd.readStream(0xc00000c2a0, 0x2b, 0xc0000ce560, 0x0, 0x0)
        /Users/mikefarah/dev/go/src/github.com/mikefarah/yq/cmd/utils.go:457 +0x25a
github.com/mikefarah/yq/v3/cmd.readAndUpdate(0x6c5da0, 0xc000006018, 0xc00000c2a0, 0x2b, 0xc0000d2e70, 0x0, 0x0)
        /Users/mikefarah/dev/go/src/github.com/mikefarah/yq/cmd/utils.go:337 +0x263
github.com/mikefarah/yq/v3/cmd.updateDoc(0xc00000c2a0, 0x2b, 0xc0000d2e40, 0x1, 0x1, 0x6c5da0, 0xc000006018, 0x1, 0x0)
        /Users/mikefarah/dev/go/src/github.com/mikefarah/yq/cmd/utils.go:296 +0xdd
github.com/mikefarah/yq/v3/cmd.mergeProperties(0xc0000f1680, 0xc0000d2b10, 0x2, 0x3, 0x0, 0x0)
        /Users/mikefarah/dev/go/src/github.com/mikefarah/yq/cmd/merge.go:73 +0x426
github.com/spf13/cobra.(*Command).execute(0xc0000f1680, 0xc0000d29f0, 0x3, 0x3, 0xc0000f1680, 0xc0000d29f0)
        /Users/mikefarah/dev/go/pkg/mod/github.com/spf13/[email protected]/command.go:826 +0x467
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000f0280, 0x2, 0xc0000d29c0, 0xc000046058)
        /Users/mikefarah/dev/go/pkg/mod/github.com/spf13/[email protected]/command.go:914 +0x302
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/mikefarah/dev/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
main.main()
        /Users/mikefarah/dev/go/src/github.com/mikefarah/yq/yq.go:13 +0x67

Expected behavior

value: newValue #锌褉芯胁械褉懈褌褜

Additional context
Exception apperars because i am using Russian letters in comments
yq verision 2.4.1 works fine ( maybe because it's ignoring comments)

bug

All 3 comments

Appears to be a bug in the underlying yaml.v3 parser I'm using

https://github.com/go-yaml/yaml/issues/538

I also raised this which appears to be related: https://github.com/go-yaml/yaml/issues/578

This has been fixed in 3.3.2

Was this page helpful?
0 / 5 - 0 ratings

Related issues

pcrocker-pivotal picture pcrocker-pivotal  路  3Comments

andrewsav-bt picture andrewsav-bt  路  6Comments

tadayosi picture tadayosi  路  6Comments

nicholasdille picture nicholasdille  路  7Comments

dakky picture dakky  路  5Comments