Yaml: Composite types do not unmarshal correctly (i.e. behavior differs from json.Unmarshal)

[bcronin]

Looks like yaml.Unmarshal() and json.Unmarshal() handle composite types differently. The yaml unmarshalling does not appear to read the tags on the inner type. Note ValueA is unmarshalled correctly in the json case, but not in the yaml case.

package main

import (
    "encoding/json"
    "fmt"

    "github.com/go-yaml/yaml"
)

type A struct {
    ValueA int `json:"a" yaml:"a"`
}
type B struct {
    A
    ValueB int `json:"b" yaml:"b"`
}

const jsonData = `{ "a" : 1, "b" : 2 }`
const yamlData = `
a : 1
b : 2
`

func main() {
    jdata := B{}
    ydata := B{}

    json.Unmarshal([]byte(jsonData), &jdata)
    yaml.Unmarshal([]byte(yamlData), &ydata)

    // $ go run main.go
    // json: {{1} 2}
    // yaml: {{0} 2}
    fmt.Printf("json: %v\n", jdata)
    fmt.Printf("yaml: %v\n", ydata)
}

[advance512]

This works:

package main

import (
    "encoding/json"
    "fmt"

    "gopkg.in/yaml.v2"
)

type A struct {
    ValueA int `json:"a" yaml:"a"`
}
type B struct {
    A      `yaml:",inline"`
    ValueB int `json:"b" yaml:"b"`
}

const jsonData = `{ "a" : 1, "b" : 2 }`
const yamlData = `
a : 1
b : 2
`

func main() {
    jdata := B{}
    ydata := B{}

    json.Unmarshal([]byte(jsonData), &jdata)
    yaml.Unmarshal([]byte(yamlData), &ydata)

    // $ go run main.go
    // json: {{1} 2}
    // yaml: {{1} 2}
    fmt.Printf("json: %v\n", jdata)
    fmt.Printf("yaml: %v\n", ydata)
}

[advance512]

(I just added a tag on the A anonymous field.) Whether or not this is "good behavior" or not depends. I agree that, generally, go-yaml should work in a similar way to go-json. In this case, however, I don't really like the way go-json works - it just seems too "magicky".

That being said, adding yaml:",inline" isn't a pretty solution either.

[advance512]

On second thought, perhaps setting anonymous fields to automatically be "inline" does make sense.

[niemeyer]

This is indeed inconsistent with the standard json package, and the main reason why it is inconsistent is because it was implemented before the json package supported that feature.

That said, I actually prefer the way it works in the yaml package, to be honest. This allows keeping the fact of whether the fields of the B type in your example should be exported or not orthogonal to whether you want them inlined in the yaml output.

For example, this works:

foo Foo `yaml:",inline"`

It correctly inlines the field in the yaml output, despite the fact that in Go this is not an anonymous field.

In either case, this is all theory and background. We're not changing this now because this would break existent code in a hard way to anticipate, which is really bad for a package to do. I hope we can agree on that one, whether you consider this behavior a feature or a wart.

[bcronin]

The backward compatibility issue is certainly an important consideration. Thanks for the explanation.

(Tangential aside not directly related to go-yaml: I wish Go had a standardized package versioning mechanism so that a backwards incompatible change like this could be considered and still rolled out smoothly without unexpectedly interfering with the existing users of the library.)

@niemeyer Thanks for your work on this library, by the way. It's much appreciated.

[advance512]

@bcronin - go-yaml uses pkg.in to offer the standardized package versioning you speak of.

You'll notice the correct way to import the library is:

import "gopkg.in/yaml.v2"

which means - version 2. It would be very easy to create a new branch, let's call it - v3, and break backwards compatibility there. No one would expect it to work like v2, just like no one expects v2 to work exactly like v1.

So, even though I agree with @niemeyer about the existing behavior being nicer (in my opinion), I don't his comment about breaking backwards compatibility. Just that this change probably doesn't warrant a v3 on its own.

[rushiagr]

Thanks for the insights! I for one can say that this feature/misfeature surprised me and I ended up spending about an hour before I reached this Github issue. Since this is an important and surprising behavior, may I ask if we highlight this in the readme?