Go: proposal: encoding/json add required tag

[khomyakov42]

Add "required" tag for Unmarshal

[odeke-em]

This issue is a duplicate of rejected proposal https://github.com/golang/go/issues/16426, for which the verdict was that it makes sense to do validation in an external library.

Closing this issue unless someone else thinks we can do something more.

[cespare]

Please reopen. This is not a duplicate of #16426. That proposal was vastly more complex.

I think we should do this, or else provide another mechanism for users to detect that a key was missing. (Another mechanism could be a Decoder mode a la Use Number.)

In fact, I thought we already had an issue for this, but I cannot find it now.

I wrote this in https://github.com/golang/go/issues/6901#issuecomment-210728870:

Today, I'm aware of two workarounds:

• Separately unmarshal the object into a map in order to obtain and inspect the key set.
• Use pointers for all field values. For example, the github.com/google/go-github API wrapper uses pointer fields as a rule for this reason. (But even then, you can't distinguish between "k":null and a missing key.)

[odeke-em]

For sure, sorry about that.

[adg]

Add "required" tag for Unmarshal

This proposal lacks detail. Can you please expand upon this?

[minux]

If we do add a way to specify a field must present when unmarshaling, i
suggest we make it universal so that various encoders could all use the
same mechanism. (That is, a global required tag rather than a json specific
one.)

[khomyakov42]

If "required" tag, the corresponding field must be present in json format, otherwise the MissingrequiredField error. Example:

func TestJsonRequiredTag1(t *testing.T) {
    data := []byte(`{}`)
    var obj1 struct { Value string `json:"required"`}
    if err := json.Unmarshal(data, &obj1); err != nil {
        if _, ok := err.(MissingRequiredFields); !ok {
            t.Fatal("Expected MissingRequiredFields error");
        }
    } else {
        t.Fatal("Expected MissingRequiredFields error");
    }
}
func TestJsonRequiredTag2(t *testing.T) {
    data := []byte(`{"Value": null}`)
    var obj2 struct { Value string `json:"required"`}
    if err := json.Unmarshal(data, &obj2); err != nil {
        t.Fatal(err)
    }
    if obj2.Value != "" {
        t.Fatal("Expected empty string")
    }
}
func TestJsonRequiredTag3(t *testing.T) {
    data := []byte(`{}`)
    var obj3 struct { Value *string `json:"required"`}
    if err := json.Unmarshal(data, &obj3); err != nil {
        if _, ok := err.(MissingRequiredFields); !ok {
            t.Fatal("Expected MissingRequiredFields error");
        }
    } else {
        t.Fatal("Expected MissingRequiredFields error");
    }
    if obj3.Value != nil {
        t.Fatal("Expected nil")
    }
}
func TestJsonRequiredTag4(t *testing.T) {
    data := []byte(`{"Value": null}`)
    var obj4 struct { Value []string `json:"required"`}
    if err := json.Unmarshal(data, &obj4); err != nil {
        t.Fatal(err);
    }
    if len(obj4.Value) != 0 {
        t.Fatal("Expected empty array")
    }
}
func TestJsonRequiredTag5(t *testing.T) {
    data := []byte(`{}`)
    var obj5 struct { Value *[]string `json:"required"`}
    if err := json.Unmarshal(data, &obj5); err != nil {
        t.Fatal(err);
    }
    if obj5.Value != nil {
        t.Fatal("Expected nil")
    }
}

[bradfitz]

Sorry, as useful as this might be, we really have to start declining changes to the json package, at least before they've been prototyped and used externally first.

[omeid]

The problem with this issue is that any prototype and/or alternative would need to completly try to replace the json package and or require double decoding.

An alternative solution is to expose some of the json encoding internals to allow for implementing superset packages that uses package json internally.

Would a proposal for such changes will be considered?