Protobuf: proto: Message containing a Oneof marshals to nothing

[felzix]

I'm not convinced that I've found a bug but I can't figure out what I'm doing wrong so I'm filing a bug report.

What version of protobuf and what language are you using?
Version: v3.6.1

What did you do?
I wanted a protobuf that I could store just about any simple value, mimicking the flexibility of JSON. I created Primitive for this.

The following code are excerpts.

syntax = "proto3";

message Primitive {
    oneof value {
        string string2 = 1;
        int64 int2 = 2;
        bool bool2 = 3;
        float float2 = 4;
    }
}

I added 2 to the field names to make sure the marshaling problem isn't somehow caused by reserved words. This seemed unlikely anyway because an underscore was automatically added in the generated code after "string" before I added the 2's.

package main

import (
    "./types
    "github.com/gogo/protobuf/proto"
    "reflect"
    "testing"
)

func TestPrimitiveProtobuf (t *testing.T) {
    primitiveInt := &types.Primitive{Value: &types.Primitive_Int2{Int2: 12}}
    data, err := proto.Marshal(primitiveInt)

    t.Log(reflect.TypeOf(data))
    t.Log(data)
    t.Log(len(data))
    t.Log(err)

    if err != nil {
        t.Fatal("marshaling error: ", err)
    }

    t.Error("fail to logs can be seen")
}

The output:

    f_test.go:57: []uint8
    f_test.go:56: []
    f_test.go:58: 0
    f_test.go:59: <nil>

Or in other words, marshaling a Primitive results in a zero-length array of uint8. Without throwing an error.

My test code also unmarshals data and compares the result but I removed that code because the problem seems to be with the marshaling.

What did you expect to see?

Marshaled data that contains something that can be unmarshaled back into the original data.

Or an error that the marshaling failed.

What did you see instead?

The marshaled data is empty and err is nil.

Anything else we should know about your project / environment?

Mac OS X

I also used an earlier version of protoc with the same result.

I successfully marshaled and unmarshaled a simpler message:

message Age {
    uint64 ticks = 1;
}

Here's the generated code for marshaling:

// XXX_OneofFuncs is for the internal use of the proto package.
func (*Primitive) XXX_OneofFuncs() (func(msg proto.Message, b *proto.Buffer) error, func(msg proto.Message, tag, wire int, b *proto.Buffer) (bool, error), func(msg proto.Message) (n int), []interface{}) {
    return _Primitive_OneofMarshaler, _Primitive_OneofUnmarshaler, _Primitive_OneofSizer, []interface{}{
        (*Primitive_String2)(nil),
        (*Primitive_Int2)(nil),
        (*Primitive_Bool2)(nil),
        (*Primitive_Float2)(nil),
    }
}

func _Primitive_OneofMarshaler(msg proto.Message, b *proto.Buffer) error {
    m := msg.(*Primitive)
    // value
    switch x := m.Value.(type) {
    case *Primitive_String2:
        b.EncodeVarint(1<<3 | proto.WireBytes)
        b.EncodeStringBytes(x.String2)
    case *Primitive_Int2:
        b.EncodeVarint(2<<3 | proto.WireVarint)
        b.EncodeVarint(uint64(x.Int2))
    case *Primitive_Bool2:
        t := uint64(0)
        if x.Bool2 {
            t = 1
        }
        b.EncodeVarint(3<<3 | proto.WireVarint)
        b.EncodeVarint(t)
    case *Primitive_Float2:
        b.EncodeVarint(4<<3 | proto.WireFixed32)
        b.EncodeFixed32(uint64(math.Float32bits(x.Float2)))
    case nil:
    default:
        return fmt.Errorf("Primitive.Value has unexpected type %T", x)
    }
    return nil
}

[dsnet]

What you're seeing seems wrong. What version of the proto package are you using? That usually has a greater impact of behavior than the generated code.

[dsnet]

Using:

• v1.2.0 of the proto package and protoc-gen-go plugin
• v3.5.1 of protoc compiler
• v1.11.1 of the go toolchain

I am unable to reproduce the issue:

$ cat main.proto
syntax = "proto3";

message Primitive {
    oneof value {
        string string2 = 1;
        int64 int2 = 2;
        bool bool2 = 3;
        float float2 = 4;
    }
}

$ protoc --go_out=. main.proto



md5-55767bf549d47a5636bebf0c80b55704



$ cat main.go
package main

import (
    "fmt"

    proto "github.com/golang/protobuf/proto"
)

func main() {
    m := &Primitive{Value: &Primitive_Int2{Int2: 12}}
    b, err := proto.Marshal(m)
    fmt.Println(b, err)
}



md5-55767bf549d47a5636bebf0c80b55704



$ go run main.go main.pb.go
[16 12] <nil>

[felzix]

@dsnet

This is the latest git commit in my local github.com/golang/protobuf package, which is what the generated code is using:

commit 7011d38ac0d201eeddff4a4085a657c3da322d75 (HEAD -> master)
Author: Damien Neil <[email protected]>
Date:   Fri Sep 28 15:12:48 2018 -0700

    protoc-gen-go: put all imports in one section (#720)

    Keep plugin imports and non-plugin imports in the same section, but
    don't try to split stdlib and non-stdlib imports into different
    sections. Consistent with using astutil to insert all imports from a
    blank slate.

I pulled the latest and rebuilt but it had the same result. (git commit hash ddf22928ea3c56eb4292a0adbbf5001b1e8e7d0d).

I originally used v3.5.1 of protoc but updated to the latest, v3.6.1, before filing this bug.

[felzix]

I ran with your steps and got the same result as you. I'll investigate more. Thanks!

[dsnet]

I just noticed that you're importing "github.com/gogo/protobuf/proto" instead of "github.com/golang/protobuf/proto". I'm not sure the gogo fork does, but seems to be amiss.

[felzix]

@dsnet That was the problem! Thanks so much!