Go: encoding/gob: encoder should ignore embedded structs with no exported fields
by alan.strohm:
What steps will reproduce the problem? If possible, include a link to a program on play.golang.org. 1. http://play.golang.org/p/ayZMy_HFKE What is the expected output? "worked" I don't see why hidden1 (an unexported field in an embedded struct) should be handled differently than hidden2 (a top level unexported field). Making the embedded struct type itself unexported ("inner" vs "Inner") works but seems unecessary. What do you see instead? gob: type main.Inner has no exported fields
gopherbot
All 22 comments
This is a tough one. The reason to leave things alone is that it's a very common user error to attempt to marshal a struct with no exported fields, and the error message helps diagnose the case with a clear error. However, diagnosing it in this more complicated case might be possible.
_Labels changed: added priority-later, removed priority-triage._
_Status changed to Accepted._
robpike
on 11 Jul 2013
For consideration, we have: var req http.Request It has req.URI.User field of type *url.Userinfo. And url.Userinfo doesn't have exported fields. This blocks possibility of dumping http.Request to file using gob. I also wonder why does it happen when req.URI.User==nil.
orian
on 12 Sep 2013
dsymonds
on 2 Dec 2013
This is a real issue when you have structs that contain an os.File field, for example. Go has annotations right? Seems like a good use for them here: annotate the field as being ignorable by gob without error.
glycerine
on 17 Dec 2013
@robpike is there any indication when this is going to be implemented?
mever
on 16 Dec 2014
Would it be possible to make the encoder configurable with a handful of flags? Have an interface that structs with unexported fields can meet to serialize / deserialize themselves? Running into problems with a library's configuration struct that includes an x509.CertPool not having any exported fields...
robmccoll
on 10 Feb 2016
First, surely it's wiser to change the x509 library (which can be done as a strict add-on by providing a GobEncoder interface) than to change the transport layer.
Second, if you have configurations in the transport layer you introduce the likelihood that both sides of the transport may need to be configured the same, and that introduces a whole new set of problems.
robpike
on 10 Feb 2016
What configuration struct are you having problems with? If you wrote it
yourself, I was hoping you could add gob:"-" to exclude the field of type
x509.CertPool, like you can in the json and xml encoders. That doesn't work
today, but maybe we should do that for Go 1.7.
rsc
on 10 Feb 2016
tl;dr my expectation is that gob should be able to cleanly serialize the total visible serializable state of any instance of a Go type.
I was trying to serialize and deserialize cluster configurations within the gocql library: https://godoc.org/github.com/gocql/gocql#ClusterConfig
(Also, embarrassed to say I missed the encoding.Binary{Marshaler,Unmarshaler} interfaces)
Unfortunately I didn't write gocql, so for now I'm using my own struct and copying values over. Either way I find the behavior of gob in this circumstance to be somewhat inconsistent with the rest of the library.
Given that gob does nice things around skipping function pointers, channels, unexported fields, etc. cleanly and making a best effort around type matching when the tags and field names don't match, throwing an error if a struct with no exported fields is visible anywhere in your object hierarchy is surprising. The spirit of gob otherwise seems to be very much do the sane thing with best effort around all exposed state and while I would understand the condition of no exported fields anywhere in the hierarchy being an error, it would be convenient to be able to override the behavior with respect to non-root nodes in the object tree so to speak. Not asking for a change in default behavior, but perhaps increased configurability. Also a change would have no effect on the binary format or the unmarshal / deserialize operation.
Happy to propose a change and put up a PR if you are amenable to it?
robmccoll
on 15 Feb 2016
:+1: this can be annoying if you're trying to serialize a type from a dependency and there's a struct with no exported fields embedded in something. In that case the answer shouldn't have to be "change the downstream library just to deal with this".
kyhavlov
on 31 Jul 2016
There's a pretty common pattern in the code I'm working on where a sync.RWMutex is embedded into structs that need mutual exclusion, e.g.:
type foo struct {
sync.RWMutex
Foo string
Bar int
}
When trying to encode structs like this I get an error with gob: type sync.RWMutex has no exported fields. It's simple enough to make the mutex into a named field, but handling this case automatically would be really nice.
eklitzke
on 29 Sep 2016
Have another way to solve locks for struct fields and convert them to gob?
ianberdin
on 29 Oct 2016
@happierall I know this is quite an old issue, but here is how we solved it:
If we have the following structure foo:
type foo struct {
sync.RWMutex
Bar string
Baz int
}
We create an Encode struct:
type fooEncode struct {
Bar *string
Baz *int
}
We then do this to encode:
fooInstance.RLock()
defer fooInstance.RUnlock()
encodable := fooEncode {
Bar: &fooInstance.Bar,
Baz: &fooInstance.Baz,
}
err := gobEncoder.Encode(encodable)
...
This is a lot of work for just embedding sync.RWMutex, but we embed more than just sync.RWMutex (it was worth it for us).
bryanjeal
on 20 Oct 2017
Any word on this? My use case is having structs that embed log15.Logger. For the one struct that I encode with gob I've had to work-around this by storing it on unexported field, giving the ugly mystruct.logger.Debug("foo"), instead of mystruct.Debug("foo") used everywhere else in my code.
sb10
on 27 Feb 2018
Change https://golang.org/cl/117875 mentions this issue: encoding/gob: fix handling of embedded structs without exported fields
gopherbot
on 11 Jun 2018
Unexported fields were correctly handled except structs without exported field for which an error message is returned.
The check returning the error in the Encode is removed in the submitted fix as documentation is not excluding the case. The Decode part is also fixed for idem-potency. Tests have been amended.
iWdGo
on 11 Jun 2018
What is the rationale for the "limitation"? The fields of my structs are intentionally unexported because I do not want any other parts of the program to have direct access to it, but inside this file I would like to encode and decode this struct because I have visibility and it is OK or should be OK to do that.
What should I use instead then? Would encoding/binary work?
Nevermind, I got "invalid type".
odiferousmint
on 7 Mar 2019
The Encoder you are using isn't part of your code. It is another package -
one that might be in the standard library, but still separate from yours.
Within your package could either create a temporary struct to Encode/Decode or you
can write your own Encoder/Decoder within your package.
bryanjeal
on 7 Mar 2019
The Encoder you are using isn't part of your code. It is another library - one that might be in the standard library, but still separate from your package. Your package could either create a temporary struct to Encode/Decode or you can write your own Encoder/Decoder within your package.
…
On Thu, 7 Mar 2019 at 10:30, odiferousmint @.*> wrote: Why is this even a thing, what is the rationale for it? The fields of my structs are intentionally unexported because I do not want any other parts of the program to have direct access to it, but inside this file I would like to encode and decode this struct because I have visibility and it is OK or should be OK to do that. What should I use instead then? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5819 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AJS5jKsaeNQ49LcLAYhPAW1_tn4LzWT5ks5vUUzDgaJpZM4DI-zO .
I see. Thanks. :)
odiferousmint
on 7 Mar 2019
Related issues
michaelsafyan
·
3Comments
natefinch
·
3Comments
longzhizhi
·
3Comments
Miserlou
·
3Comments
dominikh
·
3Comments
Most helpful comment
There's a pretty common pattern in the code I'm working on where a
sync.RWMutexis embedded into structs that need mutual exclusion, e.g.:When trying to encode structs like this I get an error with
gob: type sync.RWMutex has no exported fields. It's simple enough to make the mutex into a named field, but handling this case automatically would be really nice.