Protobuf: proto: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct ...

Created on 20 Apr 2020  路  25Comments  路  Source: golang/protobuf

I am recently try to build a project which I had build it multiple times successfully. But recently I get this error while building grpcurl as part of build process, is there any imcompatible changes made to your repo?
/# Build grpcurl
go get github.com/fullstorydev/grpcurl
/# google.golang.org/protobuf/proto
/root/go/src/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct ...

needs-investigation
Source
picture m-liaee
👍5

Most helpful comment

Per @relunctance's comment (https://github.com/golang/protobuf/issues/1094#issuecomment-617072918), it seems to work on go1.13, but not go1.11. I wonder if the build problems have something to do with the lack of module support.

At a base level, our unit tests checks that our tests all pass from go1.9 to go1.14.

picture dsnet on 24 Apr 2020
👍2

All 25 comments

$ go get github.com/fullstorydev/grpcurl
go: downloading github.com/fullstorydev/grpcurl v1.5.1
go: github.com/fullstorydev/grpcurl upgrade => v1.5.1
go: downloading github.com/jhump/protoreflect v1.5.0

Can you give the exact steps to reproduce the error?

neild picture neild on 20 Apr 2020

go version : go11 

cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct ...

go 1.13 is ok
but go1.11 is faild

relunctance picture relunctance on 21 Apr 2020

https://travis-ci.com/github/faucetsdn/faucetagent

relunctance picture relunctance on 21 Apr 2020

I'm extremely puzzled as to how you're getting that particular error, since I don't believe the ProtoMethods method has ever returned a *protoreflect.Message. It looks to me as if your Travis build is using a stale or corrupted cache; I can run the commands it invokes (https://github.com/faucetsdn/faucetagent/blob/master/test-dependencies.sh#L26) locally without any errors.

Try dropping your Travis cache?

neild picture neild on 21 Apr 2020

Hi. I,ve already dropped cache and the same thing is happening. there is something else I could do?
If i compile a project that don't uses this it compiles successfully.
Thank you so much for your support.!

Edinsonmyosa picture Edinsonmyosa on 23 Apr 2020

What versions of Go are you using?

dsnet picture dsnet on 23 Apr 2020

I Have Version 1.11

Edinsonmyosa picture Edinsonmyosa on 23 Apr 2020

Per @relunctance's comment (https://github.com/golang/protobuf/issues/1094#issuecomment-617072918), it seems to work on go1.13, but not go1.11. I wonder if the build problems have something to do with the lack of module support.

At a base level, our unit tests checks that our tests all pass from go1.9 to go1.14.

dsnet picture dsnet on 24 Apr 2020
👍2

I've run the following on my workstation and I unfortunately cannot reproduce the problems:

for GO in go1.9 go1.10 go1.11 go1.12 go1.13 go1.14; do
    echo $GO
    /bin/rm -rf /tmp/gopath
    GOPATH=/tmp/gopath $GO get github.com/fullstorydev/grpcurl
done
dsnet picture dsnet on 24 Apr 2020

Hi I just updated my go version and it worked, thank you so much guys!

Edinsonmyosa picture Edinsonmyosa on 24 Apr 2020

I am working on a project with hyperledger-fabric 1.4 where I build and deploy a chaincode that has protobuf and its dependencies and I also started getting this error although everything worked fine before a recent docker update

I have github.com/golang/protobuf v1.4.1 in my go.mod of my chaincode and when deploying it in the blockchain (so building the binary and deploying it in a separate docker container) it fails with a building error:

Error: could not assemble transaction, err proposal response was not successful, error code 500, msg error starting container: error starting container: Failed to generate platform-specific docker build: Error returned from build: 2 "# Hyperledger/AbstractDealChaincode/vendor/google.golang.org/protobuf/proto
chaincode/input/src/Hyperledger/AbstractDealChaincode/vendor/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument

Unfortunately the images of hyperledger fabric used in my setup use go of versions 1.11 and 1.12 so I have no option of switching to go 1.13

TSRL picture TSRL on 12 May 2020
👍1

I would love to fix this issue, but without a reproduction, we are unable to do anything about this. My best guess is that this is due a bug in the build cache introduced in go1.10.

Can you try disabling the build cache? https://golang.org/doc/go1.11#gocache and https://golang.org/doc/go1.12#gocache

dsnet picture dsnet on 12 May 2020

I would love to fix this issue, but without a reproduction, we are unable to do anything about this. My best guess is that this is due a bug in the build cache introduced in go1.10.

Can you try disabling the build cache? https://golang.org/doc/go1.11#gocache and https://golang.org/doc/go1.12#gocache

I will try this soon as I will have to install a newer version of docker engine but for now I solved this problem for myself by downgrading the version of docker from the latest one (so from 19.03.08 to 19.03.05: https://docs.docker.com/engine/release-notes/), you may see there are latest releases had some updates with grpc. Thus you may reproduce the bug by installing the latest docker version and build anything with github.com/golang/protobuf v1.4.1 inside a container with go < 1.13

TSRL picture TSRL on 12 May 2020

Hi, I use docker-compose for local tests for microservices and get same problem:

/go/src/test/sms-service/vendor/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument

My Dockerfiles were:

FROM golang:1.11.2-stretch
RUN go get github.com/cespare/reflex
COPY reflex.conf /
ENTRYPOINT ["reflex", "-c", "/reflex.conf"]

I tried to change go version to 1.13 but nothing changed. I use Ubuntu btw, my colleague use Mac and it works on his machine

danikk741 picture danikk741 on 21 May 2020

Hi, I use docker-compose for local tests for microservices and get same problem:

/go/src/pingocean-front/sms-service/vendor/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument

My Dockerfiles were:

FROM golang:1.11.2-stretch
RUN go get github.com/cespare/reflex
COPY reflex.conf /
ENTRYPOINT ["reflex", "-c", "/reflex.conf"]

I tried to change go version to 1.13 but nothing changed. I use Ubuntu btw, my colleague use Mac and it works on his machine

Hello,

I had exactly the same problem in docker as I described earlier. For me downgrading from latest version of docker fixed this problem

TSRL picture TSRL on 21 May 2020

Hi, I use docker-compose for local tests for microservices and get same problem:

/go/src/pingocean-front/sms-service/vendor/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument

My Dockerfiles were:

FROM golang:1.11.2-stretch
RUN go get github.com/cespare/reflex
COPY reflex.conf /
ENTRYPOINT ["reflex", "-c", "/reflex.conf"]

I tried to change go version to 1.13 but nothing changed. I use Ubuntu btw, my colleague use Mac and it works on his machine

Hello,

I had exactly the same problem in docker as I described earlier. For me downgrading from latest version of docker fixed this problem

What version of docker is ok?

danikk741 picture danikk741 on 21 May 2020

Hi, I use docker-compose for local tests for microservices and get same problem:

/go/src/pingocean-front/sms-service/vendor/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument

My Dockerfiles were:

FROM golang:1.11.2-stretch
RUN go get github.com/cespare/reflex
COPY reflex.conf /
ENTRYPOINT ["reflex", "-c", "/reflex.conf"]

I tried to change go version to 1.13 but nothing changed. I use Ubuntu btw, my colleague use Mac and it works on his machine

Hello,
I had exactly the same problem in docker as I described earlier. For me downgrading from latest version of docker fixed this problem

What version of docker is ok?

19.03.05 works for me

TSRL picture TSRL on 21 May 2020

Downgraded to 18.03.1, same problem

danikk741 picture danikk741 on 21 May 2020

Wut?

Step 2/4 : RUN go get -v github.com/cespare/reflex
 ---> Running in 462659270cf1
github.com/cespare/reflex (download)
github.com/fsnotify/fsnotify (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
github.com/kballard/go-shellquote (download)
github.com/kr/pty (download)
github.com/creack/pty (download)
github.com/ogier/pflag (download)
golang.org/x/sys/internal/unsafeheader
github.com/kballard/go-shellquote
golang.org/x/sys/unix
github.com/creack/pty
github.com/ogier/pflag
github.com/kr/pty
github.com/fsnotify/fsnotify
github.com/cespare/reflex

cespare/reflex does not even use protobufs

What is the full output of your docker-compose up? Because it seems highly unlikely that this has anything to do with cespare/reflex.

puellanivis picture puellanivis on 21 May 2020

Wut?

Step 2/4 : RUN go get -v github.com/cespare/reflex
 ---> Running in 462659270cf1
github.com/cespare/reflex (download)
github.com/fsnotify/fsnotify (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
github.com/kballard/go-shellquote (download)
github.com/kr/pty (download)
github.com/creack/pty (download)
github.com/ogier/pflag (download)
golang.org/x/sys/internal/unsafeheader
github.com/kballard/go-shellquote
golang.org/x/sys/unix
github.com/creack/pty
github.com/ogier/pflag
github.com/kr/pty
github.com/fsnotify/fsnotify
github.com/cespare/reflex

cespare/reflex does not even use protobufs

What is the full output of your docker-compose up? Because it seems highly unlikely that this has anything to do with cespare/reflex.

I did not say that reflex is the problem. Actually I fixed problem by changing volumes and workdir in docker-compose file from:

volumes:
      - ./test-service:/app
      - $GOPATH/src:/go/src
working_dir: /app

to:

volumes:
      - ./test-service:/go/src/test/test-service
      - ./test-service/vendor:/go/src
working_dir: /go/src/test/test-service

It fixed problem with protobuff error

danikk741 picture danikk741 on 22 May 2020

No, you did not say that reflex was the problem, but the only information you included in your post was the single error message line, and an apparently irrelevant Dockerfile, what are we supposed to assume from the inclusion of that Dockerfile?

      - $GOPATH/src:/go/src

So you were using the local $GOPATH/src from your computer, which brought with it all the existing state of your own computer, into the docker build process.

puellanivis picture puellanivis on 23 May 2020

go version 1.11

[gql@vxved ~/test/go/hbase]$ go get -v github.com/tsuna/gohbase
google.golang.org/protobuf/proto
# google.golang.org/protobuf/proto
../../../gopath/src/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument
relunctance picture relunctance on 8 Jun 2020

go version 1.13
Error: could not assemble transaction, err proposal response was not successful, error code 500, msg error starting container: error starting container: Failed to generate platform-specific docker build: Error returned from build: 2 "# chaincode/vendor/google.golang.org/protobuf/proto chaincode/input/src/chaincode/vendor/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument
What can I do?

jueaiminge picture jueaiminge on 28 Jul 2020

@jueaiminge

I recommend that you avoid using vendor directories as they often cause type-variance issues where two types that should be identical are not.

dsnet picture dsnet on 29 Jul 2020

Hi, I use docker-compose for local tests for microservices and get same problem:

/go/src/pingocean-front/sms-service/vendor/google.golang.org/protobuf/proto/proto_methods.go:18:23: cannot use m.ProtoMethods() (type *protoreflect.Message) as type *struct { pragma.NoUnkeyedLiterals; Flags uint64; Size func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Flags uint8 }) struct { pragma.NoUnkeyedLiterals; Size int }; Marshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8 }) (struct { pragma.NoUnkeyedLiterals; Buf []byte }, error); Unmarshal func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message; Buf []byte; Flags uint8; Resolver interface { FindExtensionByName(protoreflect.FullName) (protoreflect.ExtensionType, error); FindExtensionByNumber(protoreflect.FullName, protowire.Number) (protoreflect.ExtensionType, error) } }) (struct { pragma.NoUnkeyedLiterals; Flags uint8 }, error); Merge func(struct { pragma.NoUnkeyedLiterals; Source protoreflect.Message; Destination protoreflect.Message }) struct { pragma.NoUnkeyedLiterals; Flags uint8 }; CheckInitialized func(struct { pragma.NoUnkeyedLiterals; Message protoreflect.Message }) (struct { pragma.NoUnkeyedLiterals }, error) } in return argument

My Dockerfiles were:

FROM golang:1.11.2-stretch
RUN go get github.com/cespare/reflex
COPY reflex.conf /
ENTRYPOINT ["reflex", "-c", "/reflex.conf"]

I tried to change go version to 1.13 but nothing changed. I use Ubuntu btw, my colleague use Mac and it works on his machine

Hello,
I had exactly the same problem in docker as I described earlier. For me downgrading from latest version of docker fixed this problem

What version of docker is ok?

19.03.05 works for me

Docker 19 is the version where I see similar error

davidkhala picture davidkhala on 28 Aug 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

tfmertz picture tfmertz  路  4Comments
seungryulchoisc picture seungryulchoisc  路  3Comments
leduy99 picture leduy99  路  4Comments
tamird picture tamird  路  3Comments
emcfarlane picture emcfarlane  路  4Comments