Etcd: `go get github.com/cores/etcd/client` fails on generated code

Created on 19 Oct 2017 · 34Comments · Source: etcd-io/etcd

As of the current commit (a8f9de2abf824db8a3aa7358227653ca00d9f755), I'm unable to cleanly go get github.com/coreos/etcd/client for use as a library. The build fails when looking at keys.generated.go.

Running go generate resolves compilation, though go test github.com/coreos/etcd/client still fails.

kmitchell @ kmitchell-laptop ~
└─ 0 $ ▶ go get github.com/coreos/etcd/client
# github.com/coreos/etcd/client
go3/src/github.com/coreos/etcd/client/keys.generated.go:72:6: r.EncodeArrayStart undefined (type codec.encDriver has no field or method EncodeArrayStart)
go3/src/github.com/coreos/etcd/client/keys.generated.go:80:6: r.EncodeMapStart undefined (type codec.encDriver has no field or method EncodeMapStart)
go3/src/github.com/coreos/etcd/client/keys.generated.go:83:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:91:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:93:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:102:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:109:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:111:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:119:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:126:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
go3/src/github.com/coreos/etcd/client/keys.generated.go:126:6: too many errors

Workarounds I've used:

Vendoring an older version of etcd/client and codecgen packages works around the issue.
clientv3 builds and passes tests.
Cloning the etcd repo separately, then using glide to pull in the repo's dependencies, results in a successful build & test of github.com/coreos/etcd/client. This isn't viable on its own for use as a library in other projects, but merging the etcd glide files with the project's glide file could work to get a functional build.

Help Wanted

Source

kemitche

👍1

Most helpful comment

just remove keys.generated.go fix the problem. keys.generated.go will be generated automatically

xiezhenye on 23 Oct 2017

👍7 😄4 🎉3 ❤2

All 34 comments

https://github.com/ugorji/go/commit/c3953defa8f202d85a6d73eead7b77073bf5701a

is what broke this.

georgekarrv on 19 Oct 2017

I have failing application builds due to this. :( I've been able to revert back to older containers for the time being. How soon can my team expect a fix/rollback? Thanks!

andrewmeissner on 19 Oct 2017

^^ @georgekarrv @andrewmeissner

codecgen tool goes with the codecgen version.

My sense is that folks updated the codec library, but didn't rebuild the codecgen tool before re-generating the files.

The sequence should be something similar to:

go get -u github.com/ugorji/go/codec (this should download the codec library)
go get -u github.com/ugorji/go/codec/codecgen (this will rebuild the codecgen executable)
codecgen ..... (now run the codecgen executable)

I think it may be sufficient to re-build the codecgen executable, so it matches the codec library.

ugorji on 19 Oct 2017

@ugorji I've never had to use the codecgen executable before. How/Where should I run that?

andrewmeissner on 19 Oct 2017

Someone from the coreos team runs it. At a certain point, they will update their vendored copy of github.com/ugorji/go/codec/..., rebuild the codecgen executable, regenerate files like keys.generated.go .

ugorji on 19 Oct 2017

We've regenerated here https://github.com/coreos/etcd/pull/8574.

gyuho on 19 Oct 2017

@gyuho I've deleted etcd from my gopath, and am still getting the same errors on go get. Am I still missing something?

andrewmeissner on 19 Oct 2017

Did you try go get -v -u github.com/ugorji/go/codec/... as @ugorji suggested?

gyuho on 19 Oct 2017

@gyuho yes :( I also tried completely removing them + etcd again from my gopath and still get the same errors.

andrewmeissner on 19 Oct 2017

@gyuho I think you need to re-generate the files again and check into source control. They were last generated on sept 19 or so. I updated the codecgen version to 8.

I also think you should consider vendoring codec in etcd client, since you use generated files in there.

ugorji on 19 Oct 2017

@ugorji Ok thanks! Will try that.

gyuho on 19 Oct 2017

Any ETA on this? I can't resume active development until this is fixed... :(

andrewmeissner on 20 Oct 2017

@andrewmeissner Help is always welcome! Currently we are busy with clientv3 fixes. Sorry!

gyuho on 20 Oct 2017

@gyuho took a stab at it here: https://github.com/coreos/etcd/pull/8721

andrewmeissner on 20 Oct 2017

Should be fixed via https://github.com/coreos/etcd/pull/8721.

gyuho on 20 Oct 2017

👍1

just remove keys.generated.go fix the problem. keys.generated.go will be generated automatically

xiezhenye on 23 Oct 2017

👍7 😄4 🎉3 ❤2

Commands to type:

rm ../github.com/coreos/etcd/client/keys.generated.go
go get github.com/coreos/etcd/client

kenfire on 23 Nov 2017

Not sure why this issue got closed. I just tried out v3.3.2 and the keys.generated.go file is still being generated, which still makes it impossible to build in a Heroku deployment.

drauschenbach on 25 Mar 2018

@dahefanteng can you please provide full steps to reproduce?

hexfusion on 25 Mar 2018

Well here are my steps seems reproduce against master.

Dockerfile                                                                                                                                                                          ◀◀ [buffers]
FROM golang:latest
RUN go get github.com/coreos/etcd/client

docker run c99c9e4a4bc9  find ./ -name keys.generated.go
./src/github.com/coreos/etcd/client/keys.generated.go

hexfusion on 25 Mar 2018

Looks like I am confused, this file is checked into version control... @dahefanteng can you outline the problem?

hexfusion on 25 Mar 2018

Here's what happens when I disable the documented work-around to this issue:

dep ensure
# Work-around for https://github.com/coreos/etcd/issues/8715
#-rm vendor/github.com/coreos/etcd/client/keys.generated.go (<== COMMENTED OUT)

$ go install
# github.com/BixData/bixserver/vendor/github.com/coreos/etcd/client
vendor/github.com/coreos/etcd/client/keys.generated.go:71:6: r.EncodeArrayStart undefined (type codec.genHelperEncDriver has no field or method EncodeArrayStart)
vendor/github.com/coreos/etcd/client/keys.generated.go:79:6: r.EncodeMapStart undefined (type codec.genHelperEncDriver has no field or method EncodeMapStart)
vendor/github.com/coreos/etcd/client/keys.generated.go:83:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:91:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:93:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:102:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:109:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:111:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:119:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:126:6: z.EncSendContainerState undefined (type codec.genHelperEncoder has no field or method EncSendContainerState)
vendor/github.com/coreos/etcd/client/keys.generated.go:126:6: too many errors

The problem is that the work-around works fine in some environments, but it never worked in environments such as Buildstep or Heroku, where dep ensure is being run for us without any room for us to hook extra hacks around it.

drauschenbach on 25 Mar 2018

Oh but if I upgrade to 3.3.2 (from 3.2.11), the compile error goes away. OK things seem fine.

drauschenbach on 25 Mar 2018

Oh but if I upgrade to 3.3.2 (from 3.2.11), the compile error goes away. OK things seem fine.

Hi @drauschenbach, so you feel this issue is no longer a problem for you?

hexfusion on 25 Mar 2018

Strange, I've upgraded my Gopkg.toml to etcd 3.3.2, but a Heroku deployment still seems to choke on this issue: https://gist.github.com/drauschenbach/cbff93872d3f52269ccc70972b7c94b8

My Gopkg.toml excerpt:

[metadata.heroku]
  root-package = "github.com/...myapp..."
  go-version = "1.9"
  install = ["./..."]

[[constraint]]
  name = "github.com/coreos/etcd"
  version = "3.3.2"

drauschenbach on 25 Mar 2018

$ go install
github.com/BixData/bixserver/vendor/github.com/coreos/etcd/client

@drauschenbach if you can allow me access to the repo, or create a simple example for testing I can try to explore further but without repeatable steps it is hard to debug.

hexfusion on 26 Mar 2018

I upgraded and regenerated with latest https://github.com/coreos/etcd/pull/9494.

gyuho on 27 Mar 2018

@drauschenbach I recommend to vendor etcd client with same ugorji/go/codec git revision for release branches. It's been making breaking changes quite often, while client development has stopped in favor of clientv3. Now the tip even requires backward-incompatible changes (see https://github.com/coreos/etcd/pull/9494 and https://github.com/coreos/etcd/issues/9447).

Please let me know if you have any other concerns.

Otherwise, I will close this issue.

gyuho on 28 Mar 2018

Got an update from ugorji/go/codec maintainer https://github.com/coreos/etcd/issues/9447#issuecomment-377281385. Let's wait!

gyuho on 29 Mar 2018

@gyuho please try again and let me know if this is fully fixed on tip. If it is, I will do some sanity testing and tag a github.com/ugorji/go/[email protected] release this weekend.

ugorji on 6 Apr 2018

@ugorji Thanks, I tried it and found some other issues here https://github.com/gyuho/etcd/commit/1743d96f519d9f7b488ec26edc4128cf757ced6d#r28455954.

gyuho on 6 Apr 2018

@gyuho please try again. Should be fixed now with ugorji/go@b4c50a2

ugorji on 7 Apr 2018

@gyuho released https://github.com/ugorji/go/releases/tag/v1.1.1

ugorji on 9 Apr 2018

@ugorji Thanks!

Closing via https://github.com/coreos/etcd/pull/9494.

gyuho on 9 Apr 2018

Was this page helpful?

0 / 5 - 0 ratings