Etcd: go get error for 3.4.0

[vtolstov]

go get go.etcd.io/[email protected]
go: finding golang.org/x/net master
go: finding go.etcd.io/etcd v3.4.0
go: finding go.etcd.io/etcd v3.4.0
go get go.etcd.io/[email protected]: go.etcd.io/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v3

[fiorix]

Having the same issue with v3.4.1.

[jborlum]

Same here. Would be great to have this fixed. It has been an issue since January.

[vitaminmoo]

• @latest is v3.3.15+incompatible according to go get for golang 1.13.1
• explicit @v3.4.1 doesn't work due to the module path in that version being go.etcd.io/etcd instead of go.etcd.io/etcd/v3 (I think)
• Using go get -u go.etcd.io/etcd/clientv3@a14579f (the sha1 of the v3.4.1 release) results in usage of v0.5.0-alpha.5.0.20190917205325-a14579fbfb1a with has the right commit at the end, but a wacky old version (I believe it gets the right code)
• Using go get -u go.etcd.io/etcd/[email protected] gets the right commit, but with a less wrong tag than above

Full reproduction:

graham@graham:~/tmp/etcd-mod$ go version
go version go1.13.1 linux/amd64
graham@graham:~/tmp/etcd-mod$ cat main.go 
package main

import "go.etcd.io/etcd/clientv3"

func main() { clientv3.New(clientv3.Config{}) }
graham@graham:~/tmp/etcd-mod$ go mod init etcd-mod
go: creating new go.mod: module etcd-mod
graham@graham:~/tmp/etcd-mod$ go mod tidy
go: finding golang.org/x/crypto latest
go: finding github.com/coreos/pkg latest
go: finding github.com/coreos/go-systemd latest
go: finding github.com/xiang90/probing latest
go: finding golang.org/x/time latest
go: finding github.com/tmc/grpc-websocket-proxy latest
go: finding github.com/golang/groupcache latest
graham@graham:~/tmp/etcd-mod$ cat go.mod 
module etcd-mod

go 1.13

require (
    github.com/coreos/bbolt v1.3.3 // indirect
    github.com/coreos/etcd v3.3.15+incompatible // indirect
    github.com/coreos/go-semver v0.3.0 // indirect
    github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f // indirect
    github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f // indirect
    github.com/dgrijalva/jwt-go v3.2.0+incompatible // indirect
    github.com/golang/groupcache v0.0.0-20191002201903-404acd9df4cc // indirect
    github.com/google/btree v1.0.0 // indirect
    github.com/google/uuid v1.1.1 // indirect
    github.com/gorilla/websocket v1.4.1 // indirect
    github.com/grpc-ecosystem/go-grpc-middleware v1.1.0 // indirect
    github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
    github.com/grpc-ecosystem/grpc-gateway v1.11.3 // indirect
    github.com/jonboulle/clockwork v0.1.0 // indirect
    github.com/prometheus/client_golang v1.1.0 // indirect
    github.com/soheilhy/cmux v0.1.4 // indirect
    github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5 // indirect
    github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 // indirect
    go.etcd.io/bbolt v1.3.3 // indirect
    go.etcd.io/etcd v3.3.15+incompatible
    go.uber.org/multierr v1.2.0 // indirect
    golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc // indirect
    golang.org/x/time v0.0.0-20190921001708-c4c64cad1fd0 // indirect
    google.golang.org/grpc v1.24.0 // indirect
    sigs.k8s.io/yaml v1.1.0 // indirect
)
graham@graham:~/tmp/etcd-mod$ go get -u go.etcd.io/etcd/[email protected]
go: finding go.etcd.io v3.4.1
go: finding go.etcd.io/etcd/clientv3 v3.4.1
go: finding go.etcd.io/etcd v3.4.1
go: finding go.etcd.io/etcd v3.4.1
go: finding go.etcd.io/etcd/clientv3 v3.4.1
go get go.etcd.io/etcd/[email protected]: go.etcd.io/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v3
graham@graham:~/tmp/etcd-mod$ go get -u go.etcd.io/etcd/clientv3@a14579f
go: finding go.etcd.io a14579f
go: finding go.etcd.io/etcd a14579f
go: finding go.etcd.io/etcd/clientv3 a14579f
go: finding github.com/coreos/go-systemd latest
go: finding golang.org/x/sys latest
go: finding github.com/coreos/pkg latest
go: finding golang.org/x/net latest
go: finding google.golang.org/genproto latest
graham@graham:~/tmp/etcd-mod$ cat go.mod 
module etcd-mod

go 1.13

require (
    github.com/coreos/bbolt v1.3.3 // indirect
    github.com/coreos/etcd v3.3.15+incompatible // indirect
    github.com/coreos/go-semver v0.3.0 // indirect
    github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f // indirect
    github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f // indirect
    github.com/gogo/protobuf v1.3.0 // indirect
    github.com/golang/groupcache v0.0.0-20191002201903-404acd9df4cc // indirect
    github.com/google/uuid v1.1.1 // indirect
    github.com/gorilla/websocket v1.4.1 // indirect
    github.com/grpc-ecosystem/go-grpc-middleware v1.1.0 // indirect
    github.com/grpc-ecosystem/grpc-gateway v1.11.3 // indirect
    github.com/prometheus/client_golang v1.1.0 // indirect
    github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5 // indirect
    go.etcd.io/etcd v0.5.0-alpha.5.0.20190917205325-a14579fbfb1a
    go.uber.org/multierr v1.2.0 // indirect
    golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc // indirect
    golang.org/x/net v0.0.0-20191003171128-d98b1b443823 // indirect
    golang.org/x/sys v0.0.0-20191003212358-c178f38b412c // indirect
    golang.org/x/text v0.3.2 // indirect
    golang.org/x/time v0.0.0-20190921001708-c4c64cad1fd0 // indirect
    google.golang.org/genproto v0.0.0-20191002211648-c459b9ce5143 // indirect
    google.golang.org/grpc v1.24.0 // indirect
)

[havardk]

FWIW, there was an attempt to change the import paths in https://github.com/etcd-io/etcd/pull/10640 but it was reverted in https://github.com/etcd-io/etcd/pull/10766.

[moggle-mog]

the same, when I use go get -u, I get:

go get: github.com/coreos/[email protected]+incompatible updating to
    github.com/coreos/[email protected]+incompatible: parsing go.mod:
    module declares its path as: go.etcd.io/etcd
            but was required as: github.com/coreos/etcd

go.mod has invalid module name

[tehsphinx]

the same, when I use go get -u, I get:

go get: github.com/coreos/[email protected]+incompatible updating to
  github.com/coreos/[email protected]+incompatible: parsing go.mod:
  module declares its path as: go.etcd.io/etcd
          but was required as: github.com/coreos/etcd

go.mod has invalid module name

What was the path you tried to go get -u? github.com/coreos/etcd or go.etcd.io/etcd?

[moggle-mog]

go get -u github.com/coreos/etcd

------------------ 原始邮件 ------------------
发件人: "Gabriel Nelle"[e-mail];
发送时间: 2019年10月8日(星期二) 下午4:23
收件人: "etcd-io/etcd"[e-mail];
抄送: "Rinker"[e-mail]; "Comment"[e-mail];
主题: Re: [etcd-io/etcd] go get error for 3.4.0 (#11154)

the same, when I use go get -u, I get:
go get: github.com/coreos/[email protected]+incompatible updating to github.com/coreos/[email protected]+incompatible: parsing go.mod: module declares its path as: go.etcd.io/etcd but was required as: github.com/coreos/etcd

go.mod has invalid module name

That was the path you tried to go get -u? github.com/coreos/etcd or go.etcd.io/etcd?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[halseth]

Seeing same issue.

[tehsphinx]

@moggle-mog: have you tried with go.etcd.io/etcd?

Just curious. I worked around it by using go get -u go.etcd.io/etcd/[email protected] for now.

[moggle-mog]

@tehsphinx Sorry, unavailable again. Because of framework limit, I must use github.com/coreos/etcd as import path

[fiorix]

Is there a resolution path for this issue? Is there any option other than adding the /v3 suffix to go.mod, and updating all import paths to go.etcd.io/etcd/v3 ?

This change should work for anyone using modules or not, so long they use a Go compiler 1.11 or newer. The part I don't understand is how the old coreos import is affected by this change.

[fiorix]

Tried adding /v3 and /v4 and look for errors in travis - binaries like etcd-dump-logs failed stuck on the incompatible version.

[jefferai]

Can you either please remove the go.mod from the project or fix it?

If you are exposing a go module, you should be following its rules. That means that your go.mod needs to expose the actual major version as the module -- go.etcd.io/etcd/v3.

The behavior right now is that go modules, despite you having a go.mod, decides that your repo is incompatible and gets the latest tag made, regardless of which branch that tag actually points to. And the only way to get released tags with go modules is to get the commit ID and explicitly use that.

If you changed that and Travis broke, please fix Travis. It is unsurprising there would need to be changes since it means that you will have to update import paths. But that is trivial to do, it will make go modules actually work correctly, and it has to be done at some point anyways.

[obsider]

Here is a temporary workaround (change the v3.4.3 to the tag/release you want)

git clone --branch v3.4.3 https://github.com/etcd-io/etcd.git
cd etcd
echo -n v0.0.0-
TZ=UTC git --no-pager show \
  --quiet \
  --abbrev=12 \
  --date='format-local:%Y%m%d%H%M%S' \
  --format="%cd-%h"

Take note of the v0.0.0-... (v0.0.0-20191023171146-3cf2f69b5738 for tag v3.4.3 for example) and change your go.mod etcd import like this

go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738

It should now work. thanks to https://stackoverflow.com/a/59440771 for the tip on how to compute what's after v0.0.0-

[horkhe]

Hey guys, is there at least a tentative ETA on when this is going to be fixed?

[terorie]

Please delete go.mod, it's invalid and more harmful for imports than just omitting it.

_EDIT_

Unfortunately removal of go.mod is not a bulletproof solution either. Quite a few third party packages pull in v3.4.* using a workaround (https://github.com/etcd-io/etcd/pull/11477, great work by obsider). While most imports seem to refer to a specific commit, there is still a risk of breaking one of those workaround imports. One could argue that breaking a workaround is a lesser evil than fixing a bad import bug.

Still, afaik there are only two _correct_ approaches:

• Set correct module path in go.mod and tag a new release: module go.etcd.io/etcd/v3. Ideally test this using a fake GOPROXY first.
• Remove go mod entirely, and rely on legacy/fallback import.

[fredczj]

Still doesn't work:

go get go.etcd.io/etcd/[email protected]: go.etcd.io/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v3

[jefferai]

@fredczj The maintainers aren't paying attention and don't care anyways.

They could solve this issue in two seconds by removing the go.mod entirely but instead they leave an invalid file and cause problems for every single downstream project, and it's been that way for six months.

Alternately they could fix this by fixing the import path in go.mod, creating a tag with the new go.mod so that it's actually fetchable (which #11221 failed to do), and fix their CI. More work, but better rewards.

We're all just screaming into the void at this point, though.

[fredczj]

For those who cannot wait so do I, I have forked the project and done what @jefferai said.
It is very bad and ugly as I won't maintain this (ofc), but for prototyping, this may help:
go get github.com/fredczj/etcd/[email protected]

[terorie]

@fredczj That's a good idea. Do you think it is possible to use the replace directive to leave import paths in the sources untouched?

[fredczj]

Yes @terorie , this works:
in go.mod:
replace github.com/etcd-io/etcd => github.com/fredczj/etcd v3.4.6+incompatible

in your .go:

import ( ...
    "github.com/etcd-io/etcd/clientv3"
...)

then

    cli, err := clientv3.New(clientv3.Config{
        Endpoints:   []string{"http://127.0.0.1:2379"},
        DialTimeout: 5 * time.Second,
    })...

[terorie]

@fredczj Sorry, I should have formulated that clearer :)
I meant, do you think it's possible to make use of replace in your fork to leave the imports untouched and reduce the size of the diff?
(particularly https://github.com/fredczj/etcd/commit/f93b8113b9e9140f89ea15abb98c147f3955d5fb)
That should make keeping track with upstream easier (less merge conflicts) for people looking for long-term adoption of your fork.

[jefferai]

@fredczj It shouldn't be marked incompatible if everything was done correctly.

[fredczj]

@fredczj It shouldn't be marked incompatible if everything was done correctly.

I just removed the go.mod file, which made de +incompatible suffix come.

[fredczj]

@fredczj Sorry, I should have formulated that clearer :)
I meant, do you think it's possible to make use of replace in your fork to leave the imports untouched and reduce the size of the diff?
(particularly fredczj@f93b811)
That should make keeping track with upstream easier (less merge conflicts) for people looking for long-term adoption of your fork.

It is not so easy if we want to keep the v3xxx version.

Basically, with regard to https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning , the final directory should be named v3 to match version v3.x.y.
Here in the project, we have 'etcd' as main, then 'clientv3' (and even not 'v3').
Then, creating a module in clientv3 is not so easy, as the sub directories are pointing sometimes on go.etcd.io, sometimes on github.com/coreos... It is not so straight to indicate which sub to write for a good replace.
To clean this, I guess there is some clean-up to do that will be painful to merge.

Only the solution of @jefferai to remove the go.mod helps go use quickly the clientv3 resources available in etcd project at the moment.

[jefferai]

the final directory should be named v3 to match version v3.x.y.

This is incorrect. The problem is your fork makes literally the same mistake that this repository does.

[fredczj]

the final directory should be named v3 to match version v3.x.y.

This is incorrect. The problem is your fork makes literally the same mistake that this repository does.

I don't get it sorry; I'm probably too new to golang.

If I set v3 as you stated earlier, in the module name, go look for a v3 directory.
missing github.com/fredczj/etcd/go.mod and .../v3/go.mod at revision v3.4.6

Would this mean I actually need a v3 directory in which putting all the etcd project?
I could also change the clientv3 name into v3, but all this would change a few paths, what @terorie was looking to work around...

Am I missing something else?

[jefferai]

Your go.mod module needs to be module github.com/fredczj/etcd/v3.

Aside from that, internally you need to ensure that all of your import lines are referencing the new package name, including the /v3/ before the rest of the path. Once all that's working, you need to tag it.

This is one of those times when it would be nice if Go modules supported patch versions so that you could have patch tags on top of the real version (v3.4.4) instead of making up your own tag versions that will eventually conflict with what's in this repo.

[iamtakingiteasy]

this is still an issue

[jefferai]

Sure is

[horkhe]

Well, at least now we have a corona virus to blame for the delay.

[mritd]

When will this problem be solved? It wasted me at least an hour to deal with dependency issues!

[vtolstov]

@philips ?

[win5do]

@philips ?

(⊙o⊙)…

[vtolstov]

may be for etcd this is beginning of the end

[philips]

Let me look into this.

I have spent the last 9 months trying to train/empower others to take ownership of infrastructure I have owned throughout the kube ecosystem. And unfortunately part of that has meant I haven’t spent much time focused on any one project.

But, this issue seems to still be falling through the cracks for months. So, I will try to figure it out.

[vtolstov]

@philips thank you! i'm believe that this is simple

[philips]

This will be fixed in etcd v3.5.0 via #11823

[vtolstov]

Thanks, we really need it. Do you have info, when the release?

[terorie]

@vtolstov Refer to the milestone for release blockers https://github.com/etcd-io/etcd/milestone/37

[xkeyideal]

go mod why github.com/coreos/etcd  2 ↵  6906  17:58:16
go: finding module for package github.com/grpc-ecosystem/go-grpc-prometheus
go: finding module for package github.com/spf13/pflag
go: finding module for package github.com/spf13/cobra
go: finding module for package github.com/soheilhy/cmux
go: finding module for package github.com/prometheus/client_golang/prometheus
go: finding module for package sigs.k8s.io/yaml
go: finding module for package github.com/jonboulle/clockwork
go: finding module for package github.com/prometheus/client_golang/prometheus/promhttp
go: finding module for package github.com/grpc-ecosystem/go-grpc-middleware
go: finding module for package golang.org/x/time/rate
go: finding module for package github.com/xiang90/probing
go: finding module for package github.com/coreos/bbolt
go: finding module for package github.com/golang/groupcache/lru
go: finding module for package github.com/grpc-ecosystem/grpc-gateway/runtime
go: finding module for package github.com/tmc/grpc-websocket-proxy/wsproxy
go: finding module for package github.com/dgrijalva/jwt-go
go: finding module for package github.com/grpc-ecosystem/grpc-gateway/utilities
go: finding module for package github.com/google/btree
go: finding module for package github.com/coreos/go-systemd/v22/internal/dlopen
go: found github.com/grpc-ecosystem/go-grpc-prometheus in github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: found github.com/soheilhy/cmux in github.com/soheilhy/cmux v0.1.4
go: found github.com/spf13/cobra in github.com/spf13/cobra v1.0.0
go: found sigs.k8s.io/yaml in sigs.k8s.io/yaml v1.2.0
go: found github.com/jonboulle/clockwork in github.com/jonboulle/clockwork v0.1.0
go: found github.com/grpc-ecosystem/grpc-gateway/runtime in github.com/grpc-ecosystem/grpc-gateway v1.14.4
go: found github.com/tmc/grpc-websocket-proxy/wsproxy in github.com/tmc/grpc-websocket-proxy v0.0.0-20200427203606-3cfed13b9966
go: found github.com/prometheus/client_golang/prometheus in github.com/prometheus/client_golang v1.6.0
go: found github.com/spf13/pflag in github.com/spf13/pflag v1.0.5
go: found golang.org/x/time/rate in golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1
go: found github.com/coreos/go-systemd/v22/internal/dlopen in github.com/coreos/go-systemd/v22 v22.0.0
go: found github.com/grpc-ecosystem/go-grpc-middleware in github.com/grpc-ecosystem/go-grpc-middleware v1.2.0
go: found github.com/xiang90/probing in github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: found github.com/dgrijalva/jwt-go in github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: found github.com/google/btree in github.com/google/btree v1.0.0
go: found github.com/coreos/bbolt in github.com/coreos/bbolt v1.3.4
go: found github.com/golang/groupcache/lru in github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
go: github.com/coreos/etcd imports
github.com/coreos/etcd/etcdmain imports
github.com/coreos/etcd/etcdserver imports
github.com/coreos/etcd/mvcc/backend imports
github.com/coreos/bbolt: github.com/coreos/[email protected]: parsing go.mod:
module declares its path as: go.etcd.io/bbolt
but was required as: github.com/coreos/bbolt

[tehsphinx]

The correct import path for bbolt is go.etcd.io/bbolt. It seems to imported incorrectly somewhere.

[krisai]

asdsadad

[Awezome]

it works when add this to go.mod file
replace go.etcd.io/etcd => go.etcd.io/etcd v0.0.0-20200520232829-54ba9589114f

and wait for v3.5 ...

[maja42]

Why is this issue closed? Is there a new one we can use for tracking?

$ go get go.etcd.io/etcd/v3
go get go.etcd.io/etcd/v3: module go.etcd.io/etcd@upgrade found (v2.3.8+incompatible), but does not contain package go.etcd.io/etcd/v3

It's still not possible to "go get" this library when using modules, although this issue exists for a year now.
And the workaround from @vitaminmoo (Okt.2019) do not work for me since I also depend on the new protobuf version google.golang.org/protobuf v1.25.0 (which is the reason I think).

Both

go get -u go.etcd.io/etcd/clientv3@a14579f 
go get -u go.etcd.io/etcd/[email protected] 

result in this error

# go.etcd.io/etcd/clientv3/balancer/resolver/endpoint
../../../../pkg/mod/go.etcd.io/[email protected]/clientv3/balancer/resolver/endpoint/endpoint.go:112:78: undefined: resolver.BuildOption
../../../../pkg/mod/go.etcd.io/[email protected]/clientv3/balancer/resolver/endpoint/endpoint.go:180:31: undefined: resolver.ResolveNowOption
# go.etcd.io/etcd/clientv3/balancer/picker
../../../../pkg/mod/go.etcd.io/[email protected]/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions
../../../../pkg/mod/go.etcd.io/[email protected]/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions

Using the newest commit hash causes yet another issue:

$ go get -u go.etcd.io/etcd/v3@844091dda3a549cd81ff83dcc1396c9ebe7bb6c2
go: finding module for package google.golang.org/grpc/naming
../../../../pkg/mod/go.etcd.io/etcd/[email protected]/clientv3/naming/grpc.go:25:2: module google.golang.org/grpc@latest found (v1.31.0), but does not contain package google.golang.org/grpc/naming

Right now, my go.mod file contains go.etcd.io/etcd/v3 v3.3.0-rc.0.0.20200811185847-844091dda3a5 (which is the same as using the redirect @Awezome suggested), but I can't compile because I get the following issue:

$ go mod tidy
go: finding module for package google.golang.org/grpc/naming
<my package> imports
        go.etcd.io/etcd/v3/clientv3 tested by
        go.etcd.io/etcd/v3/clientv3.test imports
        go.etcd.io/etcd/v3/integration imports
        go.etcd.io/etcd/v3/proxy/grpcproxy imports
        google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.31.0), but does not contain package google.golang.org/grpc/naming

[maja42]

Is this related to https://github.com/etcd-io/etcd/issues/12124?

I got it running with the workaround posted there:

require (   
    go.etcd.io/etcd v0.5.0-alpha.5.0.20200518003842-9bad82fee544
    google.golang.org/protobuf v1.25.0
)
replace google.golang.org/grpc => google.golang.org/grpc v1.26.0

The rewrite is required in my own library, as well as every other module/application that depends on my library.

(go get go.etcd.io/etcd/v3 still doesn't work)

[jalavosus]

This issue should definitely not be closed, unless another issue (which I haven't seen referenced...) does a better job of explaining the issue or has better conversation.

Why the maintainers of this repo -- a repo with functionality needed by many developers -- have been dragging their feet on properly implementing Go modules support and actually, yaknow, _maintaining their library_ is beyond me.

[philips]

I understand this is broken in v3.4.z. We, like most of the early adopter Go projects, are put in a tough spot with this migration. As a rule we don’t backport non-bug fixes to the previously released minor versions and if we “fix” this we break everyone who has been using etcd v3.4 (released April 2019) since before Go changed this default.

This is closed because it is fixed in the master branch (since https://github.com/etcd-io/etcd/issues/11154#issuecomment-620886957) and when v3.5 comes out it will (hopefully) be fixed.

I understand it is frustrating but we are in a position of not making anyone happy. So, please be considerate and kind when commenting here.