Etcd: go.mod:incorrect pacakge dependency in etcd 3.3 without go.mod

[cfc4n]

shell:

cfc4n@cnxct-mt:~/Project/golang $mkdir etcd_3.3_mod
cfc4n@cnxct-mt:~/Project/golang $cd etcd_3.3_mod
cfc4n@cnxct-mt:~/Project/golang $touch main.go

and type code with:

package main

import (
    "github.com/coreos/etcd/clientv3" //equal to go.etcd.io/etcd/clientv3
    "time"
)

func main() {
    cfg := clientv3.Config{
        Endpoints:        []string{"https://localhost:2379"},
        DialTimeout:      time.Second * 5,
    }

    _,e := clientv3.New(cfg)
    if e != nil {
        panic(e)
    }
}

condition 1

cfc4n@cnxct-mt:~/Project/golang/etcd_3.3_mod $go mod init etcd_3.3_mod
go: creating new go.mod: module etcd_3.3_mod
cfc4n@cnxct-mt:~/Project/golang/etcd_3.3_mod $go build -o test
go: finding github.com/coreos/pkg latest
go: finding github.com/coreos/go-systemd latest
# github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
../../../../gopath/pkg/mod/github.com/coreos/[email protected]+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:114:78: undefined: resolver.BuildOption
../../../../gopath/pkg/mod/github.com/coreos/[email protected]+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:182:31: undefined: resolver.ResolveNowOption
# github.com/coreos/etcd/clientv3/balancer/picker
../../../../gopath/pkg/mod/github.com/coreos/[email protected]+incompatible/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions
../../../../gopath/pkg/mod/github.com/coreos/[email protected]+incompatible/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions

condition 2

error logs:

cfc4n@cnxct-mt:~/Project/golang/etcd_3.3_mod $go mod init etcd_3.3_mod
go: creating new go.mod: module etcd_3.3_mod
cfc4n@cnxct-mt:~/Project/golang/etcd_3.3_mod $go get go.etcd.io/etcd
go: finding github.com/coreos/pkg latest
go: finding golang.org/x/time latest
go: finding golang.org/x/net latest
go: finding github.com/golang/groupcache latest
go: finding github.com/coreos/go-systemd latest
go: finding github.com/tmc/grpc-websocket-proxy latest
go: finding github.com/xiang90/probing latest
go: finding google.golang.org/genproto latest
go: finding golang.org/x/crypto latest
go: github.com/coreos/[email protected]: parsing go.mod: unexpected module path "go.etcd.io/bbolt"
go: error loading module requirements

rootcause

There is not go.mod in https://github.com/etcd-io/etcd/tree/v3.3.22 . via https://pkg.go.dev/go.etcd.io/etcd/?tab=doc .
go mod will get lastest version package default.

package | ver in client used clientv3 lib | ver in etcd
------------------------------ | -------------------------- | --------------------------
github.com/coreos/etcd | v3.3.22+incompatible // indirect |
github.com/coreos/go-semver | v0.3.0 // indirect | v0.2.0
github.com/coreos/go-systemd | v0.0.0-20191104093116-d3cd4ed1dbcf // indirect | v0.0.0-20190620071333-e64a0ec8b42a
github.com/coreos/pkg | v0.0.0-20180928190104-399ea9e2e55f // indirect | v0.0.0-20180108230652-97fdf19511ea
github.com/gogo/protobuf | v1.3.1 // indirect | v1.2.1
github.com/google/uuid | v1.1.1 // indirect | v1.0.0
go.etcd.io/etcd | v3.3.22+incompatible // indirect |
go.uber.org/zap | v1.15.0 // indirect | v1.10.0
google.golang.org/grpc | v1.30.0 // indirect | v1.23.0

If we build project in etcd repo ,it works. because go mod will translate package version from glide.lock with correct version.
but if used clientv3 sdk ,it will failed ,go mod will not translate package for indirect package .and error logs with this issue.

other issue

12028

12009

11931

11829

11808

11772

11749

11721

11707

and other complain : https://colobu.com/2020/04/09/accidents-of-etcd-and-go-module/ 🤣

solution

• add go.mod and go.sum into etcd 3.3 repo for GO111MODULE=on ( build this project and go get go.etcd.io/etcd for other project )
• keep use glide.lock and glide.yaml for GO111MODULE=off

/cc @gyuho @mitake @spzala

[cfc4n]

test and verify:

exec cmd: go get go.etcd.io/etcd@cdc1c8f02

cfc4n@cnxct-mt:~/Project/golang/etcd_3.3_mod $cat go.mod
module etcd_3.3_mod

go 1.12

require go.etcd.io/etcd v0.0.0-20200625034214-cdc1c8f02ff1
cfc4n@cnxct-mt:~/Project/golang/etcd_3.3_mod $

It was fixed.

@gyuho Can you release a new version for this issue?
Must of people use 3.3.22 now .
ps: ChangeLog PR:https://github.com/etcd-io/etcd/pull/12079

[bozaro]

Still broken in etcd 3.4: https://github.com/etcd-io/etcd/issues/12109

[ptabor]

Shouldn't we be concerned that this change is introducing inconsistent approaches between branches:

A) release-3.2 -> legacy - no go.mod at all

B) release-3.3. -> was no go.mod (as 3.2), but this PR introduces >> module go.etcd.io/etcd
<< that is effectively violating go versioning rule [1] and creating misleading experience.

Moreover release-3.3. still using github.com/coreos/etcd package naming, so the module name
go.etcd.io/etcd is incompatible.

C) release-3.4 -> Is using module go.etcd.io/etcd violating the go versioning rules.

D) release-3.5 in particular with #11820 introduces:

• proper module name "go.etcd.io/etcd/v3"
• change of import paths (#11820) such that they follow the convention.

State A) is proper legacy state (+incompatible) according to go modules:

# go get go.etcd.io/[email protected]

go: go.etcd.io/etcd v3.2.22 => v3.2.22+incompatible

State D) is proper go.mod state.

# go get go.etcd.io/etcd/v3@master

works and resolves to

go.etcd.io/etcd/v3 v3.3.0-rc.0.0.20200701155350-15884e908549

The referenced https://github.com/etcd-io/etcd/commit/15884e90854981494e6889bdb663439612b547a7 is fresh commit.
The awkward v3.3.0 is due to: f7a395f0308d6c921937b3b5d7c716ffb8018e9e that is the last tagging on the branch.
We might consider tagging this branch as v3.5.0-pre to mitigate this misleading resolution.

State B) & D) are not supported and causing problems for tools:

# go get go.etcd.io/etcd/[email protected]

go get go.etcd.io/etcd/[email protected]: go.etcd.io/etcd/[email protected]: invalid version: go.mod has non-.../v3 module path
"go.etcd.io/etcd" (and .../v3/go.mod does not exist) at revision v3.4.9

# go get go.etcd.io/[email protected]

go get go.etcd.io/[email protected]: go.etcd.io/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v3

# go get go.etcd.io/[email protected]

go get go.etcd.io/[email protected]+incompatible: go.etcd.io/[email protected]+incompatible: invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required

---

Some workaround is to do:

# go get go.etcd.io/[email protected]

That leads to awkward binding of:

go: go.etcd.io/etcd release-3.4 => v0.5.0-alpha.5.0.20200626183007-99e893d28560
go: downloading go.etcd.io/etcd v0.5.0-alpha.5.0.20200626183007-99e893d28560

Options forward

A) Revert this change and preserve the supported legacy behavior of 3.2 on 3.3 branch.

B) Backport #11820 to 3.3. & 3.4 patch, changing the module name to: go.etcd.io/etcd/v3 and import paths.
Its a very big change for patch release (formally disallowed)

C) (might by combined with A) Accelerate release of 3.5.0 branch to give customers a proper go-mod compatible dependency.
Prior to 3.5.0 release I would consider moving etcdv3client to a separate go module, as majority of customers want to depend on the go client alone and not on whole etcd implementation.

[cfc4n]

About option A, If revert this change ,The problem still appears like tihs issue. some package dependency incorrect..

In release-3.3 ,the module name still was go.etcd.io/etcd in go.mod after run go get github.com/coreos/etcd/clientv3 without this PR.and same as with this PR . So I can't understand the difference between with them in release-3.3

by the way,I am also confused module name with go.etcd.io/etcd and go.etcd.io/etcd/v3.

@ptabor

[ptabor]

Have you tried playing with replace() [1] in your go.mod ?

Kubernetes is using:

replace (
github.com/coreos/bbolt => github.com/coreos/bbolt v1.3.2
github.com/coreos/etcd => github.com/coreos/etcd v3.3.10+incompatible
)

While in theory, following directives would be a cleaner representation:

replace (
github.com/coreos/bbolt => go.etcd.io/bbolt
github.com/coreos/etcd => go.etcd.io/etcd
)

[1] https://golang.org/ref/mod#tmp_15

[cfc4n]

Yeah,I know it. But use go mod replace is an extra operation. and many people do not know how to solve this who encounter this problem.

[xiegeo]

The proper way is to release a new v4 for go.mod. For any version > 1, pre and post go.mod are not compatible. Any workaround such as replace just breaks go get for that project and causes more problems down the road.

[ptabor]

George: I assume that the plan is that at the time etcd will release: go.etcd.io/etcd/v3 (that represents branch 3.5) that is proper v3 module that holds the package compatibility rule. So all the etcd/v3 packages will be compatible... they will just start late in the history. I don't think we can afford major version (v4) any time soon.

CFC4N: Extra steps are unfortunate, but they can be solved for 3.2 & 3.3 by proper documentation. I think current change grows the complexity.

[cfc4n]

CFC4N: Extra steps are unfortunate, but they can be solved for 3.2 & 3.3 by proper documentation. I think current change grows the complexity.

I don't think so. and I think this change makes it easy to use client v3.

[ldelossa]

@hexfusion is there any focus on fixing these import issues? I am trying to incorporate ectd into several projects and this is becoming pretty painful.

[mohdahmad17]

@ldelossa are you using a workaround other than using a fork?

[ldelossa]

@mohdahmad17 im importing via the latest commit hash directly.

[bcmills]

@ptabor, I notice that there are not currently any v1.*.* tags in the repo. That opens up another possibility (E) for users on Go 1.14 or higher, enabled by https://golang.org/doc/go1.14#incompatible-versions.

Option (E) is: change the module declaration back to module go.etcd.io/etcd and start tagging releases as v1.*.* (either v1.0.0, or perhaps v1.5.0 to reflect the discontinuity between v3.4.0 and the new version). Then users can continue to use the unversioned import path, which de facto already indicates the v3 API.

[ptabor]

@bcmills That would be my preference.

I would tag the unreleased branch (master) as 0.3.2, 0.3.3, 0.3.4 (or 0.32.0, 0.33.0, 0.34.0) to indicate no promise of backward compatibility between them,
still keeping it visually possible to recognize whether its etcd 3.2 or 3.3. or 3.4.

[xiegeo]

@ptabor By the way you can use "-" to indecate pre-release versions

You can specify pre-release versions by appending a hyphen and dot separated identifiers (for example, v1.0.1-alpha or v2.2.2-beta.2). Normal releases are preferred by the go command over pre-release versions, so users must ask for pre-release versions explicitly (for example, go get example.com/[email protected]) if your module has any normal releases.

https://blog.golang.org/publishing-go-modules