Etcd: Embedding etcd into an existing golang project ,And is a cluster.
I tried this example at https://godoc.org/github.com/coreos/etcd/embed and changed some parameters. I don't know how to build a distributed cluster. Can I have a more detailed example? I also read this https://github.com/etcd-io/etcd/issues/3670 question, I still don't know how to start. I want to know if I embed etcd in my program and form a three-node cluster.
offer365
All 8 comments
Maybe this will give you a better place to start try to emulate how we use embed etcd in functional tests with a more extensive Config example.
hexfusion
on 22 Mar 2019
The example showed how to embed and start a single etcd process. To form a cluster, you need to set some parameters such as --listen-peer-urls, --initial-cluster. If configured correctly, the etcd processes will be able to form a cluster automatically. An example configuration of 3-node cluster: https://github.com/etcd-io/etcd/blob/master/Procfile
jingyih
on 22 Mar 2019
Hello, I'm trying to setup a cluster using etcd embedded, I tried setting up the parameters as described above but even though I set the initialCluster value when the service starts the log shows it as empty. Would appreciate any help or configuration examples to set up a cluster using the embedded service
cfg := embed.NewConfig()
cfg.Dir = db.Path
cfg.Logger = "zap"
cfg.Debug = true
cfg.LogLevel = "info"
cfg.APUrls = parsePeers(db.Peers)
cfg.LPUrls = parsePeers([]string{"0.0.0.0"})
cfg.ACUrls = parseClients(db.Peers)
cfg.LCUrls = parseClients([]string{"0.0.0.0"})
cfg.InitialCluster = "name0=http://0.0.0.0:2380,name1=http://10.0.1.197:2380"
// cfg.LogOutputs = []string{db.Path + "/LOG"}
db.server, err = embed.StartEtcd(cfg)
{
"level": "info",
"ts": "2019-11-25T16:04:41.777+0800",
"caller": "embed/etcd.go:299",
"msg": "starting an etcd server",
"etcd-version": "3.5.0-pre",
"git-sha": "Not provided (use ./build instead of go build)",
"go-version": "go1.13",
"go-os": "windows",
"go-arch": "amd64",
"max-cpu-set": 12,
"max-cpu-available": 12,
"member-initialized": true,
"name": "default",
"data-dir": "data/default.etcd",
"wal-dir": "",
"wal-dir-dedicated": "",
"member-dir": "data\\default.etcd\\member",
"force-new-cluster": false,
"heartbeat-interval": "100ms",
"election-timeout": "1s",
"initial-election-tick-advance": true,
"snapshot-count": 100000,
"snapshot-catchup-entries": 5000,
"initial-advertise-peer-urls": [
"http://0.0.0.0:2380",
"http://10.0.1.197:2380"
],
"listen-peer-urls": [
"http://0.0.0.0:2380"
],
"advertise-client-urls": [
"http://0.0.0.0:2379",
"http://10.0.1.197:2379"
],
"listen-client-urls": [
"http://0.0.0.0:2379"
],
"listen-metrics-urls": [],
"cors": [
"*"
],
"host-whitelist": [
"*"
],
"initial-cluster": "",
"initial-cluster-state": "new",
"initial-cluster-token": "",
"quota-size-bytes": 2147483648,
"pre-vote": false,
"initial-corrupt-check": false,
"corrupt-check-time-interval": "0s",
"auto-compaction-mode": "",
"auto-compaction-retention": "0s",
"auto-compaction-interval": "0s",
"discovery-url": "",
"discovery-proxy": ""
}
benitogf
on 25 Nov 2019
after deleting the database and starting with the cluster configuration I'm getting a different error:
2019-11-25 18:05:21.378853 I | --initial-advertise-peer-urls has http://10.0.1.197:2380 but missing from --initial-cluster=idx0=http://0.0.0.0:2380,idx0=http://10.0.1.197:2380
But acording to the log the initial-cluster is:
"initial-cluster":"idx0=http://0.0.0.0:2380,idx1=http://10.0.1.197:2380"
and initial-advertise-peer-urls:
"initial-advertise-peer-urls":["http://0.0.0.0:2380","http://10.0.1.197:2380"]
Full crash log:
[WARNING] Deprecated '--debug' flag is set to true (use '--log-level=debug' instead
[WARNING] Deprecated '--debug' flag is set to true with inconsistent '--log-level=info' flag
{"level":"info","ts":"2019-11-25T18:05:21.365+0800","caller":"embed/etcd.go:117","msg":"configuring peer listeners","listen-peer-urls":["http://0.0.0.0:2380"]}
{"level":"info","ts":"2019-11-25T18:05:21.366+0800","caller":"embed/etcd.go:127","msg":"configuring client listeners","listen-client-urls":["http://0.0.0.0:2379"]}
{"level":"info","ts":"2019-11-25T18:05:21.367+0800","caller":"embed/etcd.go:299","msg":"starting an etcd server","etcd-version":"3.5.0-pre","git-sha":"Not provided (use ./build instead of go build)","go-version":"go1.13","go-os":"windows","go-arch":"amd64","max-cpu-set":12,"max-cpu-available":12,"member-initialized":false,"name":"idx0","data-dir":"data/default.etcd","wal-dir":"","wal-dir-dedicated":"","member-dir":"data\\default.etcd\\member","force-new-cluster":false,"heartbeat-interval":"100ms","election-timeout":"1s","initial-election-tick-advance":true,"snapshot-count":100000,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["http://0.0.0.0:2380","http://10.0.1.197:2380"],"listen-peer-urls":["http://0.0.0.0:2380"],"advertise-client-urls":["http://0.0.0.0:2379","http://10.0.1.197:2379"],"listen-client-urls":["http://0.0.0.0:2379"],"listen-metrics-urls":[],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"idx0=http://0.0.0.0:2380,idx1=http://10.0.1.197:2380","initial-cluster-state":"new","initial-cluster-token":"etcd-cluster","quota-size-bytes":2147483648,"pre-vote":false,"initial-corrupt-check":false,"corrupt-check-time-interval":"0s","auto-compaction-mode":"","auto-compaction-retention":"0s","auto-compaction-interval":"0s","discovery-url":"","discovery-proxy":""}
{"level":"info","ts":"2019-11-25T18:05:21.370+0800","caller":"etcdserver/backend.go:79","msg":"opened backend db","path":"data\\default.etcd\\member\\snap\\db","took":"997µs"}
{"level":"info","ts":"2019-11-25T18:05:21.377+0800","caller":"embed/etcd.go:360","msg":"closing etcd server","name":"idx0","data-dir":"data/default.etcd","advertise-peer-urls":["http://0.0.0.0:2380","http://10.0.1.197:2380"],"advertise-client-urls":["http://0.0.0.0:2379","http://10.0.1.197:2379"]}
{"level":"info","ts":"2019-11-25T18:05:21.378+0800","caller":"embed/etcd.go:364","msg":"closed etcd server","name":"idx0","data-dir":"data/default.etcd","advertise-peer-urls":["http://0.0.0.0:2380","http://10.0.1.197:2380"],"advertise-client-urls":["http://0.0.0.0:2379","http://10.0.1.197:2379"]}
2019-11-25 18:05:21.378853 I | --initial-advertise-peer-urls has http://10.0.1.197:2380 but missing from --initial-cluster=idx0=http://0.0.0.0:2380,idx0=http://10.0.1.197:2380
exit status 1
Can refer to my implementation https://github.com/offer365/example/tree/master/etcd/embedder
offer365
on 27 Nov 2019
@offer365 thanks for the example, I had was trying to include all the nodes in cfg.APUrls but it worked using only the actual node ip
a small example of some utilities that I used to handle the different formats and ports required by etcd so I'm able to start a cluster sending only the peers IPs:
func parsePeers(eps []string) []url.URL {
neps := make([]url.URL, len(eps))
for i, ep := range eps {
u, err := url.Parse("http://" + ep + ":2380")
if err != nil {
return []url.URL{}
}
neps[i] = *u
}
return neps
}
func parseClients(eps []string) []url.URL {
neps := make([]url.URL, len(eps))
for i, ep := range eps {
u, err := url.Parse("http://" + ep + ":2379")
if err != nil {
return []url.URL{}
}
neps[i] = *u
}
return neps
}
func parseClusterClients(eps []string) []string {
neps := make([]string, len(eps))
for i, ep := range eps {
u := "http://" + ep + ":2379"
neps[i] = u
}
return neps
}
func parseInitialCluster(eps []string) string {
neps := ""
for i, ep := range eps {
neps += "idx" + strconv.Itoa(i) + "=http://" + ep + ":2380"
if i+1 < len(eps) {
neps += ","
}
}
return neps
}
cfg := embed.NewConfig()
cfg.Name = "idx" + strconv.Itoa(peerPositionInList) // this param is the index of the current IP in the IPs list
cfg.Dir = dbPath
cfg.Logger = "zap"
cfg.Debug = true
cfg.LogLevel = "info"
cfg.APUrls = parsePeers([]string{"0.0.0.0"})
cfg.LPUrls = parsePeers([]string{"0.0.0.0"})
cfg.ACUrls = parseClients(dbPeers)
cfg.LCUrls = parseClients([]string{"0.0.0.0"})
cfg.InitialCluster = parseInitialCluster(db.Peers)
server, err = embed.StartEtcd(cfg)
client, err = clientv3.New(clientv3.Config{
Endpoints: parseClusterClients(db.Peers),
AutoSyncInterval: time.Second * 5,
DialTimeout: 5 * time.Second,
})
Would like to know how to add or remove peers from the cluster after its initialized, guess that this would require a discovery service but there's no documentation on that yet https://etcd.io/docs/v3.4.0/op-guide/dev-internal/discovery_protocol.md
benitogf
on 28 Nov 2019
I think cfg.APUrls, cfg.LPUrls are the urls of the current host broadcast and listening members, cfg.ACUrls, cfg.LCUrls are the urls of the current host broadcast and listening clients. cfg.InitialCluster is a member-owned address. If you want to add or remove peer nodes from the cluster after initialization, server, err = embed.StartEtcd (cfg). Maybe the server has the method you want, you look for it。
我认为 cfg.APUrls，cfg.LPUrls，是当前主机广播和监听成员的url,cfg.ACUrls ，cfg.LCUrls 是当前主机广播和监听客户端的url。cfg.InitialCluster 是成员所有的地址。如果你想在初始化后从集群中添加或删除对等节点，server, err = embed.StartEtcd(cfg)。可能 server 中有你想要的方法，你找找看
------------------ 原始邮件 ------------------
发件人: "benito"<[email protected]>;
发送时间: 2019年11月28日(星期四) 中午1:43
收件人: "etcd-io/etcd"<[email protected]>;
抄送: "李涛"<[email protected]>;"Mention"<[email protected]>;
主题: Re: [etcd-io/etcd] Embedding etcd into an existing golang project ,And is a cluster. (#10554)
@offer365 thanks for the example, I had was trying to include all the nodes in cfg.APUrls but it worked using only the actual node ip
a small example of some utilities that I used to handle the different formats and ports required by etcd so I'm able to start a cluster sending only the peers IPs:
func parsePeers(eps []string) []url.URL { neps := make([]url.URL, len(eps)) for i, ep := range eps { u, err := url.Parse("http://" + ep + ":2380") if err != nil { return []url.URL{} } neps[i] = *u } return neps } func parseClients(eps []string) []url.URL { neps := make([]url.URL, len(eps)) for i, ep := range eps { u, err := url.Parse("http://" + ep + ":2379") if err != nil { return []url.URL{} } neps[i] = *u } return neps } func parseClusterClients(eps []string) []string { neps := make([]string, len(eps)) for i, ep := range eps { u := "http://" + ep + ":2379" neps[i] = u } return neps } func parseInitialCluster(eps []string) string { neps := "" for i, ep := range eps { neps += "idx" + strconv.Itoa(i) + "=http://" + ep + ":2380" if i+1 < len(eps) { neps += "," } } return neps } cfg := embed.NewConfig() cfg.Name = "idx" + strconv.Itoa(peerPositionInList) // this param is the index of the current IP in the IPs list cfg.Dir = dbPath cfg.Logger = "zap" cfg.Debug = true cfg.LogLevel = "info" cfg.APUrls = parsePeers([]string{"0.0.0.0"}) cfg.LPUrls = parsePeers([]string{"0.0.0.0"}) cfg.ACUrls = parseClients(dbPeers) cfg.LCUrls = parseClients([]string{"0.0.0.0"}) cfg.InitialCluster = parseInitialCluster(db.Peers) server, err = embed.StartEtcd(cfg) client, err = clientv3.New(clientv3.Config{ Endpoints: parseClusterClients(db.Peers), AutoSyncInterval: time.Second * 5, DialTimeout: 5 * time.Second, })
Would like to know how to add or remove peers from the cluster after its initialized, guess that this would require a discovery service but there's no documentation on that yet https://etcd.io/docs/v3.4.0/op-guide/dev-internal/discovery_protocol.md
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
offer365
on 28 Nov 2019
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] picture](https://avatars3.githubusercontent.com/in/1724?v=4&s=40)
stale[bot]
on 6 Apr 2020
Related issues
itnikita
·
3Comments
zhousoft
·
3Comments
govine
·
3Comments
kghost
·
4Comments
invidian
·
3Comments