Etcd: Export 'snapshot restore' logic

[Quentin-M]

The logic to restore a snapshot is currently located in the command package, unexported. That's over 200 lines of logic right there.

While most of the features that etcd offers can be used directly from Go - including making snapshots and starting embed servers - restoring snapshot cannot. This means that for that very feature only, one must shell out to the etcdctl binary, which means one must have that binary on disk in the first place. It also means the snapshot to restore must be sync'd to disk first, rather than being able to use a io.Reader.

I don't know how valuable that command is for non-human operators but we could imagine apps restoring snapshots to easily put fixtures (e.g. tests) in place for example, or initialize data.

Would it be a task worth the effort for you guys? Thanks in advance.

cc @xiang90

[xiang90]

Snapshotting is server <-> client communication process, so it is in client library. Restore is a local logic, thus it is inside etcdctl not client library.

Exposing the restore logic is a nice thing to have. I assume some users (probably this is also your motivation?) prefer directly invoking funcs from Go instead of depending on the etcdctl binary.

I do not this is urgent to put into the most recent 3.3 release. But we probably can do this in 3.4 or later.

/cc @jpbetz

[luxas]

+1, I would really like to have this as an etcd Go client consumer in a Kubernetes project

[shubheksha]

Hi, I'd like to work on this issue if it's not taken already. This would be my first contribution so I'd appreciate some help regarding where to get started. 😄

[xiang90]

Sure! Glad you want to help.

On Dec 6, 2017 9:58 AM, "Shubheksha Jalan" [e-mail] wrote:

Hi, I'd like to work on this issue if it's not taken already. This would
be my first contribution so I'd appreciate some help regarding where to get
started. 😄

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/coreos/etcd/issues/8632#issuecomment-349683464, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AERby5NO7iDpYIv3MYZWfVmKeDINt0DQks5s9rmxgaJpZM4Pp0Or
.

[Quentin-M]

Here is another good reason to implement this while my code uses etcd 3.2.13, both CoreOS Stable 1576 and Alpine Linux ships with etcd binary 3.2.9/3.2.8. Because I must use the binary to restore a snapshot, here is what happens when I try:

# ETCDCTL_API=3 etcdctl snapshot restore [...]

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt.Open(0xc4201dc3e0, 0x1b, 0xc400000180, 0xc4201af718, 0xc4201dc420, 0x20, 0x0)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt/db.go:237 +0x497
github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/mvcc/backend.newBackend(0xc4201dc3e0, 0x1b, 0x5f5e100, 0x2710, 0x280000000, 0x493929)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/mvcc/backend/backend.go:129 +0xb4
github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/mvcc/backend.NewDefaultBackend(0xc4201dc3e0, 0x1b, 0x0, 0xc4201dc480)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/mvcc/backend/backend.go:119 +0x15e
github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/etcdctl/ctlv3/command.makeDB(0xc4201dc200, 0x18, 0x7ffd779f5775, 0xb, 0x1)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/etcdctl/ctlv3/command/snapshot_command.go:375 +0x38a
github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/etcdctl/ctlv3/command.snapshotRestoreCommandFunc(0xc4201d1800, 0xc420157e90, 0x1, 0x1)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/etcdctl/ctlv3/command/snapshot_command.go:196 +0x34e
github.com/coreos/etcd/cmd/vendor/github.com/spf13/cobra.(*Command).execute(0xc4201d1800, 0xc420157e50, 0x1, 0x1, 0xc4201d1800, 0xc420157e50)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/spf13/cobra/command.go:572 +0x203
github.com/coreos/etcd/cmd/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x11ff5e0, 0xd7cdbd, 0xb, 0xc4201afed8)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/spf13/cobra/command.go:662 +0x394
github.com/coreos/etcd/cmd/vendor/github.com/spf13/cobra.(*Command).Execute(0x11ff5e0, 0x0, 0x0)
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/spf13/cobra/command.go:618 +0x2b
github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/etcdctl/ctlv3.Start()
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/vendor/github.com/coreos/etcd/etcdctl/ctlv3/ctl_nocov.go:25 +0x8e
main.main()
    /build/amd64-usr/var/tmp/portage/dev-db/etcdctl-3.2.9/work/gopath/src/github.com/coreos/etcd/cmd/etcdctl/main.go:40 +0x1d0

However, no problem whatsoever with the 3.2.13 binary.

(of course, I can also install that version manually, to solve the issue - but it could still mess someone up)