Velero: [Epic] add support for restore hooks

[marccampbell]

Describe the problem/challenge you have
When backing up some volumes, we have a pre and post backup hook implemented. This is useful when we have a process that has a lock on a file and we need to copy data between volumes to ensure that the data Velero is backing up will be uncorrupted and usable. But on a restore, there is no hook to execute a pre or post restore script. This makes automating a restore a little difficult because we need to rely on a separate set of tools when restoring vs when backing up.

Describe the solution you'd like
One possible solution to this is to add restore hooks, so they can be specified similar to how backup hooks are specified. If we are restoring data that was collected using a pre-hook, then we could write the reverse post-restore hook in the configuration to put the data back into the format or restart a process.

[konvergence]

For a postgres dump done with a pre-hook backup : pg_dumpall --clean > ${DATA_FOLDER}/dumpall.out
it will usefull to run a restore post hook to run a psql < ${DATA_FOLDER}/dumpall.out

[konvergence]

If there was post hook for restore I could use it to restore a dump previously done with pre hook backup.

Example if we have this annotation for pre hook backup

kubectl -n bitamani-postgres-11-hook  annotate pod/postgres-hook-postgresql-0   backup.velero.io/backup-volumes=data --overwrite
kubectl -n bitamani-postgres-11-hook  annotate pod/postgres-hook-postgresql-0 pre.hook.backup.velero.io/timeout=3600s --overwrite
kubectl -n bitamani-postgres-11-hook  annotate pod/postgres-hook-postgresql-0   pre.hook.backup.velero.io/command='["bash", "-c", "mkdir -p /bitnami/postgresql/dumps; eval export PGPASSWORD=${POSTGRES_PASSWORD}; pg_dumpall --clean --username=postgres | gzip -4 > /bitnami/postgresql/dumps/dumpall.out.gz" ]' --overwrite

the following post hook restore could be used to restore the dump

kubectl -n bitamani-postgres-11-hook  annotate pod/postgres-hook-postgresql-0 post.hook.restore.velero.io/timeout=3600s --overwrite
kubectl -n bitamani-postgres-11-hook  annotate pod/postgres-hook-postgresql-0   post.hook.restore.velero.io/command='["bash", "-c", ""bash", "-c", "eval export PGPASSWORD=${POSTGRES_PASSWORD}; cat /bitnami/postgresql/dumps/dumpall.out.gz | gunzip | psql --username=postgres" ]' --overwrite

[nrb]

@skriss @michmike I think this is a good candidate for v1.5; though we allow for backup quiescing, we don't really allow for unquiescing that data, which makes it less than useful in the case of databases, as the OP points out.

[aclevername]

This is would be really neat, we are also in the same spot as @konvergence where we are planning on using the hooks to trigger the database dump during backup, and would love to be able to automate restore as well

[skriss]

@marccampbell I think we'd definitely like to do this, but it's not currently prioritized. If you're interested in taking the lead on a design proposal, let us know and we can help provide input!

[marccampbell]

@skriss i'd love to take the lead on a design proposal for this.

[aclevername]

@skriss i'd love to take the lead on a design proposal for this.

We (@cloudfoundry-incubator/bosh-backup-and-restore-team) would love to help contribute on this too :)

[skriss]

@marccampbell @aclevername that'd be great!

we have a design docs directory here: https://github.com/vmware-tanzu/velero/tree/master/design

There's a template, as well as all of our existing docs.

I'd recommend first putting up a draft PR with an incomplete design doc that covers goals/non-goals and high-level design, so folks have a chance to comment on those high-level sections before you spend time going into more detail.

[stephbman]

@marccampbell we'll be moving the deep dive out by a few days to mid/late next week. I will send details your way on that.

[michael-niemand]

I'd appreciate a feature like this as well!

[nrb]

Closing issue with the release of v1.5.1