Velero: Option to choose another GVK besides the preferred-version

[gildub]

Describe the problem/challenge you have
A use case is when exporting a resource from a version of K8s being older than the version of the cluster restoring that resource into.

Let's say the backup cluster has api-server supporting:

• "some-api-group/v1" <- preferred-version
• "some-api-group/v2beta1"

And the restore side has:

• "some-api-group/v2"
• "some-api-group/v2beta1"

At the moment this is not possible because the preferred version would export the resource represented in v1 and the api-server on the restore side doesn't support it.

Describe the solution you'd like
Instead of having the version automatically selected from the preferred-version,
being able to choose another API-Group/version (of course only if that GVK is supported by the api-server), would allow to export the involved resource using "some-api-group/v2beta1" and make the import possible into the destination.

Anything else you would like to add:
Overall the idea is to help creating GKVs export/import paths when 2 clusters are apart version-wise.
And the more 2 clusters are away from each others the higher the GVK gap, and this is where such option could really help. And even if a destination doesn't support the exported version, being able to bring it closer to a supported version would help too.

Environment:

• Velero version (use velero version): 1.2.0+
• Kubernetes version (use kubectl version): 1.7+
• Kubernetes installer & version: any
• Cloud provider or hardware configuration: any
• OS (e.g. from /etc/os-release): any

[skriss]

@gildub what if Velero backed up all versions of a resource, and then intelligently restored the appropriate version based on which versions it found in the target cluster? Would that meet your need without requiring you to provide the mapping?

[gildub]

@skriss, that would be ideal!
Now I'm not sure how you're going to do that magic unless using a similar approach as oc convert.

[skriss]

Now I'm not sure how you're going to do that magic unless using a similar approach as oc convert.

Well, I think for starters we wouldn't do anything too magical. In the example you gave, we'd back up both v1 and v2beta1 representations of the resources. Then, on restore, we'd see that the target cluster supported v2beta1 and v2. Since the only overlap is v2beta1, we'd restore that version of the resource. So, no conversion required by Velero.

The next step would be to support conversion when there is no overlapping version between the backup and the target cluster. That would be pretty magical..

[skriss]

xref #2310

[brito-rafa]

Hi @gildub, if I code a prototype of this request, would you be able to help me with the testing... like running a backup on your environment?

[gildub]

@brito-rafa, I could definitely help you test.

[brito-rafa]

Thanks! I give you the testing image once I have the code running with 100% backward compatible with the standard format. I am particularly interested to see APIGroups that are not native to K8s.

[brito-rafa]

Hi @gildub

So I have a prototype of the velero backup server with the option to backup all versions. To test, please follow these steps:

1. Install Velero as usual on your cluster.

2. Create a file "velero-migration-patch.yaml" with the following content:

spec:
  template:
    spec:
      containers:
      - args:
        - server
        - --log-level
        - debug
        - --features=Migration
        name: velero
        image: quay.io/brito_rafa/velero:dev-2251-0326-D

1. Run the kubectl patch command:

kubectl patch deployment velero --patch "$(cat velero-migration-patch.yaml)" -n velero

1. Take a normal backup and verify the content of it. You will see additional directories for each version of the resource lists (along with the default preferred group version).

At this point, this version only takes the backup of all API group versions.
The Restore would be coded on the next phase, probably after we confirm this version is fully backward compatible, and the logic makes sense.

Here what I would like to confirm:

1. It is indeed 100% backward compatible (my canned tests say yes). I would benefit you to run some restores from the "Migration" backup.
2. What the expected outcome of your CRDs with this prototyped version? Is it getting all API group versions?

Any help is appreciated!

Cheers,
Rafa

[brito-rafa]

Just an update, please see script how to test the latest release.

[gildub]

@brito-rafa, that's really great, thank you!

[djwhatle]

Hey @brito-rafa, I'm taking over for @gildub on testing this out. Will try this out today and report back.

[djwhatle]

@brito-rafa I used your test script to check out the changes you've made. Looks like it works as described 👍

• [x] Existing "legacy" directory structure has been preserved (top level namespaces dir)
• [x] Conversions of each resource to all apiVersions known to apiserver are included in archive
• [x] Export of resource using preferredVersion is marked accordingly

_New Format of Backup Content Archive_ (includes legacy dir, plus labeled version dirs)

[eleanor-millman]

Looks like it was finished!