Velero: Document resource inclusion/exclusion logic

[nrb]

Having a document that describes the logic for including/excluding resources with combination of backup specs would be a useful reference for users to understand what their options are, as well as a place to point people instead of re-explaining.

[carlisia]

Things to consider adding:

• a bit of set theory that we use
• the logic
• examples
• list of things/examples not possible to select

[prydonius]

This should also include the standard label used to exclude resources introduced in #1588

[nrb]

I would propose putting this in it's own document under Use, because it's a key part of using the backup and restore commands.

[skriss]

👍 agree - I'd like to expand the Use section to have more examples of ways you can use Velero, that cover all of these different flags/etc.

[jaxxstorm]

question @carlisia and @nrb - why was a label chosen to exclude resources instead of an annotation? The accepted convention is generally to use annotations for this kind of exclusion decision

[nrb]

@jaxxstorm Mostly for the query capabilities that labels provide. Annotations don't provide any way of querying via the client APIs.

Can you provide a link to the conventions you're describing?

[jaxxstorm]

I am yet to find another kubernetes based project or operator that use labels in this way. Some examples:

https://github.com/hjacobs/kube-downscaler - uses annotations for exclusion logic
https://github.com/kubernetes-incubator/external-dns - uses annotations for operational logic

I guess if you want to make this queryable the using labels seems fine, but I haven't found any other project that follows this pattern

[prydonius]

@jaxxstorm that's a good point. In fact with #1843 we no longer query using exclusion label, so we could consider changing it to an annotation. Of course, that would be a breaking change for anyone using the label right now, maybe we could support both? WDYT @skriss @nrb @carlisia?

[jaxxstorm]

i think supporting both seems reasonable

[nrb]

Yeah, I'd be +1 on doing both and dropping the label in v2.0. I consider the well-known labels/annotations in a project to be part of the public API, even if it makes the process of changing a little onerous.

For further context, the use of the label came as an extension of an example in our docs before it was actually codified.

[carlisia]

More logic for backup/restore:

IncludeClusterResources is a nullable bool, so has 3 possible modes:

• true: all cluster-scoped resources that are selected by the resource/label selectors are restored
• false: no cluster-scoped resources are restored
• nil ("auto"): only cluster-scoped resources that are dependencies of other namespaced resources being restored, are restored (e.g. we're restoring a pod that uses a PVC/PV, so we'd restore the PV as well). the way this is actually implemented in the code is by allowing "additional items" to be restored, even if they're cluster-scoped, when IncludeClusterResources=nil, since "additional items" are how we capture relationships between resources.

Source: @skriss https://github.com/vmware-tanzu/velero/pull/2118#discussion_r357326180

[mtritabaugh]

I'd like to assign this to myself.