velero restore drops certain fields of CustomResourceDefinition resources.

[subodh-dharma]

What steps did you take and what happened:

1. Perform backup of customresourcedefinition resources on a _source_ cluster using velero binary
2. Verified the backup contains the required resources with all fields as is 👍
3. Perform restore of the resources backed up in step 1 on a _target_ cluster
4. The restored CRD resources had some fields dropped.
5. The CRD resources were not processed/reconciled by any observable controllers

What did you expect to happen:
Velero restore using binary should restore CRD resources with all fields preserved.

Anything else you would like to add:

To verify if this is the case in general when trying to restore the CRD resource, I tried exporting the CRD resource manually to a YAML file from _source_ cluster, and perform kubectl create -f backup_cr.yaml on the _target_ cluster. The kubectl create command however preserves all the fields.

Environment:

• Velero version (use velero version): 1.4.0
• Velero features (use velero client config get features): backup, restore
• Kubernetes version (use kubectl version): 1.17
• Kubernetes installer & version: 1.17
• Cloud provider or hardware configuration: vSphere
• OS (e.g. from /etc/os-release): ubuntu

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• :+1: for "I would like to see this bug fixed as soon as possible"
• :-1: for "There are more important bugs to focus on right now"

[nrb]

Thanks for this report @subodh-dharma. v1.4.2 had some critical bug fixes relating to CRDs. Would you be able to retry this with v1.4.2 and see if the same behavior persists?

[subodh-dharma]

@nrb I had a discussion on slack channel #velero, it seems like velero restoration drops the status field. I was expecting it to be a part of the CR resource while performing a restore as it was part of it when performing a backup. 🤷‍♂️

I am not sure if it is a good idea to have a flag-switch in velero restoration to allow preserving the status field. At least in my case, there is some business logic that depends on the fields inside the status field.

To resolve this case, there can be two options that I can think of:

1. Create a custom plugin to allow preserving the status field.
2. Have a flag-switch in the velero itself to allow preserving any fields that are outside of the spec and metadata fields.

As of now, we cannot use velero _as-is_ since it doesn't restore the status field.

I would like to get your thoughts on this.

[ashish-amarnath]

@nrb We've seen these requests quite a bit.
WDYT of implementing a generic RestoreItemAction plugin that skips resetting the status fields if there is an annotation on the resource?

[ncdc]

I believe you'll need to create without status, then immediately follow-up with a 2nd call to update the status. The apiserver does not allow you to create with status (it forcibly clears it out). Because there is a gap and it's not an atomic operation, we originally decided not to try to restore status at all.

[nrb]

Closing as duplicate of #1687