Velero: Some of the volumes are not getting backed up although annotated

[anupash147]

What steps did you take and what happened:
In order to take a velero backup with volumes we need to annotate the pods, i practically tried it on a lame namespace and it worked. When i tried the same on a namespace which had 4+ volumes i see only 2 have been successfully backed up by restic and rest of the 4 shows below error

time="2020-05-14T22:09:24Z" level=warning msg="No volume named mysql-pv-claim found in pod <namespace>/mysql-6bcbfbb6b8-tvgt4, skipping" backup=velero/mybackup-bk-v2 group=v1 logSource="pkg/restic/backupper.go:135" name=mysql-6bcbfbb6b8-tvgt4 namespace=<namespace> resource=pods

I verified that the pvc name is correct and is properly annotated in the pod
see below

Name:           mysql-6bcbfbb6b8-tvgt4
Namespace:      <namespace>
.
.
.
Annotations:    backup.velero.io/backup-volumes: mysql-pv-claim

What did you expect to happen:
I was expect all of the volumes in the particular namespace be backed up so that i can restore and make a restore in another cluster.

The output of the following commands will help us better understand what's going on:
Here is the gist of the logs https://gist.github.com/anupash147/9b2cd95eac9cec9890c2a32ec3ce805f

Environment:

• Velero version (use velero version): v1.2.0
• Velero features (use velero client config get features):
• Kubernetes version (use kubectl version): 1.15
• Kubernetes installer & version: 1.15
• Cloud provider or hardware configuration: aws
• OS (e.g. from /etc/os-release): amazonlinux

[ashish-amarnath]

@anupash147 Can you please share your pod spec?
The error is because the pod, that you are wanting to backup, does not have any volume named mysql-pv-claim in its spec.Volumes

[anupash147]

@ashish-amarnath : I have added a gist, what more info is needed

[ashish-amarnath]

@anupash147
What I was asking was the spec of the pod kubeflow/mysql-6bcbfbb6b8-tvgt4
The warning message is indicating that this pod, pod kubeflow/mysql-6bcbfbb6b8-tvgt4 has an annotation

backup.velero.io/backup-volumes=mysql-pv-claim

But mysql-pv-claim is not a volume that is mounted by kubeflow/mysql-6bcbfbb6b8-tvgt4.

[skriss]

Note that the value of the annotation needs to be the pod volume name, not the PVC name.

[anupash147]

@ashish-amarnath i had checked it here is the image confirmation

[ashish-amarnath]

@anupash147
Looks like you are using the PVC name* as the value for the annotation.
As Steve mentioned, the value of the annotation should be the **pod volume name, in this case it should be my-sql-persistent-storage

[anupash147]

i see.. thanks @ashish-amarnath & @skriss for point it out..

[ashish-amarnath]

No problem! Closing this issue as the question has been answered.