Openebs: Attach the disk used LocalPV Device from one Node to another.

[ajeshbaby]

OpenEBS Version - 1.10

Usecase/STR

1. Use a GPD disk on a Node in the GKE/physical node as OpenEBS Local PV Device. Populate data,
2. Scale down the node or Assume the physical node is having some infra issue and need to migrate to a new node.
3. Scale up the node. The new node will be having different hostname, Attach the disk to this node, and want to retrieve the data as the disk is intact.

The current issue, The Local PV has a mapping to Node name, But the hostname is different in scale-up case, Can't change the name in existing PV.

[ajeshbaby]

How to attach LocalPV Device, The steps I followed in GKE and info taken from the below link
https://github.com/ranjithwingrider/solution-app/blob/master/Prometheus/day2-ops-scale-down.md

The steps I followed in GKE and info taken from the above link

• Find the mapping PV, BD, Diskname, Hostname

You can get the device name from get bd output, Also the old hostname which it was attached, this would get changed as part attaching the disk to new node automatically.
PV would still have the old hostname as a selector.

kubectl get pv pvc-b20bd6ad-14e7-11ea-9ec3-42010a80007 -o yaml 
kubectl get  bd blockdevice-c6b6bbdf69d43f8c8482db796e2be1ae -n openebs -o yaml

• Check the application for any node selection and label the node

kubectl get sts od-elasticsearch-logging-local-pv -o yaml
kubectl label node gke-gecko-elastic-search-5ba106ba-e0m4 elasticsearch=localpv

• Take the backup of PVC

kubectl get pv pvc-b20bd6ad-14e7-11ea-9ec3-42010a80007 -o yaml  > elastic_pv.yaml
````

- Attach the disk to the node and verify the BD got updated with the same.

gcloud compute instances attach-disk gke-gecko-elastic-search-5ba106ba-e0m4 --disk elasticsearch-localpv-production --device-name elasticsearch-localpv-production --zone us-central1-a
kubectl get bd blockdevice-c6b6bbdf69d43f8c8482db796e2be1ae -n openebs

- Delete the PV (Ensure you have taken the backup of PV), It wont delete, you need to remove the finalizer 

kubectl delete pv pvc-b20bd6ad-14e7-11ea-9ec3-42010a80007

- Edit the PV yaml and update the hostname in this case gecko-elastic-search-5ba106ba-e0m4 

Apply the same

kubectl apply -f elastic_pv.yaml
```
Boom. the Pod will be running.

[github-actions[bot]]

Issues go stale after 90d of inactivity.

[kmova]

One of the ways we can solve this is using the following solution:

• Setup the Local Pv with a custom node affinity label - managed by NDM
• NDM can do the following:
  
  
  
  • NDM can add a label to the node whenever a BDC is bound to a BD ( BD=BDC )
  
  
  • NDM when moving a BD from one node to another node - should move the label ( BD=BDC) to new node (remove from old node and add to new node)
  
  
  • NDM can delete the label (BD=BDC) when the BD goes into unclaimed state.
  
  

The above will help to set the node affinity on PV to match node label ( BD=BDC). So the Local PVs will automatically be available on the new node - instead of the current workaround where we have to delete old pv - create new one with new node affinity settings.

Thanks @bobek for the above idea!

[github-actions[bot]]

Issues go stale after 90d of inactivity.