Openebs: Cannot create raidz cStor pool

[NullOranje]

Description

Cannot create a cstore pool of type raidz in a new OpenEBS pool

Expected Behavior

Creation of a 3 disk raidz cStor storage pool

Current Behavior

Creation results in errors in the API server. Choosing a different poolType does not cause this behavior

Possible Solution

Steps to Reproduce

Install new OpenEBS cluster via helm: helm install --namespace openebs --name openebs stable/openebs

Create a new StoragePoolClaim via kubectl:

$ cat test2.yaml 
apiVersion: openebs.io/v1alpha1
kind: StoragePoolClaim
metadata:
  name: cstor-raid-pool
  annotations:
    cas.openebs.io/config: |
      - name: PoolResourceRequests
        value: |-
          memory: 2Gi
      - name: PoolResourceLimits
        value: |-
          memory: 4Gi
spec:
  name: cstor-raid-pool
  type: disk
  poolSpec:
    poolType: raidz
  blockDevices:
    blockDeviceList:
      - blockdevice-1f769d4195c6b8de31117ce4d8a4bbb3
      - blockdevice-37158ac3486598551a524ddf4140b5df
      - blockdevice-5f0ace467430b8085e8d7ce01f433f30
---

kubectl create -f test2.yaml

Confirm the creation of the SPC

$ kubectl get spc
NAME              AGE
cstor-raid-pool   3s

No CSP will be created

$ kubectl get csp
No resources found in default namespace.

Errors appear in the API log:

I1004 19:08:59.783777       6 storagepoolclaim.go:159] Finalizer storagepoolclaim.openebs.io/finalizer added on storagepoolclaim cstor-raid-pool
I1004 19:08:59.799678       6 handler.go:273] Provisioning pool 1/3 for storagepoolclaim cstor-raid-pool
E1004 19:08:59.805939       6 handler.go:276] Pool provisioning failed for 1/3 for storagepoolclaim cstor-raid-pool: failed to build cas pool for spc cstor-raid-pool: aborting storagepool create operation as no claimed block devices available: No valid block devices are available to claim
I1004 19:08:59.805964       6 handler.go:273] Provisioning pool 2/3 for storagepoolclaim cstor-raid-pool
E1004 19:08:59.812550       6 handler.go:276] Pool provisioning failed for 2/3 for storagepoolclaim cstor-raid-pool: failed to build cas pool for spc cstor-raid-pool: aborting storagepool create operation as no claimed block devices available: No valid block devices are available to claim
I1004 19:08:59.812575       6 handler.go:273] Provisioning pool 3/3 for storagepoolclaim cstor-raid-pool
E1004 19:08:59.820455       6 handler.go:276] Pool provisioning failed for 3/3 for storagepoolclaim cstor-raid-pool: failed to build cas pool for spc cstor-raid-pool: aborting storagepool create operation as no claimed block devices available: No valid block devices are available to claim
I1004 19:08:59.823486       6 spc_lease.go:124] Lease removed successfully on storagepoolclaim

Changing the poolType to something else (e.g., striped) will successfully create a pool

$ kubectl get csp
NAME                   ALLOCATED   FREE    CAPACITY   STATUS    TYPE      AGE
cstor-raid-pool-9qg7   168K        1016G   1016G      Healthy   striped   8m37s
cstor-raid-pool-d6wr   215K        1016G   1016G      Healthy   striped   8m37s
cstor-raid-pool-ie1x   208K        1016G   1016G      Healthy   striped   8m37s

Screenshots

Your Environment

• kubectl get nodes:
  NAME STATUS ROLES AGE VERSION
  csp-north-test-master1 Ready master 83d v1.14.3
  csp-north-test-server1 Ready 83d v1.14.3
  csp-north-test-server2 Ready 83d v1.14.3
  csp-north-test-server3 Ready 83d v1.14.3

• OS (from /etc/os-release): RHEL 7.7
• Kernel (from uname -a): Linux csp-north-test-server1 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 16 14:19:51 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
• Install tools: kubespray (v.1.14.3)
• Others:

[mittachaitu]

@NullOranje above SPC configuration indicates striped pool.

_NOTE: I have tried to explain each pool type as below_

striped

This configuration requires a minimum of one block device per node where the pool needs to be created. Suppose if 10GB disk attached to the node then the pool will be created on entire disk and pool size will be 10GB.

• SPC spec configuration will look like:

spec:
  name: cstor-striped-pool
  type: disk
  poolSpec:
    poolType: striped
  blockDevices:
    blockDeviceList:
      - blockdevice-1 ##<BlockDevice(bd1) From Node1>
      - blockdevice-2 ##<BlockDevice(bd2) From Node2>
      - blockdevice-3 ##<BlockDevice(bd3) From Node3>

mirrored

This configuration requires a minimum of _two blockdevices per node_ where the pool needs to be created. In this configuration if two 10GB disks are attached to the node then pool will be created on top of two disks and pool size will be 10GB. Data will be replicated/mirrored to another disk.

• SPC spec configuration will look like:

spec:
  name: cstor-mirrored-pool
  type: disk
  poolSpec:
    poolType: mirrored
  blockDevices:
    blockDeviceList:
      - blockdevice-1 ##<BlockDevice(bd1) From Node1>
      - blockdevice-2 ##<BlockDevice(bd2) From Node1>
      - blockdevice-3 ##<BlockDevice(bd3) From Node2>
      - blockdevice-4 ##<BlockDevice(bd4) From Node2>
      - blockdevice-5 ##<BlockDevice(bd5) From Node3>
      - blockdevice-6 ##<BlockDevice(bd6) From Node3>

raidz

This configuration requires a minimum of _three blockdevices per node_ _(i.e. 2+1 where 2 block devices are used to replica data (copy) and one block device is used for parity)_. In this configuration if _three_, 10GB disks attached to the node then pool will be created on top of these disks and pool size will be 10GB. Data will be replicated on two disks and one disk will be used for parity checking.

NOTE: Usually for raidz it should take 2^n + 1 blockdevices are required[where n > 0] but SPC has issue (or) limitation with it but CSPC(an updated version of SPC) doesn't have any such kind of limitations.
SPC spec configuration will look like:

spec:
  name: cstor-raidz-pool
  type: disk
  poolSpec:
    poolType: raidz
  blockDevices:
    blockDeviceList:
      - blockdevice-1 ##<BlockDevice(bd1) From Node1>
      - blockdevice-2 ##<BlockDevice(bd2) From Node1>
      - blockdevice-3 ##<BlockDevice(bd3) From Node1>
      - blockdevice-4 ##<BlockDevice(bd4) From Node2>
      - blockdevice-5 ##<BlockDevice(bd5) From Node2>
      - blockdevice-6 ##<BlockDevice(bd6) From Node2>
      - blockdevice-7 ##<BlockDevice(bd7) From Node3>
      - blockdevice-8 ##<BlockDevice(bd8) From Node3>
      - blockdevice-9 ##<BlockDevice(bd9) From Node3>

Similarly, _raidz2_ implies 4 devices for data and 2 devices for parity.

NOTE: Order of populating block device doesn't matter.

[mittachaitu]

@NullOranje try with the above configurations if it works we can close this issue.

[NullOranje]

Yes, it was me not understanding the configuration. Thanks.