Cluster-api-provider-azure: Add support for Ultra Disks for data disks

[CecileRobertMichon]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

This depends on #964

See also https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types and https://github.com/Azure/aks-engine/pull/3872.

Azure ultra disks deliver high throughput, high IOPS, and consistent low latency disk storage for Azure IaaS VMs. Validation needs to take all limitations into account https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types#ga-scope-and-limitations.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• cluster-api-provider-azure version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[pierluigilenoci]

/remove-lifecycle stale

[pierluigilenoci]

@CecileRobertMichon any update on this?

[Ankitasw]

/assign

[shysank]

@CecileRobertMichon @pierluigilenoci what's the expectation here? We already allow ultra ssd for data disks. Do we want to implement the ga limitations as part of azure machine validation?

[CecileRobertMichon]

@shysank in order to enable ultra ssd you also need VM properties to have AdditionalCapabilities.UltraSSDEnabled. I don't believe we're setting that now.

[shysank]

oh, Thank you! Never would have found that :) Just to confirm on the scope:

• Set AdditionalCapabilities.UltraSSDEnabled in virtualmachines.go => compute.VirtualMachineProperties
• Update Data disk validation to take all ga limitations into account

cc: @Ankitasw

[CecileRobertMichon]

Correct. I would also add:

Set AdditionalCapabilities.UltraSSDEnabled in virtualmachines.go => compute.VirtualMachineProperties

This we should only set when that capability is supported and make sure there are no downsides to enabling it without using it on VMs that do support it (tbh it's a bit of a weird API to have to enable the feature separately from where it's used)

[pierluigilenoci]

Thank you all! I can't wait to be able to use them in the clusters.

[CecileRobertMichon]

@pierluigilenoci btw I know you are/were an AKS Engine user, I'd appreciate any feedback you have on transitioning to CAPZ and/or anything that is blocking that transition. Feel free to jot down any thoughts you have in https://github.com/kubernetes-sigs/cluster-api-provider-azure/discussions/1079. Thanks!

[Ankitasw]

@CecileRobertMichon i have some questions regarding the issue
for machine type check in GA limitations, shall we add all the supported VM series in for eg enum kind and validate that if USSD is enabled, the vmseries should be in the one listed in the enum, and for vmseries, shall we add a new field in the DataDisk.
Can we make use of SKUs API here? Not quiet versed with the concepts.
and for redundancy check, we should check that if ussd is enabled , we should not use Availability set but we should use the zones, let me know if my understanding is correct

[CecileRobertMichon]

@Ankitasw you should be able to use the resource skus API for validation. Note that the validation cannot be done in the webhooks as it requires calls to Azure (and is subscription-specific). We already have a SKU cache in place that you should be able to reuse. See how it is used here as an example: https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/master/azure/services/virtualmachines/virtualmachines.go#L358

For UltraSSD, the locationInfo will contain info on whether a SKU supports ultra SSD in that location. You can test it out locally on the CLI.

For example, note how az vm list-skus -l eastus2 -s Standard_D8s_v3

"locationInfo": [
      {
        "location": "eastus2",
        "zoneDetails": [
          {
            "Name": [
              "1",
              "3",
              "2"
            ],
            "capabilities": [
              {
                "name": "UltraSSDAvailable",
                "value": "True"
              }
            ],
            "name": null
          }
        ],
        "zones": [
          "3",
          "1",
          "2"
        ]
      }
    ],

But az vm list-skus -l eastus2 -s Standard_D8_v3 (no S after the 8) returrns:

"locationInfo": [
      {
        "location": "eastus2",
        "zoneDetails": [],
        "zones": [
          "3",
          "1",
          "2"
        ]
      }
    ],

We should be able to use that info to validate the SKU / location supports UltraSSD in virtualmachines.go. You may need to add a new helper to sku.go for location capabilities since HasCapability only checks the top level SKU Capabilities currently, not locationInfo.zoneDetails.capabilities.

For redundancy options, the checks above should already take care of some of it since we don't put VMs in availability sets unless the location doesn't support zones, which means no availability sets will be used for the locations that support ultra SSD. It might get tricky to do deeper validation on disk type / redundancy since some of that is at the Cluster level. If you have any ideas let me know. In the meantime I would focus on validating the capability for the SKU as described above and linking to the GA limitations doc in the data disk documentation in the CAPZ book.

Hope this helps!