Cluster-api-provider-azure: Configurable OS disk caching

[CecileRobertMichon]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

Currently we don't set the disk caching type for OS disks (defaults to None).

The azure-sdk-for-go defines three caching types:

const (
// CachingTypesNone ...
CachingTypesNone CachingTypes = "None"
// CachingTypesReadOnly ...
CachingTypesReadOnly CachingTypes = "ReadOnly"
// CachingTypesReadWrite ...
CachingTypesReadWrite CachingTypes = "ReadWrite"
)
Let's enable user-configurable caching types by adding a new field in AzureMachineSpec.OSDisk and leveraging validation and defaulting webhooks. We should also define a sane default for control plane and worker OS disks.

Reference PR in AKS Engine: https://github.com/Azure/aks-engine/pull/2863

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• cluster-api-provider-azure version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[CecileRobertMichon]

/help

[k8s-ci-robot]

@CecileRobertMichon:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[muang0]

Hi @CecileRobertMichon ! Cool if I work on this one? :smile:

[muang0]

Would you mind clarifying about the defaults for worker and control plane disks? :smile: I think I'm a bit confused on that one

We should also define a sane default for control plane and worker OS disks

Thanks for the help!

[CecileRobertMichon]

@jroden based on https://github.com/Azure/aks-engine/pull/2863/files#diff-fd43f080184ed15731e5d7e0d917d6e1R697 I'd say we want ReadWrite as the default but maybe @jackfrancis can help confirm here.

[muang0]

@CecileRobertMichon thanks for the help!

So for example, if the osDisk is to be attached to a worker or control plane instance we would default to ReadWrite, but in all other cases we would default to None?

[alexeldeib]

Let's do either None or ReadOnly for the OSDisk, ReadWrite or ReadOnly for data disks (in order of preference). I don't really think the write cache is our friend under load...