Aks-engine: Temporary storage only shows 30GB even though underlying VM has 200GB.

[ericsuhong]

I provisioned a cluster using aks-engine with Standard_D8_v3 VMs, which have 200GB temporary storage (https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-general)

However, when I get nodes statistics, it shows that only 30GB is available:

kind: Node
  metadata:
    annotations:
      node.alpha.kubernetes.io/ttl: "0"
      volumes.kubernetes.io/controller-managed-attach-detach: "true"
    labels:
      agentpool: agentpool
      beta.kubernetes.io/arch: amd64
      **beta.kubernetes.io/instance-type: Standard_D8_v3**
      beta.kubernetes.io/os: linux
      failure-domain.beta.kubernetes.io/region: eastus
      failure-domain.beta.kubernetes.io/zone: eastus-1
      kubernetes.azure.com/cluster: IAMKubernetesPRODEUS-a
      kubernetes.azure.com/role: agent
      kubernetes.io/arch: amd64
      kubernetes.io/hostname: k8s-agentpool-12915089-vmss000000
      kubernetes.io/os: linux
      kubernetes.io/role: agent
      node-role.kubernetes.io/agent: ""
      storageprofile: managed
      storagetier: Standard_LRS
    name: k8s-agentpool-12915089-vmss000000
  status:
    allocatable:
      attachable-volumes-azure-disk: "16"
      cpu: "8"
      **ephemeral-storage: "28043041951"**
      hugepages-1Gi: "0"
      hugepages-2Mi: "0"
      memory: 32150428Ki
      pods: "250"
    capacity:
      attachable-volumes-azure-disk: "16"
      cpu: "8"
      **ephemeral-storage: 30428648Ki**
      hugepages-1Gi: "0"
      hugepages-2Mi: "0"
      memory: 32918428Ki
      pods: "250"

Describe the bug
Standard_D8_v3 VM has 200 GB Temp Storage, but we only see 30GB.

Steps To Reproduce
Create a cluster using aks-engine using any VM with > 30GB temporary storage. You will only see 30GB in Kubernetes.

Expected behavior
We must be able to fully utilize the full capacity of underlying VM's Temporary storage.

AKS Engine version
0.39.2

Kubernetes version
1.15.3

Additional context

[devigned]

Have you specified "storageProfile" : "Ephemeral", in your agentPoolProfiles

Check out our ephemeral-disks example.

[ericsuhong]

Turns out that I was misunderstanding the concept.

Also, thanks for letting me know about OSDiskSizeGB and ephemeral storage settings.

[mohamedmansour]

@ericsuhong @devigned I can't seem to make this work on my end. I have a Standard_F32s_v2 which contains 256GB of temporary storage. When I created the cluster:

az aks create -g test -n test --enable-addons monitoring --generate-ssh-keys \
   --node-vm-size  Standard_F32s_v2  --node-count 1 --attach-acr test

It only shows 100GB:

    "agentPoolProfiles": [
      {
        "count": 1,
        "maxPods": 110,
        "name": "nodepool1",
        "orchestratorVersion": "1.15.10",
        "osDiskSizeGb": 100,
        "osType": "Linux",
        "provisioningState": "Succeeded",
        "type": "VirtualMachineScaleSets",
        "vmSize": "Standard_F32s_v2"
      }

How do I update this clusters agentPoolProfiles to set storageProfile = Ephemeral? Are there instructions how I can do this?