Aks-engine: centraluseuap only support 1 fault domain for availabilityset

[yixwang]

Describe the bug
Getting the following error when deploying to centraluseuap

"error": {
2019-11-07 23:46:39.577101: "code": "InvalidParameter",
2019-11-07 23:46:39.577109: "message": "AvailabilitySet 'master-availabilityset-32855380' with platformFaultDomainCount = 1 can only support platformUpdateDomainCount = 1.",
2019-11-07 23:46:39.577118: "target": "platformUpdateDomainCount"
2019-11-07 23:46:39.577125: }

in the azuredeploy.json, we found the following:
{
"apiVersion": "[variables('apiVersionCompute')]",
"location": "[variables('location')]",
"name": "[variables('masterAvailabilitySet')]",
"properties": {
"platformFaultDomainCount": "[if(contains(split('canadacentral,centralus,eastus,eastus2,northcentralus,northeurope,southcentralus,westeurope,westus',','),variables('location')),3,if(equals('centraluseuap',variables('location')),1,2))]",
"platformUpdateDomainCount": 3
},
"sku": {
"name": "Aligned"
},
"tags": null,
"type": "Microsoft.Compute/availabilitySets"
},

It seems that centraluseuap only supports 1 and "platformUpdateDomainCount": 3 is causing the issue in this case

Steps To Reproduce

aks-engine generate and deploy with the ARM template.

Expected behavior
platformUpdateDomainCount should be 1 for centraluseuap

AKS Engine version
0.41.4

Kubernetes version
1.15.4

Additional context
We use AvailabilitySet for master agent pool

[mboersma]

I can reproduce this against centraluseuap.

Why doesn't this happen against regions where platformFaultDomainCount is 2? AKS Engine hard-codes the platformUpdateDomainCount to 3 regardless, so I don't understand why 1:3 in centraluseuap is an error but 2:3 in uksouth (for example) is not.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[shgoldbe]

I'm also facing this issue

[amankohli]

Do we have a wrokaround for the issue? We need to deploy kubernetes on the region and are blocked due to this

[mboersma]

In your cluster template (apimodel), you can override this value:

    "masterProfile": {
      "count": 1,
      "dnsPrefix": "",
      "vmSize": "Standard_D2_v3",
      "platformUpdateDomainCount": 1
    },

I tried this just now in centraluseuap and it did get past the platformFaultDomainCount = 1 can only support platformUpdateDomainCount = 1 error, but then I hit a CSE error during provisioning. I'll try to look into that next.

[amankohli]

Hi ,

Below is the solution we have implemented as a workaround:

i) Create a aks engine kubernetes cluster with single master.

ii) Edit the apimodel.json

iii) Scale the master nodes to 3

Do we a command to scale the master, I have used a command the aksegine scale command to scale the node but it doesn't work on the master?

[mboersma]

Create a aks engine kubernetes cluster with single master.

Did you accomplish this by adding "platformUpdateDomainCount": 1 in "masterProfile"? That should have worked for me but didn't, as I said above.

The bug described here prevents us from provisioning in centraluseuap at all, but it sounds like you're having an issue just with multiple masters. If that's the specific problem, could you open a new issue so we can keep them separate?

Do we a command to scale the master...?

No, the aks-engine scale command is only for node pools.

[amankohli]

Yes, we are able to create a k8s cluster on centraleuap by adding "platformUpdateDomainCount": 1 in "masterProfile" on the config file

[jackfrancis]

@amankohli Are you not able to create a cluster w/ > 1 nodes in the masterProfile?

[amankohli]

Yes we are unable to create a cluster with more than 1 node in the master profile, we always got an error saying the template is invalid.
@jackfrancis

[jackfrancis]

Hi @amankohli @yixwang could you please open up a new issue if there is still an issue? This thread has diverged into multiple issues.

Thanks!