Azure-cli: `az aks create --enable-managed-identity --vnet-subnet-id` failing

[mathieu-benoit]

Describe the bug

When creating an AKS cluster with Managed Identity and by providing a Subnet Id, it will fail with the error message below.

Command Name
az aks create

Errors:

'NoneType' object has no attribute 'client_id'
Traceback (most recent call last):
python3.6/site-packages/knack/cli.py, ln 215, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 625, in execute
    raise ex
cli/core/commands/__init__.py, ln 689, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
cli/core/commands/__init__.py, ln 682, in _run_job
    six.reraise(*sys.exc_info())
...
cli/core/commands/__init__.py, ln 324, in __call__
    return self.handler(*args, **kwargs)
azure/cli/core/__init__.py, ln 531, in default_command_handler
    return op(**command_args)
cli/command_modules/acs/custom.py, ln 1750, in aks_create
    service_principal_profile.client_id, scope=scope):
AttributeError: 'NoneType' object has no attribute 'client_id'

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• _Put any pre-requisite steps here..._
• az aks create -l {} -n {} -g {} --no-ssh-key -k {} -s {} -c {} --enable-managed-identity --vnet-subnet-id {}

Expected Behavior

Environment Summary

Linux-4.19.84-microsoft-standard-x86_64-with-debian-buster-sid
Python 3.6.5
Installer: DEB

azure-cli 2.3.1

Extensions:
azure-firewall 0.3.0

Additional Context

FYI: without the --vnet-subnet-id parameter it's working fine.

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/aks-pm.

[yonzhan]

aks

[jluk]

cc @feiskyer / @andyzhangx

[andyzhangx]

cc @norshtein

[ams0]

Works for me on 2.3.1. The full command line is:

az aks create -k 1.17.3 --enable-managed-identity   --load-balancer-sku Standard --network-plugin kubenet -c 2 -s Standard_B4ms  --nodepool-name base  -g k8s  --network-policy calico -n playground2   --vnet-subnet-id /subscriptions/subID/resourceGroups/k8s/providers/Microsoft.Network/virtualNetworks/aksvnet/subnets/vnet111 --network-policy calico --aks-custom-headers CustomizedUbuntu=aks-ubuntu-1804 --enable-aad

[mathieu-benoit]

@ams0, do you have the aks-preview extension installed too?

[ams0]

yes,

$> az -v
azure-cli                          2.3.1

command-modules-nspkg              2.0.3
core                               2.3.1
nspkg                              3.0.4
telemetry                          1.0.4

Extensions:
aro                                0.2.0
show-deployment                  Unknown
aks-preview                       0.4.41

[mathieu-benoit]

Yep, in my case I don't have this aks-preview extension, so don't know if it's related to it by altering the behavior? I'm, by the way, able to reproduce the issue locally and on an Azure Pipelines agent too.

[palma21]

should be CLI-specific issue, also +@JunSun17 who took care of noneType error type before

[mathieu-benoit]

Any update on this? To summarize, it looks like we can't create an AKS cluster by bringing our own VNET/Subnet while leveraging --enable-managed-identity (and not having the aks-preview extension installed). FYI: same issue with the latest Azure CLI 2.4.0.

[jluk]

@norshtein could you please investigate this MI CLI issue?

[asubmani]

You would need to grant the "Managed Identity" of the cluster "Network contributor" rights.
CLIENT_ID=$(az aks show -g $RG -n $AKS --query identity.principalId -o tsv)

az role assignment create --assignee $CLIENT_ID --role "Network Contributor" --scope $AKSSUBNET_ID

The MI for the cluster gets created early, open a new terminal windows to retrieve the managed identity. Next grant "network contributor" access to the subnets resourceId

[asubmani]

A few notes to az cli team and AKS owners:

We had similar issue with Terraform that is now resolved.

Ever since Managed Identities went GA for AKS, seems like the API expects only "Managed Identities" in return.

az aks show -g $RG -n $AKS --query servicePrincipalProfile.clientId -o tsv

the new way to get the identity of AKS (that uses 'Managed Identity') is
az aks show -g $RG -n $AKS --query identity.principalId -o tsv

My guess is that every client would need to handle that they now get the Principalid of the "Managed Identity" if the cluster is using "Managed Identity". Most cli tools today assume that only SPN's are used.

[mathieu-benoit]

Hi @asubmani, in the case I'm reporting, I'm not able to create the cluster, so not able to run any az aks show command.

[asubmani]

Hi @asubmani, in the case I'm reporting, I'm not able to create the cluster, so not able to run any az aks show command.

In my case (I used terraform), I could run az aks show command within 30 seconds on initiating deployment. The "Managed Identity" gets created early on as it is this identity that creates stuff in the MC_* resource group.

BTW: I am still having issues with deployment, its just that I overcame the "identity" issue now :)

[mathieu-benoit]

Just to make sure here, the issue is when not using the aks-preview extension. Tested multiple times by multiple people.

The error message comes from this line https://github.com/Azure/azure-cli/blob/dev/src/azure-cli/azure/cli/command_modules/acs/custom.py#L1772 because few lines before there is this service_principal_profile = None which differs from what the code in the aks-preview extension has: https://github.com/norshtein/azure-cli-extensions/blob/master/src/aks-preview/azext_aks_preview/custom.py#L820.

I actually found out a workaround by using this parameter while running the AKS cluster creation command: --skip-subnet-role-assignment. And then assigned myself the associated/skipped role assignment to the AKS's Managed Identity (like described in this comment: https://github.com/Azure/azure-cli/issues/12864#issuecomment-618045651).

[palma21]

@norshtein ^

[jrudley]

az extension add --name aks-preview
az extension update --name aks-preview

az aks create --resource-group $aksRg
--name $aksRg --enable-vmss
--node-count 1 --vm-set-type VirtualMachineScaleSets
--generate-ssh-keys --kubernetes-version 1.16.7
--load-balancer-sku standard --network-plugin azure
--vnet-subnet-id $subnetId --docker-bridge-address 172.17.0.1/16
--dns-service-ip 10.2.0.10 --service-cidr 10.2.0.0/24
--enable-managed-identity

"azure-cli": "2.5.1",
"azure-cli-command-modules-nspkg": "2.0.3",
"azure-cli-core": "2.5.1",
"azure-cli-nspkg": "3.0.4",
"azure-cli-telemetry": "1.0.4",
"extensions": {
"aks-preview": "0.4.45"
}
}

No issues here

[mathieu-benoit]

Yep @jrudley, that's the point, Managed Identity with AKS is now GA, it should work even if the aks-preview extension is not installed ;)

[juanjojulian]

I actually found out a workaround by using this parameter while running the AKS cluster creation command: --skip-subnet-role-assignment. And then assigned myself the associated/skipped role assignment to the AKS's Managed Identity (like described in this comment: #12864 (comment)).

Thanks @mathieu-benoit , this workaround works perfect and saved my day

[norshtein]

This bug was fixed in https://github.com/Azure/azure-cli/pull/13543 and will be available in next Azure cli release.