Azure-cli: az role assignment create is not idempotent

[jurjenoskam]

Creating the same role assignment twice results in success the first time, but the second time results in an error: The role assignment already exists.

$ az role assignment create --role Owner --assignee "[email protected]" --scope /subscriptions/<redacted>/resourceGroups/<redacted>/providers/Microsoft.Storage/storageAccounts/<redacted>
The role assignment already exists.
$ echo $?
1

According to https://github.com/Azure/azure-cli/blob/dev/doc/command_guidelines.md#standard-command-types this operation should be idempotent, i.e.: creating existing role assignments should not result in an error.

This happens with azure-cli 2.0.58 (role 2.4.0) on Linux but also with azure-cli 2.0.51 (storage 2.2.5) on Windows.

[tjprescott]

Thanks for the report @jurjenoskam.

[jurjenoskam]

Woops, I accidentally included the storage component version for Windows. The correct Azure CLI version for my Windows attempts is azure-cli 2.0.51 (role 2.1.10).

[tjprescott]

Thanks, I was wondering about that, but figured it was a typo. 😁

[marstr]

Digging into the REST API Spec for the relevant operation here. I think the trouble is that while this operation is declared as a PUT, only an HTTP 201 (created) is listed as an acceptable response. If the role assignment already exists, the service is likely sending back a 200 (OK) and getting raised as an exception here in the azure-sdk-for-python. While that likely means that we're actively catching and scrubbing this exception with an error that may not be applicable, I believe the correct fix is to update the swagger to declare the other successful response codes. For that reason, I'd like to transfer this issue to the Azure/azure-rest-api-specs repository.

[yugangw-msft]

@marstr, the error code is 409 as revealed through --debug. I agree this is service bug that we should tag as Service Attention.
At the same time, considering idempotent is a basic principle we like to maintain across CLI so that scripts can confidently rely on. I suggest we catch it, and ignore if the error code is RoleAssignmentExists. Letting service team fix it would likely take years based on the experience we have accumulated on other service issues.

Technically this is a breaking change, but I am fine we go ahead to swallow it.

[marstr]

Alright, I've spent a little time digging into this. I think that I can work around the service not exposing this as an idempotent operation themselves by recreating it in our module. To do so we'll need to modify this method to catch the 409:

https://github.com/Azure/azure-cli/blob/fe0f256ee877c6f88ff432213115e9a0bf82e97b/src/command_modules/azure-cli-role/azure/cli/command_modules/role/_multi_api_adaptor.py#L46-L52

However, in-order to hide the fact that the operation didn't actually go through, we'll need to hit the wire again to fetch the role-assignment that would have been returned by the service had it been newly created. Knowing that we'll have to create a secondary request, do you still want me to proceed, @yugangw-msft? (I'm okay with either way on this.)

[yugangw-msft]

I would suggest we proceed by fetching the assignment by leveraging the existing list_role_assignments.
Idempotent is the main command authoring guideline we would like to persist which community has appreciated very much.

[marstr]

I ended up flipping around the strategy, and asking if the role assignment exists first instead of reacting to a 409. Figured it was a much cleaner solution, and it matches how Terraform works around this problem.

[yugangw-msft]

Fixed by #9108

[alanold]

@marstr @yugangw I'm having an issue with this same issue. Running this script from CLI or devops

!/bin/bash

Obtain role defination IDs

roledef=$(az role definition list --query "[?roleName=='xxx'].id" --output tsv)
echo $fwdef

Obtain service principle app IDs

spappID=$(az ad app list --all --query "[?contains(displayName, 'sp-1111-2222')].appId" --output tsv)

Obtain subscription IDs

subprefix="/subscriptions/"
subID=($subprefix$(az account list --query "[?contains(name, 'subscription1')].id" --output tsv))

Apply role definations

az role assignment create --assignee $spappID --role $roledef --scope $subID

Initial run is successful. Subsequent runs get this error

2019-07-05T15:20:02.8680610Z ERROR: The command failed with an unexpected error. Here is the traceback:
2019-07-05T15:20:02.8681837Z
2019-07-05T15:20:02.8698790Z ERROR: list index out of range
2019-07-05T15:20:02.8699199Z Traceback (most recent call last):
2019-07-05T15:20:02.8700333Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/custom.py", line 138, in create_role_assignment
2019-07-05T15:20:02.8700719Z assignee_principal_type=assignee_principal_type)
2019-07-05T15:20:02.8701403Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/custom.py", line 157, in _create_role_assignment
2019-07-05T15:20:02.8701753Z assignee_principal_type)
2019-07-05T15:20:02.8702395Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/_multi_api_adaptor.py", line 54, in create_role_assignment
2019-07-05T15:20:02.8702775Z return client.create(scope, assignment_name, parameters)
2019-07-05T15:20:02.8703391Z File "/opt/az/lib/python3.6/site-packages/azure/mgmt/authorization/v2018_09_01_preview/operations/role_assignments_operations.py", line 327, in create
2019-07-05T15:20:02.8703800Z raise exp
2019-07-05T15:20:02.8704062Z msrestazure.azure_exceptions.CloudError: Azure Error: RoleAssignmentExists
2019-07-05T15:20:02.8704630Z Message: The role assignment already exists.
2019-07-05T15:20:02.8704864Z
2019-07-05T15:20:02.8705102Z During handling of the above exception, another exception occurred:
2019-07-05T15:20:02.8705372Z
2019-07-05T15:20:02.8705597Z Traceback (most recent call last):
2019-07-05T15:20:02.8706191Z File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 206, in invoke
2019-07-05T15:20:02.8706519Z cmd_result = self.invocation.execute(args)
2019-07-05T15:20:02.8707112Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 575, in execute
2019-07-05T15:20:02.8707416Z raise ex
2019-07-05T15:20:02.8708012Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 633, in _run_jobs_serially
2019-07-05T15:20:02.8708344Z results.append(self._run_job(expanded_arg, cmd_copy))
2019-07-05T15:20:02.8709201Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 626, in _run_job
2019-07-05T15:20:02.8709525Z six.reraise(sys.exc_info())
2019-07-05T15:20:02.8710086Z File "/opt/az/lib/python3.6/site-packages/six.py", line 693, in reraise
2019-07-05T15:20:02.8710383Z raise value
2019-07-05T15:20:02.8710971Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 603, in _run_job
2019-07-05T15:20:02.8711275Z result = cmd_copy(params)
2019-07-05T15:20:02.8711870Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 305, in __call__
2019-07-05T15:20:02.8712179Z return self.handler(args, kwargs)
2019-07-05T15:20:02.8712776Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 470, in default_command_handler
2019-07-05T15:20:02.8713083Z return op(command_args)
2019-07-05T15:20:02.8713921Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/custom.py", line 141, in create_role_assignment
2019-07-05T15:20:02.8714478Z return list_role_assignments(cmd, assignee, role, resource_group_name, scope)[0]
2019-07-05T15:20:02.8714804Z IndexError: list index out of range
2019-07-05T15:20:02.8715088Z WARNING:
2019-07-05T15:20:02.8715567Z To open an issue, please run: 'az feedback'

[alanold]

Additional information. This is running against Azure China

[etaham]

This isn't fixed. Please reopen.