Azure-sdk-for-python: Track 2 SDK fails to work with MSAL due to invalid `credential_scopes`

[jiasli]

Track 1 SDK - working

Previously in Track 1 SDK, the corresponding name of scopes is resource. The SDK itself doesn't maintain/know the resource. resource is provide by Azure CLI and used in token retrieval.

mgmt-plane - azure/cli/core/commands/client_factory.py:

resource = resource or cli_ctx.cloud.endpoints.active_directory_resource_id

data-plane - azure/cli/command_modules/role/_client_factory.py:

cred, _, tenant_id = profile.get_login_credentials(
    resource=cli_ctx.cloud.endpoints.active_directory_graph_resource_id)

Then the resource is used in get_login_credentials:

def get_login_credentials(self, resource=None, subscription_id=None, aux_subscriptions=None, aux_tenants=None):
    if aux_tenants and aux_subscriptions:

and saved in the closure for _retrieve_token.

Later on, when signed_session is called, it doesn't accept a resource and uses the one passed to get_login_credentials instead.

def signed_session(self, session=None):

Track 2 SDK - not working

In Track 2 SDK, the scopes is managed by the SDK itself:

https://github.com/Azure/azure-sdk-for-python/blob/64e01262ef5ab79132ce3d286d77e2335ffa810f/sdk/storage/azure-mgmt-storage/azure/mgmt/storage/_configuration.py#L46-L47

Then scopes is passed to get_token

https://github.com/Azure/azure-sdk-for-python/blob/5fdeae5e24e45d0ce36ae3d8942ef5f225d42940/sdk/core/azure-core/azure/core/pipeline/policies/_authentication.py#L93

If used in a sovereign cloud (like China cloud), the client needs to be created as

StorageManagementClient(credential_scopes=['https://management.core.chinacloudapi.cn/.default'])

Thus, credential_scopes becomes

['https://management.azure.com/.default', 'https://management.core.chinacloudapi.cn/.default']

which is not valid according to OAuth 2.0 authorization code flow, because it contains more than one resource:

| Parameter | Type | Description |
|---------------|----------------|--------------------|
| scope | optional | A space-separated list of scopes. The scopes must all be from a single resource, along with OIDC scopes (profile, openid, email).

This can also be verified with

az account get-access-token --scopes https://management.core.chinacloudapi.cn/.default https://management.azure.com/.default --debug

azure.core.pipeline.policies.http_logging_policy : Request URL: 'https://login.microsoftonline.com/54826b22-38d6-4fb2-bad9-b7b93a3e9c5a/oauth2/v2.0/token'
azure.core.pipeline.policies.http_logging_policy : Request method: 'POST'

azure.core.pipeline.policies.http_logging_policy : Response status: 400

msal.application : Refresh failed. invalid_scope: AADSTS70011: The provided request must include a 'scope' input parameter. The provided value for the input parameter 'scope' is not valid. The scope https://management.azure.com/.default https://management.core.chinacloudapi.cn/.default offline_access openid profile is not valid. .default scope can't be combined with resource-specific scopes.

Why this issue doesn't happen now (Track 2 SDK + ADAL)

This is because when get_token from Azure CLI is called, scopes is totally unused/discarded, thus forcing the Track 2 SDK client to use the resource provided by Azure CLI in get_login_credentials.

In the MSAL integration process (https://github.com/Azure/azure-cli/pull/14690), I try to honor the scopes provided by SDK and hit this issue.

Solution

The SDK needs to override the default credential_scopes if credential_scopes is provided in kwargs, so that the default value 'https://management.azure.com/.default' doesn't appear in self.credential_scopes.

A fix can be like this:

self.credential_scopes = kwargs.pop('credential_scopes', ['https://management.azure.com/.default'])

whose value is

['https://management.core.chinacloudapi.cn/.default']

[lmazuel]

Thanks @jiasli , as always very precise and accurate description of the issue. We work with @iscai-msft today on that. We didn't think through enough of the big picture when we used an "extend" here.

[iscai-msft]

@jiasli we've released a new autorest with a fix for this, use version 5.1.0-preview.7 in your command line (i.e. --use=@autorest/[email protected]. Let us know how it works for you and thanks for raising this issue!

[tibortakacs]

@lmazuel @iscai-msft @jiasli This change will also improve the Azure Sphere CLI experience, thank you very much for finding and resolving it so fast.

[jiasli]

Hi @lmazuel, @iscai-msft, thank you so much for the _rapid_ response. I am not currently blocked by this issue and I don't use the generated SDK in any way.

It will be nice if we can update the problematic SDKs (which are already released) soon so that the real users of those SDKs won't run into an error. 😉

[lmazuel]

Fixed in recent version of autorest (after 5.3.0)