Azure-sdk-for-net: SharedTokenCacheCredential fails in WSL when logged into az CLI

[heaths]

Seems I can't use the SharedTokenCacheCredential on WSL when logged into the az CLI or PowerShell Core.

Environment

• Ubuntu 18.04 (latest updates installed) in WSL2
• .NET core 3.1.301
• Windows 10 2004
• az version:

{
  "azure-cli": "2.7.0",
  "azure-cli-command-modules-nspkg": "2.0.3",
  "azure-cli-core": "2.7.0",
  "azure-cli-nspkg": "3.0.4",
  "azure-cli-telemetry": "1.0.4",
  "extensions": {}
}

Stack trace:

Unhandled exception. Azure.Identity.AuthenticationFailedException: DefaultAzureCredential authentication failed.
 ---> Azure.Identity.AuthenticationFailedException: SharedTokenCacheCredential authentication failed.
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'attributeValue1')
   at Microsoft.Identity.Client.Extensions.Msal.CacheAccessorLinux..ctor(String cacheFilePath, String keyringCollection, String keyringSchemaName, String keyringSecretLabel, String attributeKey1, String attributeValue1, String attributeKey2, String attributeValue2, TraceSourceLogger logger)
   at Microsoft.Identity.Client.Extensions.Msal.MsalCacheStorage.Create(StorageCreationProperties creationProperties, TraceSource logger)
   at Microsoft.Identity.Client.Extensions.Msal.MsalCacheHelper..ctor(StorageCreationProperties storageCreationProperties, TraceSource logger, HashSet`1 knownAccountIds, FileSystemWatcher cacheWatcher)
   at Microsoft.Identity.Client.Extensions.Msal.MsalCacheHelper.CreateAsync(StorageCreationProperties storageCreationProperties, TraceSource logger)
   at Azure.Identity.MsalPublicClient.InitializeAsync()
   at Azure.Identity.MsalPublicClient.EnsureInitializedAsync(Boolean async)
   at Azure.Identity.MsalPublicClient.GetAccountsAsync()
   at Azure.Identity.SharedTokenCacheCredential.GetAccountAsync()
   at Azure.Identity.SharedTokenCacheCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex)
   at Azure.Identity.SharedTokenCacheCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
   at Azure.Identity.SharedTokenCacheCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
   at Azure.Identity.DefaultAzureCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthenticateRequestAsync(HttpMessage message, Boolean async, AuthenticationChallenge challenge)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.ProcessCoreAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.HttpPipeline.SendRequest(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.GetPage[T](Uri firstPageUri, String nextLink, Func`1 itemFactory, String operationName, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.Secrets.SecretClient.<>c__DisplayClass13_0.<GetPropertiesOfSecrets>b__0(String nextLink)
   at Azure.Core.PageResponseEnumerator.<>c__DisplayClass0_0`1.<CreateEnumerable>b__0(String continuationToken, Nullable`1 pageSizeHint)
   at Azure.Core.PageResponseEnumerator.FuncPageable`1.AsPages(String continuationToken, Nullable`1 pageSizeHint)+MoveNext()
   at Azure.Pageable`1.GetEnumerator()+MoveNext()
   at test.Program.Main(String[] args) in /home/heaths/test/Program.cs:line 12

Sample code:

using System;
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

namespace test
{
    class Program
    {
        static void Main(string[] args)
        {
            var client = new SecretClient(new Uri("https://heathskv.vault.azure.net"), new DefaultAzureCredential());
            foreach (SecretProperties secret in client.GetPropertiesOfSecrets())
            {
                Console.WriteLine(secret.Id);
            }
        }
    }
}

Using packages:

    <PackageReference Include="azure.identity" Version="1.2.0-preview.4" />
    <PackageReference Include="Azure.Security.KeyVault.Secrets" Version="4.0.3" />

[sig9]

I am getting the same error on an Ubuntu 18.04 vm running in Azure. I have azure.identity 1.2.0-preview.4.

[joshfree]

Moving to August milestone

[AlexanderSher]

Moving to October milestone.
Currently DefaultAzureCredential swallows this exception, but this behavior will be reverted and fix should align with how VisualStudioCodeCredential must handle missing Keyring.

[AlexanderSher]

LinuxKeyRingAccessor implementation: https://github.com/AzureAD/microsoft-authentication-extensions-for-dotnet/blob/master/src/Microsoft.Identity.Client.Extensions.Msal/Accessors/LinuxKeyRingAccessor.cs

[ohadschn]

We're hitting a similar issue in our container (CDPx ubuntu-1804) executions:

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex)
at Azure.Identity.SharedTokenCacheCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.SharedTokenCacheCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex)
at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(HttpMessage message, Boolean async, CancellationToken cancellationToken)
at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, Boolean async)
at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
at [REDACTED - our product code]
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace -----
at Microsoft.Identity.Client.Extensions.Msal.LinuxKeyringAccessor..ctor(String cacheFilePath, String keyringCollection, String keyringSchemaName, String keyringSecretLabel, String attributeKey1, String attributeValue1, String attributeKey2, String attributeValue2, TraceSourceLogger logger)
at Microsoft.Identity.Client.Extensions.Msal.MsalCacheStorage.Create(StorageCreationProperties creationProperties, TraceSource logger)
at Microsoft.Identity.Client.Extensions.Msal.MsalCacheHelper..ctor(StorageCreationProperties storageCreationProperties, TraceSource logger, HashSet`1 knownAccountIds, FileSystemWatcher cacheWatcher)
at Microsoft.Identity.Client.Extensions.Msal.MsalCacheHelper.CreateAsync(StorageCreationPrope

It looks like a similar issue was hit in the past for a different provider: https://github.com/Azure/azure-sdk-for-net/issues/12749. Would it be possible to implement a more generic solution, where provider failures are reported more clearly (e.g. "provider not available" / "all default providers failed").

[AlexanderSher]

Issue has been caused by the MsalCacheStorage and has been fixed in microsoft.identity.client.extensions.msal version 2.12.0. Azure.Identity has updated its reference in 1.2.0-preview.6.

@ohadschn , what version of Azure.Identity do you have?

[ohadschn]

We're using Azure.Identity 1.2.3 and Azure.Core 1.5.0

[apavlunenko]

Issue is reproducible every time using 1.2.3
Downgrading to 1.2.2 resolves the problem

[schaabs]

This is fixed by https://github.com/Azure/azure-sdk-for-net/pull/15944 and will be released in the 1.3.0 release of Azure.Identity

[lextas]

I am still experiencing this bug but only when I try to access the key vault from Program.cs. I am trying to get a key from the vault to register a Serilog sink. Since Startup is the only place where I can register the logger it needs to be done there.

The following line works locally (Windows) but not on Azure (running in a Linux docker container)

var keyVault = new SecretClient(new Uri(context.Configuration.GetValue<string>("KeyVault")), new DefaultAzureCredential());

The same line is used in the UseServices block and there it runs fine. So it seems like it has to do with the fact that the Azure identity is somehow not present when starting up the container?

It's a dotnet core 3.1 solution with Azure.Identity v1.3.0.

[heaths]

@lextas I recommend opening a new issue since this seems unrelated. This bug was a problem with azure CLI credentials running in a WSL2 container. If your managed identity isn't detected from within a container that may be a different issue entirely.