Microsoft-authentication-library-for-dotnet: SuggestedCache is null for ROPC (was: Using token cache serialization for IPublicClientApplication in the Azure Functions)

[vkuzin]

We are looking for the recommended pattern to use for token cache serialization if using IPublicClientApplication in the Azure Functions?

While reviewing documentation on using Token cache for a public client, the recommended approach seems to be to use file-based token cache. This works great for standard desktop applications, but while trying to use it in the Azure Functions and throwing File Not found exceptions. Overall it feels like the file-based approach is not a good match for this scenario, and without any cache, default in-memory doesn't work ( await App.GetAccountsAsync(); always returns 0)

Is there is a way to either 1) enable in-memory cache in such a scenario or 2) implement IDistributedCache to use with Redis?

We are using Microsoft.Identity.Client; instead of Microsoft.Identity.Web, and it seems that we cant use AddInMemoryTokenCaches or AddDistributedTokenCaches in this case.

Can you please shed some light on the options to handle token cache serialization in the case of IPublicClientApplication in the Azure Functions?

[jmprieur]

@vkuzin : this is a good point, you don't need to use a file based approach in a public client application. you can definitively use an in-memory or distributed token cache. You're in the case of a hybrid aprroach, where you'll use MSAL.NET, but will use the caches provided by Microsoft.Identity.Web:

See https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/wiki/Is-MSAL.NET-right-for-me%3F and more specifically

There is a sample that illustrates what you want to do. It's using a confidential client application, but would work exactly the same way with a public client application: https://github.com/Azure-Samples/active-directory-dotnet-v1-to-v2/blob/b48c10180665260a1aec78a9acf7d1b1ff97e5ba/ConfidentialClientTokenCache/Program.cs#L83

cc: @jennyf19 @bgavrilMS

[vkuzin]

Thank you, @jmprieur, for providing this information. At this point, I am trying to get simple DistributedMemoryCache working as follows:

TokenCacheHelper

public static class TokenCacheHelper
    {
        public static IMsalTokenCacheProvider CreateTokenCache()
        {
            IServiceCollection services = new ServiceCollection();
            services.AddDistributedTokenCaches();
            services.AddDistributedMemoryCache();
            IServiceProvider serviceProvider = services.BuildServiceProvider();
            IMsalTokenCacheProvider msalTokenCacheProvider = serviceProvider.GetRequiredService<IMsalTokenCacheProvider>();
            return msalTokenCacheProvider;
        }
}

and its being used as

IPublicClientApplication app = PublicClientApplicationBuilder.Create(clientId)
                 .WithAuthority(authority)
                 .WithDefaultRedirectUri()
                 .Build();

// enable cache
 IMsalTokenCacheProvider msalTokenCacheProvider = TokenCacheHelper.CreateTokenCache();
msalTokenCacheProvider.Initialize(app.UserTokenCache);

I am running .NET Core 3.1, and if I try to use Microsoft.Identity.Web 1.9.1 with Microsoft.Extensions.DependencyInjection 5.0 fails with

Could not load file or assembly 'Microsoft.Extensions.DependencyInjection, Version=5.0.0.1, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.

Downgrading DependencyInjection to something like 3.1.14 gets rid of this error but adds

Microsoft.Extensions.DependencyInjection: Unable to resolve service for type 'Microsoft.Extensions.Logging.ILogger 1[Microsoft.Identity.Web.TokenCacheProviders.Distributed.MsalDistributedTokenCacheAdapter]' while attempting to activate 'Microsoft.Identity.Web.TokenCacheProviders.Distributed.MsalDistributedTokenCacheAdapter'

Is there a way to make this approach work with .NET Core 3.1 (if so, which package versions), or upgrade to .NET 5 is required?
Any pointers are appreciated.

[jmprieur]

@vkuzin : I cannot repro it for the moment: I just changed this line:

https://github.com/Azure-Samples/active-directory-dotnet-v1-to-v2/blob/b48c10180665260a1aec78a9acf7d1b1ff97e5ba/ConfidentialClientTokenCache/ConfidentialClientTokenCache.csproj#L5

to be

    <TargetFrameworks>net5.0; net472; netcoreapp3.1</TargetFrameworks>

and this runs fine. Is it also the case for you? This should work.

Would you have repro steps? or code to share?

[vkuzin]

@jmprieur so ruling out any complexities around .NET 5 and any other libraries we are using, the goal is to make cache work with IPublicClientApplication. If I take an example that you've shared initially: https://github.com/Azure-Samples/active-directory-dotnet-v1-to-v2/blob/b48c10180665260a1aec78a9acf7d1b1ff97e5ba/ConfidentialClientTokenCache/Program.cs#L83

Keep the CreateTokenCache function intact, but update Main() to use IPublicClientApplication as following

        static async Task Main(string[] args)
        {
            string clientId = "<client id>";
            string tenant = "<tenant id>";
            string username = "<user name>";
            string password = "<password>";
            string[] scopes = new[] { "User.Read.All" };


            IPublicClientApplication app = PublicClientApplicationBuilder.Create(clientId)
                .WithAuthority(string.Concat("https://login.microsoftonline.com/", tenant))
                .WithDefaultRedirectUri()
                .Build();

            CacheImplementationDemo cacheImplementation = CacheImplementationDemo.InMemory;

            //   Create the token cache(4 possible implementations)
            IMsalTokenCacheProvider msalTokenCacheProvider = CreateTokenCache(cacheImplementation);

            msalTokenCacheProvider.Initialize(app.UserTokenCache);


            var securePassword = new NetworkCredential("", password).SecurePassword;

            // Acquire a token(twice)
            var result = await app.AcquireTokenByUsernamePassword(scopes, username, securePassword)
                .ExecuteAsync();
            Console.WriteLine(result.AuthenticationResultMetadata.TokenSource);

            result = await app.AcquireTokenByUsernamePassword(scopes, username, securePassword)
                .ExecuteAsync();
            Console.WriteLine(result.AuthenticationResultMetadata.TokenSource);
        }

it will fail with

| $exception | {"Value cannot be null. (Parameter 'key')"} | System.ArgumentNullException

 | SerializationStackTraceString | " at Microsoft.Extensions.Caching.Memory.MemoryCache.CreateEntry(Object key)\r\n at Microsoft.Extensions.Caching.Memory.CacheExtensions.Set[TItem](IMemoryCache cache, Object key, TItem value, MemoryCacheEntryOptions options)\r\n at Microsoft.Identity.Web.TokenCacheProviders.InMemory.MsalMemoryTokenCacheProvider.WriteCacheBytesAsync(String cacheKey, Byte[] bytes)\r\n at Microsoft.Identity.Web.TokenCacheProviders.MsalAbstractTokenCacheProvider.OnAfterAccessAsync(TokenCacheNotificationArgs args)\r\n at Microsoft.Identity.Client.TokenCache.Microsoft.Identity.Client.ITokenCacheInternal.OnAfterAccessAsync(TokenCacheNotificationArgs args)\r\n at Microsoft.Identity.Client.TokenCache.Microsoft.Identity.Client.ITokenCacheInternal.SaveTokenResponseAsync(AuthenticationRequestParameters requestParams, MsalTokenResponse response)\r\n at Microsoft.Identity.Client.Cache.CacheSessionManager.SaveTokenResponseAsync(MsalTokenResponse tokenResponse)\r\n at Microsoft.Identity.Client.Internal.Requests.RequestBase.CacheTokenResponseAndCreateAuthenticationResultAsync(MsalTokenResponse msalTokenResponse)\r\n at Microsoft.Identity.Client.Internal.Requests.UsernamePasswordRequest.ExecuteAsync(CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.ApiConfig.Executors.PublicClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenByUsernamePasswordParameters usernamePasswordParameters, CancellationToken cancellationToken)\r\n at ConfidentialClientTokenCache.Program.Main(String[] args)

Running the same exact code with cache call commented out is getting access token successfully. Thoughts on what is missing in order to make this caching work with PublicClientApplicaiton?

[jmprieur]

@vkuzin thanks for this information.
This is an MSAL.NET bug: the suggestedCacheKey is null here.

The token cache serializers were built for confidential client applications, but your scenario also makes sense.

System.ArgumentNullException
HResult=0x80004003
Message=Value cannot be null. (Parameter 'key')
Source=Microsoft.Extensions.Caching.Memory
StackTrace:
at Microsoft.Extensions.Caching.Memory.MemoryCache.CreateEntry(Object key)
at Microsoft.Extensions.Caching.Memory.CacheExtensions.SetTItem
at Microsoft.Identity.Web.TokenCacheProviders.InMemory.MsalMemoryTokenCacheProvider.WriteCacheBytesAsync(String cacheKey, Byte[] bytes) in C:\gh\microsoft-identity-web\src\Microsoft.Identity.Web\TokenCacheProviders\InMemory\MsalMemoryTokenCacheProvider.cs:line 82

[bgavrilMS]

Not sure this is a bug @jmprieur. The suggestedCacheKey is, as the name implies, suggested and optional. In particular, we did not design it for the PublicClient.

[jmprieur]

@vkuzin are you sure you want to use ROPC? this will fail for other reasons in most tenants (conditional access).
Don't you want to use a secured azure function with OBO ?
or use client credentials?

See https://github.com/AzureAD/microsoft-identity-web/wiki/Azure-Functions

In Azure functions we don't think you want to use a public client.

[bgavrilMS]

Yes, OBO would be better here.

But just for the sake of the conversation, if I were to design a distributed cache for ROPC, I would use UPN / username as the cache key. I would also make sure to call AcquireTokenSilent, as AcquireTokenByUsernamePassword does not use the cache.

[vkuzin]

@jmprieur, @bgavrilMS - there is a valid use case why we need to use ROPC. We've discussed it previously, in issue #2407.

The initial pattern that I've seen shown in the Azure Samples repo.

Just to confirm my understanding based on the above discussion: MSAL doesn't currently support caching for Public Client Applications at this time, and we need to look for an alternative/custom cache solution?

[jmprieur]

@vkuzin : there is a default in memory cache in MSAL.NET, but the tokens will be lost when you Azure function exits
(Asuming you use the right pattern acquireTokenSilent/AcquireTokenXX)

I guess you want to enable persistence of tokens across calls of the Azure function to save time?

[vkuzin]

@jmprieur Correct, the default in-memory cache was not sufficient for me, as it would reset every time the azure function would be triggered by the service bus queue. It only worked in case if I had to do multiple operations on a single function trigger. As a workaround, I added quick caching based on the System.Runtime.Caching library. Below is the code sample, in case it will help someone.

1) Add TokenCacheHelper class as follows

```
public static class TokenCacheHelper
{
public static void EnableSerialization(ITokenCache tokenCache)
{
tokenCache.SetBeforeAccess(BeforeAccessNotification);
tokenCache.SetAfterAccess(AfterAccessNotification);
}

    private static readonly string TokenKey = "AuthCache";
    private static readonly ObjectCache TokenCache = MemoryCache.Default;

    private static void BeforeAccessNotification(TokenCacheNotificationArgs args)
    {
        args.TokenCache.DeserializeMsalV3(TokenCache.Contains(TokenKey)
                ? (byte[])TokenCache.GetCacheItem(TokenKey)?.Value : null);
    }

    private static void AfterAccessNotification(TokenCacheNotificationArgs args)
    {
        // if the access operation resulted in a cache update
        if (args.HasStateChanged)
        {
            CacheItemPolicy policy = new CacheItemPolicy
            {
                Priority = CacheItemPriority.Default,
                AbsoluteExpiration = DateTimeOffset.Now.AddHours(1)
            };

            var cacheItem = new CacheItem(TokenKey, args.TokenCache.SerializeMsalV3());
            TokenCache.Set(cacheItem, policy);
        }
    }
}

```

2) Enable serialization after your app object is built
TokenCacheHelper.EnableSerialization(app.UserTokenCache);

3) In case you need the ability to persist the cache to other storage, update TokenCacheHelper to use Redis Cache, Azure Storage, SQL, or whatever you desire.

[bgavrilMS]

@vkuzin - this will work if you have a limited number of tokens. If the number of tokens (i.e. user accounts) grows (say over 10.000 users), a flat cache will not perform well and for a cache size of over ~100k users you will see latencies measure in seconds.

There are 2 improvements that can be made:

1. Disable ADAL legacy cache via WithLegacyCacheCompatibility(false) - this makes a big difference
2. Consider partitioning your cache, which is what we do for web scenarios with the SuggestedCacheKey. I think you could use the username (upn) as cache key in this case.

https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/wiki/High-availability#use-the-token-cache

[jmprieur]

@bgavrilMS this is a public client application

[vkuzin]

@bgavrilMS In my case azure functions utilizing service account, therefore no need for considerations of thousand users - thank you for additional details, these might be useful for someone's case.

[henrik-me]

@vkuzin : Even if the suggested cache key is null, you can still provide your own cache key that makes for a good partitioning for your scenario.

Public client is meant to be used with few accounts and thus each cache blob should be small enough, however if you start having multiple accounts you should partition based on something like the account (user). The cache eviction also becomes important as MSAL will not handle the lifetime of the items persisted outside of MSAL. MSAL will only care for the parts that are being loaded into the public client object (this is true for confidential client as well).