Microsoft-authentication-library-for-dotnet: Add access to refresh_token in Microsoft.Identity.Client.AuthenticationResult

[MatLeger]

Is your feature request related to a problem? Please describe.
I need to use IPublicClientApplication.AcquireTokenWithDeviceCode(...) to handle authentication on some UI limited devices. These devices are running legacy code which requires both the access_token and the refresh_token. I can get the AuthenticationResult.AccessToken but there is no AuthenticationResult.RefreshToken. Using fiddler I can see the response from the server returns both an access_token and a refresh_token but for some reason the refresh token is not exposed in the result and I don't see an easy way to get to it. I need to pass both of these tokens off to the legacy code so the system can work as it always has.

Describe the solution you'd like
The easiest solution I'd like to see is to add a RefreshToken property to AuthenticationResult. Then its a matter of just handing the AuthenticationResult.AccessToken and AuthenticationResult.RefreshToken off to the legacy code and I'm done.

Or if there is a way to get at the refresh_token and someone can show me how to do it that would be appreciated as well.

```C#
IPublicClientApplication app = PublicClientApplicationBuilder
.Create(AppSettings.ClientId)
.WithTenantId(AppSettings.TenantId)
.Build();

DeviceCodeFlow oauth = new DeviceCodeFlow(app);

AuthenticationResult result = await oauth.AcquireTokenAsync(new[] { "...scopes..." }, deviceCodeCallback =>
{
// Prints message instructing user to go to https://microsoft.com/devicelogin and enter device code
Console.WriteLine(deviceCodeCallback.Message);
return Task.FromResult(0);
});

CallLegacyCodeWithTokens(result.Account.Username, result.AccessToken, result.RefreshToken);
```

[jmprieur]

@MatLeger
MSAL.NET (and MSALs in general) don't expose the refresh tokens as they handle refreshing tokens automatically when you call AcquireTokenSilent. Refreshs may become obsolete when you use them, and therefore the code using them could easily break.

Can we understand a bit better what your legacy code with Token does? and why it needs the refresh tokens? would passing an IPublicClientApplication to get a token using app.AcquireTokenSilent() be good as well?

[MatLeger]

In this case the user is authorizing our application to access their OneDrive account at a later date. The back end system that does this stores both of the tokens and when needed handles the refresh on its own. All I'm doing with MSAL.NET is adding a new (better) way for the user to authenticate on the client if the back end system doesn't already have their tokens. After the DeviceCodeFlow is complete I'm done using MSAL. I do not want to change the legacy code to use IPublicClientApplication. The tokens are all I care about.

Wouldn't AuthenticationResult.AccessToken have the same problem with becoming obsolete? Once a refresh has occurred then both tokens (if RefreshToken is added) from that original AuthenticationResult could be obsolete. In my case once the code has finished authenticating it will not be using IPublicClientApplication again so there is no chance that IPublicClientApplication will refresh them.

For me IPublicClientApplication just a way to add a DeviceCodeFlow when needed and get those tokens into the legacy code with as few lines of new code as possible.

[jmprieur]

Wouldn't AuthenticationResult.AccessToken have the same problem with becoming obsolete? Once a refresh has occurred then both tokens (if RefreshToken is added) from that original AuthenticationResult could be obsolete. In my case once the code has finished authenticating it will not be using IPublicClientApplication again so there is no chance that IPublicClientApplication will refresh them.

Yes, you are right (I did not elaborate on that, but this is true)

Did you consider have the back end perform the On-behalf-of flow (that is from the token it gets from your client, gets its own Access Token/Refresh Token): See https://docs.microsoft.com/en-us/azure/active-directory/develop/scenario-web-api-call-api-overview

Of would the backend have the same client Id as your client? In any case you can have a look at https://github.com/Azure-Samples/active-directory-dotnet-native-aspnetcore-v2

[MatLeger]

The same ClientId is used for IPublicClientApplication and on the backend. The backend code uses the Microsoft.OneDriveSDK nuget package to create an instance of a Microsoft.OneDrive.Sdk.IOneDriveClient with the refresh token using the Microsoft.OneDrive.Sdk.BusinessClientExtensions class:

C# // legacy code executing on backend that grabs refresh token from token storage and then creates IOneDriveClient with the following call: _oneDriveClient = await BusinessClientExtensions.GetSilentlyAuthenticatedClientAsync(config, refreshToken);

I'm not sure how to create an authenticated IOneDriveClient with just the access token acquired by IPublicClientApplication. Even if I can use the access token isn't the access token usually valid for a shorter amount of time? It's possible that this code may not execute soon enough and a refresh will be needed anyway. I'm a little lost on how to bridge the two worlds here without that refresh token.

[MatLeger]

I created a small test app that pulled in the IPublicClientApplication code and the Microsoft.OneDrive.Sdk that the legacy code uses and tried to get them to work together in the same method. Unfortunately I'm not seeing how to do it without the refresh token.

```C#
IPublicClientApplication app = PublicClientApplicationBuilder
.Create(AppSettings.ClientId)
.WithTenantId(AppSettings.TenantId)
.Build();

DeviceCodeFlow oauth = new DeviceCodeFlow(app);

AuthenticationResult result = await oauth.AcquireTokenAsync(new[] { "Files.ReadWrite.All" }, deviceCodeCallback =>
{
Console.WriteLine(deviceCodeCallback.Message);
return Task.FromResult(0);
});

OneDriveClient client = null;
string accessToken = result.AccessToken; // <== this won't work
//string refreshToken = "OAQABAAAAAADCoMpjJXrx..."; // <== refresh_token pulled from response from DeviceCodeFlow

try
{
AdalAuthenticationProvider adalAuthenticationProvider = new AdalAuthenticationProvider(AppSettings.ClientId, AppSettings.ReturnUrl);

// This next call wants a refresh token, not an access token. Passing an access token 
// to this call throws an exception with a 400 "bad request" error code and a message
// "AADSTS9002313: Invalid request. Request is malformed or invalid."
await adalAuthenticationProvider.AuthenticateUserWithRefreshTokenAsync(accessToken, AppSettings.ResourceUrl);

// However if I use fiddler to grab the refresh_token from the DeviceCodeFlow response 
// then this call works and I'm able to get the OneDrive root item.
//await adalAuthenticationProvider.AuthenticateUserWithRefreshTokenAsync(refreshToken, AppSettings.ResourceUrl);

client = new OneDriveClient(AppSettings.BaseUrl, adalAuthenticationProvider);

}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
throw;
}

var root = await client.Drive.Root.Request().GetAsync();

Console.WriteLine($"Root.Name = {root.Name}");

And here's the DeviceCodeFlow class that I implemented to wrap IPublicClientApplication
```C#
public class DeviceCodeFlow
{
    protected IPublicClientApplication App { get; set; }

    public DeviceCodeFlow(IPublicClientApplication app)
    {
        App = app;
    }

    public async Task<AuthenticationResult> AcquireTokenAsync(IEnumerable<string> scopes, Func<DeviceCodeResult, Task> deviceCodeCallback)
    {
        AuthenticationResult result = await AcquireTokenSilent(scopes);
        if (result == null)
        {
            result = await AcquireTokenWithDeviceCodeFlowAsync(scopes, deviceCodeCallback);
        }

        return result;
    }

    private async Task<AuthenticationResult> AcquireTokenSilent(IEnumerable<string> scopes)
    {
        IEnumerable<IAccount> accounts = await App.GetAccountsAsync();
        if (!accounts.Any()) return null;

        try
        {
            // Attempt to get a token from the cache (or refresh it silently if needed)
            return await App.AcquireTokenSilent(scopes, accounts.FirstOrDefault()).ExecuteAsync();
        }
        catch (MsalUiRequiredException)
        {
        }

        return null;
    }

    private async Task<AuthenticationResult> AcquireTokenWithDeviceCodeFlowAsync(IEnumerable<string> scopes, Func<DeviceCodeResult, Task> deviceCodeCallback)
    {
        try
        {
            return await App.AcquireTokenWithDeviceCode(scopes, deviceCodeCallback).ExecuteAsync();
        }
        catch (OperationCanceledException)
        {
            // If you use an override with a CancellationToken, and call the Cancel() method on it, 
            // then this may be triggered to indicate that the operation was cancelled. 
        }
        catch (MsalClientException ex)
        {
            // Verification code expired before contacting the server
            // This exception will occur if the user does not manage to sign-in before a time out (15 mins) and the
            // call to `AcquireTokenWithDeviceCodeAsync` is not cancelled in between
        }

        return null;
    }
}

[henrik-me]

@MatLeger @jmprieur

I'm not familiar with the OneDrive SDK, however I took a quick look at the AdalAuthenticationProvider for desktop.

I'm not sure why they use the overloads that takes a refresh token as the token cache contains both the refresh token as well as the access token. Also, it's based on ADAL and not MSAL. As far as I can see it's possible to create an AuthenticationProvider which doesn't require the refresh token.

Are you only needing the refresh token for this API or are you using the refresh token for other things?

[MatLeger]

We are using the refresh token for other things as well.

If there is a way to get the OneDrive SDK to work using only the DeviceCodeFlow access token, then I'd like to understand how that's done because it's not obvious to me how to do it. Yet the OneDrive SDK is very clear how it can _easily_ work with a refresh token.

[MatLeger]

The easiest solution I could find to get around this problem is to create my own refresh token DelegateHandler and inject that into the HttpClient pipeline and execute a lambda when the refresh token comes back in a response. Now I'm able to pass the refresh token to the OneDriveClient and keep the legacy code as is. I still feel I should not have to do this and the refresh token should be part of the AuthenticationResult.
```C#
public async Task TestAuthenticationAsync()
{
string refreshToken = string.Empty;

IMsalHttpClientFactory httpClientFactory = new HttpClientFactory(new RefreshTokenHandler(responseRefreshToken => refreshToken = responseRefreshToken));

IPublicClientApplication app = PublicClientApplicationBuilder
                                .Create(AppSettings.ClientId)
                                .WithTenantId(AppSettings.TenantId)
                                .WithHttpClientFactory(httpClientFactory)
                                .Build();

DeviceCodeFlow oauth = new DeviceCodeFlow(app);

AuthenticationResult result = await oauth.AcquireTokenAsync(new[] { "Files.ReadWrite.All" }, deviceCodeCallback =>
{
    Console.WriteLine(deviceCodeCallback.Message);
    return Task.FromResult(0);
});

Console.WriteLine($"RefreshToken: {refreshToken}");

}

public class HttpClientFactory : IMsalHttpClientFactory
{
private HttpMessageHandler MessageHandler { get; set; } = null;

public HttpClientFactory(HttpMessageHandler messageHandler)
{
    MessageHandler = messageHandler;
}

public HttpClient GetHttpClient()
{
    return MessageHandler != null ? new HttpClient(MessageHandler) : new HttpClient();
}

}

public class RefreshTokenHandler : DelegatingHandler
{
Action OnProcessRefreshToken { get; set; }

public RefreshTokenHandler(Action<string> processRefreshToken) : this(new HttpClientHandler(), processRefreshToken)
{
}

public RefreshTokenHandler(HttpMessageHandler innerHandler, Action<string> processRefreshToken)
{
    InnerHandler = innerHandler;
    OnProcessRefreshToken = processRefreshToken;
}

protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
    HttpResponseMessage response = await base.SendAsync(request, cancellationToken);

    if (response.IsSuccessStatusCode && response.RequestMessage.RequestUri.AbsolutePath.EndsWith("/oauth2/v2.0/token"))
    {
        string json = await response.Content.ReadAsStringAsync();
        if (!string.IsNullOrEmpty(json))
        {
            JObject result = JsonConvert.DeserializeObject(json) as JObject;
            if (result != null)
            {
                JToken tokenValue = null;

                if (result.TryGetValue("refresh_token", out tokenValue))
                {
                    OnProcessRefreshToken(tokenValue.ToString());
                }
            }
        }
    }

    return response;
}

}
```

[henrik-me]

Agree, that is the easiest and definitely not a very elegant situation. We are trying really hard to have people not need the refresh token and let the library handle the refresh token as that will enable the library to manage SSO state. We are also aware that there are scenarios where having the refreshtoken is needed. Ideally the OneDrive SDK would be different. We are still discussing internally if we can find a way to let developers get access to the RefreshToken with-out having to do what you went through here.
For now we can't expose the refresh token, if we end up being able to expose the refresh token we will make sure to make that very visible in our release notes.

Your opinion on exposing the refresh token matters, and we are re-considering.

[henrik-me]

@MatLeger We have been discussing this internally. For now the best option is to ask the One Drive SDK team to update. When customers ask it's much more useful than us asking them, thus I recommend that you open an issue for them. You can cross reference us here and we can follow up with them.
CC: @jmprieur

[henrik-me]

@MatLeger : Closing for now. For now we will not be exposing the RT.
CC: @jmprieur

[NourJardali]

I'm having same issue now and need access to Refresh Token. Gonna try the solution suggested above but would be better to access token from Authentication Result

[mantasaudickas]

Maybe I am looking to wrong place.. but if I want to implement a background process which syncs user calendar with our system, how can I handle this without refresh token?
Scenario is simple:

• app generates authorization url where user is redirected and gives consent to read calendar
• after success it returns authorization code, which I use to get access token (and refresh token)
• I want to put refresh token in encrypted storage and use it on scheduled basis (like check for changes each few hours..)

At the moment as I understand this is not possible with MSAL?

[jmprieur]

Does this documentation answer your question @mantasaudickas : https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/wiki/High-availability#pro-active-token-renewal ?

[mantasaudickas]

@jmprieur kind of.. it will mean that background service cannot be shutdown for ever? Load cannot be shared between multiple instances. There cannot be any errors, issues or other problems lasting longer than one hour :)
Yeah.. it answers my question, still answer not really usefull.. :)

[henrik-me]

@mantasaudickas : what you are describing seems like an obo flow. Also you have the refresh token in the cache.

[mantasaudickas]

Is that cache stored anywhere? Is it distributed? Its kind of obo flow.. normally people call it authorization code flow :)

[johnwc]

What is the purpose of not allowing access to the refresh token? If it is apart of standard oAuth flows and usage, why is the MSAL library holding it hostage? I understand that you are trying to make the MSAL library over the top nice by handling the refresh process automatically for us, but there is scenarios where we need access to that refresh token as well. A few here have pointed out that we need to be able to retrieve the refresh token and securely store it for background processes.

[henrik-me]

@johnwc : I don't believe we are holding the RT hostage. We are encapsulating it in the cache of MSAL, and you can share that cache with your background process and there MSAL will have access to the RT. The reason for doing this is to ensure optimal number of refreshes and thus calls to AAD for a new valid token. Any reason you are not sharing the cache directly with your background process?

[johnwc]

@henrik-me Yes, the background processes are azure functions that are completely separate from the front end web app. They are also written in a completely different language. Is there a way to share the cache of MSAL from a C# front end, with an azure functions python background process?

[henrik-me]

The cache is sharable across .NET, Java, Python and soon with Node (currently in preview) as well.
@rayluo for Python
@DarylThayil for Node
@SomkaPe for Java

@navyasric @jmprieur : We should have a sample on this, thoughts?

[johnwc]

@henrik-me Where is the cache stored? How do I share MSAL cache from a App Service(.Net Core 3.1) running on a Windows App Service Plan, with Azure Functions(Python) running on a Linux App Service Plan?

This is good that they are shareable, and it seems like it will solve our issue. What about scenarios where someone is not using MSAL library in a backend service and just needs the raw refresh token?

[henrik-me]

@johnwc : I would like to understand more of the scenarios in the backend service where MSAL is not used. To understand the reasoning for not using the libraries and what may be what we should focus on.

Readings about caching across the platforms:
https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-python-token-cache-serialization
https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-java-token-cache-serialization
https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-net-token-cache-serialization

[johnwc]

@henrik-me We don't have the scenario as of today, but one that I can think of easily would be a language that the MSAL does not support yet. Lile PHP for example. I know we have a fairly large PHP app that is about to be ported to Laravel, it will be using AAD B2C to secure it.

[johnwc]

@henrik-me I understand how the front end web app can use the distributed cache and store it in a SQL server, but how does the backend python service pick this account up and use the cached credentials?

[henrik-me]

The best way to demonstrate this is how Microsoft Identity Web works and how you hook in cache providers. While that is currently only operating for .NET consider that the backed service could be replaced by a version written in Python. This is something I believe we could provide a sample on at some point in time. Essentially the cache is just a blob of data that can be read by either of the libraries and there is not more to it than what it would take to get the RT to the backend service each of the libraries have functionality to take this blob and do it's magic.

https://github.com/AzureAD/microsoft-identity-web

[johnwc]

@henrik-me I was looking at the example of MSAL for web python app using cache serialization, what I don't understand is when it calls accounts = ConfidentialClientApplication.get_accounts() and then ConfidentialClientApplication.acquire_token_silent(scope, account=accounts[0]); on the backend, how would it know what to populate for get_accounts()? The backend has no context of the users, no one logged into the backend function like they do on the front end. Do I pass in the users AAD objectid as the account parameter or what?

[rayluo]

@johnwc

The accounts = get_accounts() and then acquire_token_silent(scope, account=one_account_chosen_from_accounts) pattern is initially designed for Desktop or Mobile app where end user could sign in with multiple accounts. In a web app context, that sample persists the current signed-in end user's token cache into session, and stick with a "One Token Cache per User" model, so that the get_accounts() would still work.

But if you are talking about a so-called "headless" backend service operating without an end user context, it would be a different topic. If it still needs to pull data for/from an end user, you would need to somehow persist his/her RT (thus your interest in this conversation) or persist the entire MSAL Token Cache of this end user.

[johnwc]

@rayluo I'm fine with persisting the token cache into a distributed cache like SQL or KeyVault, we will even write the needed cache provider to read/write from the distributed cache vault. I just need to know how to pick that cache up on the backend and use it for a specific user. The user logs into the front end, and then kicks off a long running process(azure functions) that needs to only run in the context of that user, so that the code only access what the user has access to.

[henrik-me]

Please see the code in Identity.web. MSAL .NET exposes what is called a cache key in the serialization method, that is build based on whether it's a user cache or an app cache.

[johnwc]

@henrik-me How do I call the MSAL python module in the backend function to retrieve one of those cached token entries? That is my dilemma, where do I pass in what user I need to retrieve the token for?

[rayluo]

MSAL Python token cache can be deserialized from a previously serialized content. You can refer to this code snippet. MSAL .Net probably also expose a serialize method.

[johnwc]

@rayluo @henrik-me Ok, so lets take your example link. I see how to add a cache class instance, understood that already. In the context of app = msal.ClientApplication(..., token_cache=cache) we are creating a ClientApplication instance as app. Now, if a user [email protected] logged in on the frontend and initiated the backend to run. In the backend, what do I call in the app object to get the cached credentials for [email protected]?? Which in turn should return the cached token.

[rayluo]

@johnwc This is the other half to my last night's answer.

MSAL .Net probably also expose a serialize method.

MSAL .Net does provide several serialize methods. You should use this latest one for cross-language token cache format. More info here.

Based on your latest description, you probably need to somehow share such serialized blob across your different components (which would be a task for you even if you were just sharing an RT anyway). However, there may be some security concern if you transmit RT or token cache among frontend and backend. (CC: @kalyankrishna1 Is this an onboarding case that you can help with?)

[johnwc]

@rayluo @henrik-me Guys... I've already said that we have written the class for storing/retrieving the token cache, and we are plugging it into the Python MSAL module as your example shows using the token_cache parameter for the constructor of ClientApplication. I need to know how to now call the instance app that is of type ClientApplication. I can only assume that I am calling the method acquire_token_silent(), but what do I pass to it to retrieve a token for a specific account??

As far as the security concern, our custom token cache Class for .Net and Python is reading/writing them into a KeyVault. I plan on sharing this example with sample code here, once I can get the above answer. So that there can be a sample of how to share a token cache between disconnected systems and languages.

Update:

I think I found what I was looking for after digging a little more in the MSAL Python docs, so correct me if I am wrong please. It looks as if I need to call the app.get_accounts() method with the username of the account that I need the token for. That will call the needed methods in the cache class and create the a account object, I will then use that object to call acquire_token_silent() passing it in as a parameter. The acquire_token_silent() method will then call the cache to retrieve the token if one exists.

[rayluo]

I think I found what I was looking for after digging a little more in the MSAL Python docs, so correct me if I am wrong please. It looks as if I need to call the app.get_accounts() method with the username of the account that I need the token for. That will call the needed methods in the cache class and create the a account object, I will then use that object to call acquire_token_silent() passing it in as a parameter. The acquire_token_silent() method will then call the cache to retrieve the token if one exists.

That is correct. It is the "step 2" in our 3-step pattern.

[johnwc]

@rayluo The one thing that I worry about from just looking over the MSAL Python caching class, is that it looks like it assumes that all of the items in the serialized cache belongs to one user, the current active user. In our case, on the web frontend we are writing the cache to the KeyVault in our custom caching class. It inherits MsalAbstractTokenCacheProvider and writes the token cache to the keyvault in the WriteCacheBytesAsync method, using the cacheKey as the KeyVault secret name. I don't think the Python MSAL module calls the cache retrieval like that. @henrik-me thoughts?

[rayluo]

The one thing that I worry about from just looking over the MSAL Python caching class, is that it looks like it assumes that all of the items in the serialized cache belongs to one user, the current active user.

Your observation is correct. And that was what I mean by this earlier comment: "In a web app context, that sample persists the current signed-in end user's token cache into session, and stick with a "One Token Cache per User" model, so that the get_accounts() would still work."

And that is not just MSAL Python. MSAL .Net also has the same guidance/requirement for web app: "In web apps or web APIs, the cache could leverage the session, ... You should keep one token cache per account in web apps or web APIs."

I'm not super familiar with Identity Web. Perhaps it already maintains such a partition of token cache for you. Would like to have my Identity Web teammates to chime in next week. (It is 0:21AM Saturday now.)

[jmprieur]

@rayluo @johnwc: yes, Microsoft.Identity.Web maintains a partition of the token cache for you, and in fact the cache key is now computed by MSAL.NET (SuggestedCacheKey property of the token serialization even args)

The cache key is for the moment:

• Client id for the app token cache. We might move to clientId + tenant ID (for multi-tenant applications)
• a hash of the user assertion in the case of OBO
• the MSAL ID (home object ID + '.' + home tenant ID) in all the other cases