Microsoft-authentication-library-for-dotnet: [Bug] "email" scope forces token refresh even if there are valid cached tokens

[wmmartins]

MSAL 4.7.1
net45

• Desktop / Mobile
  
  
  
  • [x] Interactive
  
  
  • [ ] Integrated Windows Auth
  
  
  • [ ] Username Password
  
  
  • [ ] Device code flow (browserless)
  
  

I use the following code to initialize IPublicClientApplication:

var client = PublicClientApplicationBuilder.Create(strClientId).Build();
client.UserTokenCache.EnableSerialization();

Where the implemetnation of UserTokenCache.EnableSerialization() is taken from here: https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-net-token-cache-serialization#simple-token-cache-serialization-msal-only

Then I get a token multiple times using this line:

var authResult = await client.AcquireTokenSilent(new string[] {strCustomScope, "email"}, cachedTokenAccount).ExecuteAsync()

I get new tokens every time, even if I have valid (not expired) tokens in the cache.
Because of that I get error introduced by this update: https://docs.microsoft.com/en-us/azure/active-directory/develop/reference-breaking-changes#march-2019

Removing "email" scope fixes this and I get tokens from the cache if not expired.

[jmprieur]

@wmmartins : can you please try with a more recent version of MSAL.NET? A lot of issues were fixed since 2.7.1

[wmmartins]

@jmprieur it was a typo, sorry. It's actually 4.7.1.

[bgavrilMS]

Investigating anyway, as it seems serious.

[bgavrilMS]

I definitely do not see this with the MSAL 4.7.1 and we even have integration tests that protect against this scenario. Questions:

1. What is the value of your strCustomScope ?
2. When you perform AcquireTokenInteractive do you use the same scopes?
3. Does it repro for you on MSAL 4.7.1 ?

[wmmartins]

I was able to recreate it on a demo tenant. Here is a code that reproduces the problem:

var scopes = new[] { "api://a2b0838d-c26a-45fb-852b-8e67910219e5/Custom.Scope", "email" };

var authContext = PublicClientApplicationBuilder
    .Create("6c324c34-154c-4a63-98de-aa8d0bffce45")
    .WithAuthority("https://login.microsoftonline.com/common")
    .WithRedirectUri("https://login.microsoftonline.com/common/oauth2/nativeclient")
    .Build();


var authResult = await authContext.AcquireTokenInteractive(scopes)
    .WithPrompt(Prompt.SelectAccount)
    .ExecuteAsync();

var account = authResult.Account;

authResult = await authContext.AcquireTokenSilent(scopes, account).ExecuteAsync();
authResult = await authContext.AcquireTokenSilent(scopes, account).ExecuteAsync();
authResult = await authContext.AcquireTokenSilent(scopes, account).ExecuteAsync();
authResult = await authContext.AcquireTokenSilent(scopes, account).ExecuteAsync();

Answers to your questons:

1. api://a2b0838d-c26a-45fb-852b-8e67910219e5/Custom.Scope
2. Yes
3. Yes, I'm using MSAL 4.7.1

[bgavrilMS]

I can repro this, investigating the root cause. Thank you for the detailed steps @wmmartins

[bgavrilMS]

So what seems to be happening is that if you are requesting a token for a custom scope and something else, AAD returns a token exclusively for that scope.

|Scopes in Request | Scopes in AAD Response |
|--|--|
customScope, User.Read, email | customScope
customScope email | customScope
customScope profile | customScope
User.Read email | email openid profile User.Read
email | email openid profile User.Read
tasks.read | email openid profile Tasks.Read
https://app.vssps.visualstudio.com/user_impersonation | https://app.vssps.visualstudio.com/user_impersonation
https://app.vssps.visualstudio.com/user_impersonation email | https://app.vssps.visualstudio.com/user_impersonation

@jmprieur - are you aware of this behavior? This causes apps to send a lot of requests to EVO, as essentially we are refreshing the RT on each AcquireTokenSilent

@wmmartins - as a workaround, you can stop requesting for email scope and just look at the authResult.Account.Username which is ussually the email address. You can also use the Graph API to send emails.

[jmprieur]

Yes @bgavrilMS @wmmartins, you can only request to Azure AD a token for the same resource. here there are 2 resources "api://a2b0838d-c26a-45fb-852b-8e67910219e5 and Microsoft Graph.
You can, however get consent for both resources upfront.
See https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-net-user-gets-consent-for-multiple-resources

The way to go is ask for the first scope, and then the second by executing AcquireTokenSilent. The token you'll get then will have both scopes

[jmprieur]

Closing this question. this is by design

[hpsin]

@wmmartins - as a workaround, you can stop requesting for email scope and just look at the authResult.Account.Username which is ussually the email address. You can also use the Graph API to send emails.

@bgavrilMS - note that this is not quite the right call - there should be a way to get an id token (what fills in the Account obect) and look at the email field there. That is the correct way to get an email address for the user. With more user types coming into the market that don't use the email address as the UPN, username should not be relied on for email address.

[bgavrilMS]

@hpsin - yes, MSAL exposes the IdToken to the user as a raw string (there is some work to do to expose it as a set of claims). authResult.Account.Username is populated with the "preffered_username" scope from the IDToken (with a fallback to the upn scope). So, yes, email scope is a better bet, but I don't think MSALs are handling this scope well (we have a separate email thraed about this).

[bgavrilMS]

So for all intents and purposes this is an ESTS bug - ESTS should throw an error explaining that it is not possible to request an AT for both:

• UserInfo (email scope)
• Custom scope

Instead ESTS returns an access token for the custom scope.

[hpsin]

Not quite - this is a ESTS bug that allows a client to specifically ask for both user.read and a secondary resource. Clients are always allowed to request a (single) access token for Resource+openid+email+profile - that's the spec.

This is an MSAL bug as well - it should not be checking the cache for openid scopes unless the token requested was exclusively openid +Graph scopes

[bgavrilMS]

@hpsin - I fixed the MSAL bug by not looking for the openid offline_access and profile in the cache (but not email!) MSALs do not distinguish between Graph scopes and other scopes and I wouldn't want to go down that route. ESTS needs to provide a consistent approach.

My understanding once the ESTS bug is fixed is that asking for email and any other resource except Graph will fail.

[hpsin]

@bgavrilMS why not email? They're all openid scopes. The bug is about asking for user.read and a non-Graph resource.

[bgavrilMS]

MSAL libraries ask for the openid offline_access and profile - this is required for MSALs to function, i.e. request RTs, populate the username field in the Account object. We don't expose email probably because it is not guaranteed that a user object actually has an email. This may or may not be the best design decision, but it is a decision which all ADAL and MSAL libraries follow for a very long time.

The issue is that we have customers asking for the email scope. And this is causing problems, because we get cache misses.

|request|response|impact|
|-|-|-|
|custom-scope email|custom_scope| cache miss -> RT will always be refreshed (!!) |
|graph-scope email|graph-scope email|cache hit -> AT will be reused|

We need to figure out how to handle this, preferably without MSAL being able to distinguish between a graph and non-graph scope. If AAD throws an exception when using custom_scope email, then this is a solution.

[rayluo]

IMHO:

1. The current AAD behavior (i.e. not issuing AT for multiple resources) is technically NOT a bug, because the OAuth2 specs allows authorization server to choose a subset of scopes to be fulfilled.
2. The current MSAL behavior (i.e. requesting more scopes than cached token means a cache miss) is not a bug either, because that's how the scope concept in OAuth2 is supposed to work.
3. Proposing AAD to return an explicit - presumably invalid_scope - error for situation 1 above, would work, in a sense that AAD explicitly disallow the "consenting multiple resources in one interaction" usage. But if that is not the price we want to pay, then AAD and MSAL need to come up with a non-OutOfBand way to convey "hey this is an RT that will work for ALL scopes you just requested". Perhaps something like this:

Client -> AAD: token request with scope "resource_foo/read resource_bar/write"

AAD -> Client: token response WITHOUT scope parameter, and with RT_for_foo_and_bar,
    with an immediately-expired AT.
    (Here returning an AT to satisfy the OAuth2 specs requirement for an successful response,
    but the AT is immediately expired in  order to
    trigger app developer to use MSAL's AcquireTokenSilent(). See below.)

Client -> ResourceFoo: presenting AT, got an "AT expired" error,
    and then the client would supposedly fall back to AcquireTokenSilent() anyway.
    But we will need to educate the app developer to use scope="resource_foo/read"
    which will then use RT_for_foo_and_bar to acquire a new AT_for_foo.

Client -> ResourceBar:
    Same as above, except the app developer would use scope="resource_bar/write"

From  now on, client will call AcquireTokenSilent(scope= foo or bar), as usual,
and there will always be cache hit.

[jmprieur]

@bgavrilMS @henrik-me @jennyf19 @trwalke @neha-bhargava
Give it's external we might want to close it with a link from FAQs ?
do you agree?