Azure-activedirectory-identitymodel-extensions-for-dotnet: IDX10511 and IDX10634 errors when using custom JWT Bearer with AzureAD Bearer

[udlose]

I am using AzureAD authentication along with a custom JWT Bearer auth. I'm using Microsoft.Identity.Web to facilitate the AzureAD auth. I'm not sure if this issue should be posted there so I'm posting it here since the call stack for the error is from Microsoft.IdentityModel.Tokens.

I should note that the authentication for both works as expected. It seems that this error only happens when using the non-default auth scheme. The error happens irrespective of _which_ scheme is defined as the default - i.e. if I switch the default scheme and use an endpoint with the non-default auth scheme the error is present (switching from services.AddAuthentication("CustomJWTBearerToken") to services.AddAuthentication(AzureADDefaultsJwtBearerAuthenticationScheme)

Environment:

• .NET Core 3.1.12
• System.IdentityModel.Tokens.Jwt v6.8.0
• Microsoft.Identity.Web v1.6.0

Here is my code.
In Startup:

        public void ConfigureServices(IServiceCollection services)
        {
            // elided for brevity

            services.AddProtectedWebApi(Configuration, subscribeToJwtBearerMiddlewareDiagnosticsEvents: Environment.IsLocal());

            services.AddControllers(config =>
            {
                var policy = new AuthorizationPolicyBuilder()
                    .RequireAuthenticatedUser()
                    .Build();
                config.Filters.Add(new AuthorizeFilter(policy));
            })
            .AddNewtonsoftJson();

            // elided for brevity

Here is the class for the AuthenticationExtensions extension method:

    public static class AuthenticationExtensions
    {
        // Value taken from https://github.com/aspnet/AADIntegration/blob/26c7e2cdf2fb7977c0d06becd8332aebc82177ee/src/Microsoft.AspNetCore.Authentication.AzureAD.UI/AzureADDefaults.cs#L33
        public const string AzureADDefaultsJwtBearerAuthenticationScheme = "AzureADJwtBearer";

        public static IServiceCollection AddProtectedWebApi(this IServiceCollection services, IConfiguration configuration,
            bool subscribeToJwtBearerMiddlewareDiagnosticsEvents = false)
        {
            services.AddAuthentication("CustomJWTBearerToken")
                .AddJwtBearer("CustomJWTBearerToken", options =>
                {
                    ConfigureCustomJwtBearerConfigurationOptions(services, configuration, options);
                })
                .AddMicrosoftIdentityWebApi(configuration,
                    jwtBearerScheme: AzureADDefaultsJwtBearerAuthenticationScheme,
                    subscribeToJwtBearerMiddlewareDiagnosticsEvents);

            // configure the options for validating AzuerAD Jwt Bearer Access Tokens
            services.Configure<JwtBearerOptions>(AzureADDefaultsJwtBearerAuthenticationScheme, ConfigureAzureADJwtBearerAuthenticatonOptions);

            // update the default authorization policy to accept all authentication schemes
            // https://docs.microsoft.com/en-us/aspnet/core/security/authorization/limitingidentitybyscheme?view=aspnetcore-2.2#use-multiple-authentication-schemes
            services.AddAuthorization(options =>
            {
                var defaultAuthorizationPolicyBuilder = new AuthorizationPolicyBuilder(
                        "CustomJWTBearerToken",
                        AzureADDefaultsJwtBearerAuthenticationScheme)
                    .RequireAuthenticatedUser();

                options.DefaultPolicy = defaultAuthorizationPolicyBuilder.Build();
            });

            return services;
        }

        private static void ConfigureCustomJwtBearerConfigurationOptions(IServiceCollection services,
            IConfiguration configuration, JwtBearerOptions options)
        {
            var tmoSection = configuration.GetSection("AppSettings:TokenManagementOptions");
            var audience = tmoSection.GetValue<string>("Audience");
            var issuer = tmoSection.GetValue<string>("Issuer");
            var key = tmoSection.GetValue<string>("Key");
            options.Audience = audience;
            options.ClaimsIssuer = issuer;
            options.IncludeErrorDetails = true;
            options.SaveToken = true;
            options.TokenValidationParameters = new TokenValidationParameters
            {
                ClockSkew = TimeSpan.FromMinutes(5),
                ValidateLifetime = true,
                ValidateAudience = true,
                ValidateIssuer = true,
                ValidateIssuerSigningKey = true,
                ValidIssuer = issuer,
                ValidAudience = audience,
                IssuerSigningKey = new SymmetricSecurityKey(Convert.FromBase64String(key)) {KeyId = "some key id here" }, 
                NameClaimType = "preferred_username",
                RequireSignedTokens = true,
                ValidTypes = new[] { JwtConstants.TokenType }
            };

            // validate that the options have been configured correctly
            options.Validate();
        }

        private static void ConfigureAzureADJwtBearerAuthenticatonOptions(JwtBearerOptions options)
        {
            // Microsoft.Identity.Web uses 'ClientId' as Audience and performs AudienceValidation
            // So, we don't need to explicitly initialize a value for ValidAudiences
            options.TokenValidationParameters.ValidateAudience = true;
            options.TokenValidationParameters.ValidateIssuer = true;

            // set the NameClaimType so the ASP.NET Identity is wired appropriately to pull the identity from the AzureAdToken
            options.TokenValidationParameters.NameClaimType = "preferred_username;

            // Microsoft.Identity.Web implementation sets all other relevant settings for us

            // validate that the options have been configured correctly
            options.Validate();
        }
    }

Scenario 1 - using "CustomJWTBearerToken" as default auth scheme end executing endpoint using AzureADDefaultsJwtBearerAuthenticationScheme.

Stack trace from Debug output:

Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: Failed to validate the token.

Microsoft.IdentityModel.Tokens.SecurityTokenInvalidSignatureException: IDX10511: Signature validation failed. Keys tried: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'. , KeyId: elided-for-security
'. 
kid: 'elided-for-security'. 
Exceptions caught:
 'System.NotSupportedException: IDX10634: Unable to create the SignatureProvider.
Algorithm: 'RS256', SecurityKey: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'.'
 is not supported. The list of supported algorithms is available here: https://aka.ms/IdentityModel/supported-algorithms
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(Byte[] encodedBytes, Byte[] signature, SecurityKey key, String algorithm, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
'.
token: '{"typ":"JWT","alg":"RS256","kid":"elided-for-security"}.jwt body elided'.
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, TokenValidationParameters validationParameters, SecurityToken& validatedToken)
   at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
jwt auth failed
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: CustomJWTBearerToken was not authenticated. Failure message: IDX10511: Signature validation failed. Keys tried: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'. , KeyId: elided-for-security
'. 
kid: 'elided-for-security'. 
Exceptions caught:
 'System.NotSupportedException: IDX10634: Unable to create the SignatureProvider.
Algorithm: 'RS256', SecurityKey: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'.'
 is not supported. The list of supported algorithms is available here: https://aka.ms/IdentityModel/supported-algorithms
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(Byte[] encodedBytes, Byte[] signature, SecurityKey key, String algorithm, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
'.
token: '{"typ":"JWT","alg":"RS256","kid":"elided-for-security"}.jwt body elided'.
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: CustomJWTBearerToken was not authenticated. Failure message: IDX10511: Signature validation failed. Keys tried: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'. , KeyId: elided-for-security
'. 
kid: 'elided-for-security'. 
Exceptions caught:
 'System.NotSupportedException: IDX10634: Unable to create the SignatureProvider.
Algorithm: 'RS256', SecurityKey: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'.'
 is not supported. The list of supported algorithms is available here: https://aka.ms/IdentityModel/supported-algorithms
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(Byte[] encodedBytes, Byte[] signature, SecurityKey key, String algorithm, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
'.
token: '{"typ":"JWT","alg":"RS256","kid":"elided-for-security"}.jwt body elided'.
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: Successfully validated the token.
Microsoft.AspNetCore.Authorization.DefaultAuthorizationService: Information: Authorization was successful.
Microsoft.AspNetCore.Routing.EndpointMiddleware: Information: Executing endpoint 'Portal.Controllers.TokenController.GetTokenAsync (Portal)'
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker: Information: Route matched with {action = "GetToken", controller = "Token", page = ""}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] GetTokenAsync() on controller Portal.Controllers.TokenController (Portal).
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: CustomJWTBearerToken was not authenticated. Failure message: IDX10511: Signature validation failed. Keys tried: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'. , KeyId: elided-for-security
'. 
kid: 'elided-for-security'. 
Exceptions caught:
 'System.NotSupportedException: IDX10634: Unable to create the SignatureProvider.
Algorithm: 'RS256', SecurityKey: 'Microsoft.IdentityModel.Tokens.SymmetricSecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'.'
 is not supported. The list of supported algorithms is available here: https://aka.ms/IdentityModel/supported-algorithms
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(Byte[] encodedBytes, Byte[] signature, SecurityKey key, String algorithm, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
'.
token: '{"typ":"JWT","alg":"RS256","kid":"elided-for-security"}.jwt body elided'.
Microsoft.AspNetCore.Authorization.DefaultAuthorizationService: Information: Authorization was successful.

Scenario 2 - using AzureADDefaultsJwtBearerAuthenticationScheme as default auth scheme end executing endpoint using "CustomJWTBearerToken".

Stack trace from Debug output:

Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: Failed to validate the token.

Microsoft.IdentityModel.Tokens.SecurityTokenInvalidSignatureException: IDX10511: Signature validation failed. Keys tried: 'Microsoft.IdentityModel.Tokens.X509SecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'. , KeyId: elided-for-security
'. 
kid: 'elided-for-security'. 
Exceptions caught:
 'System.NotSupportedException: IDX10634: Unable to create the SignatureProvider.
Algorithm: 'HS256', SecurityKey: 'Microsoft.IdentityModel.Tokens.X509SecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'.'
 is not supported. The list of supported algorithms is available here: https://aka.ms/IdentityModel/supported-algorithms
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(Byte[] encodedBytes, Byte[] signature, SecurityKey key, String algorithm, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
'.
token: '{"alg":"HS256","kid":"elided-for-security","typ":"JWT"}.jwt body elided'.
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, TokenValidationParameters validationParameters, SecurityToken& validatedToken)
   at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: AzureADJwtBearer was not authenticated. Failure message: IDX10511: Signature validation failed. Keys tried: 'Microsoft.IdentityModel.Tokens.X509SecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'. , KeyId: elided-for-security
'. 
kid: 'elided-for-security'. 
Exceptions caught:
 'System.NotSupportedException: IDX10634: Unable to create the SignatureProvider.
Algorithm: 'HS256', SecurityKey: 'Microsoft.IdentityModel.Tokens.X509SecurityKey, KeyId: 'elided-for-security', InternalId: 'elided-for-security'.'
 is not supported. The list of supported algorithms is available here: https://aka.ms/IdentityModel/supported-algorithms
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm, Boolean cacheProvider)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateForVerifying(SecurityKey key, String algorithm)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(Byte[] encodedBytes, Byte[] signature, SecurityKey key, String algorithm, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateSignature(String token, TokenValidationParameters validationParameters)
'.
token: '{"alg":"HS256","kid":"elided-for-security","typ":"JWT"}.jwt body elided'.
jwt message received
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Information: Successfully validated the token.
jwt token validated
Microsoft.AspNetCore.Authorization.DefaultAuthorizationService: Information: Authorization was successful.
Microsoft.AspNetCore.Routing.EndpointMiddleware: Information: Executing endpoint 'Portal.Controllers.CacheController.GetStringAsync (Portal)'
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker: Information: Route matched with {action = "GetString", controller = "Cache", page = ""}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] GetStringAsync(System.String) on controller Portal.Controllers.CacheController (Portal).
Microsoft.AspNetCore.Authorization.DefaultAuthorizationService: Information: Authorization was successful.

[brentschmaltz]

@udlose in both cases the error is indicating a mismatch between the algorithm specified in the JWS { "alg" } and the SecurityKey that the runtime choose.

In scenario 1, the key is a SymmetricKey, alg == RS256, an asymmetric algorithm that requires an RSA key.
In scenario 2, the key is a X509SecurityKey, alg == HS256, a symmetric algorithm that requires a Symmetric key.

It is almost as if the configurations are switched.

[udlose]

@brentschmaltz - Do you see anything in my code that would cause this? It's either the case that I misconfigured something (though I'm not able to see where that might've been done) or it is a minor bug in the nuget that happens when non-default auth schemes are used.

[brentschmaltz]

@udlose let's see if @jmprieur can help.
@jmprieur I am wondering if you could quickly look at this.
It appears the config used seems to be inverted.
call graphs using azuread are using the symmetric key, custom call graphs are using the asymmetric key.

[udlose]

this same issue happens even if I separate the default authorization policy like so:

            // Separate the default authorization policy from the other authentication schemes
            // otherwise, any of the schemes defined under default can be used to authenticate
            // against [Authorize] attributes with no policy defined.
            // Note that the  default authorization policy/scheme can be used as a "backup"
            // What we DO NOT want to do is what is defined here (for the reasons above):
            // https://docs.microsoft.com/en-us/aspnet/core/security/authorization/limitingidentitybyscheme?view=aspnetcore-3.1#use-multiple-authentication-schemes
            services.AddAuthorization(options =>
            {
                options.DefaultPolicy = new AuthorizationPolicyBuilder(CustomJWTBearerTokenDefaults.AuthenticationScheme)
                    .RequireAuthenticatedUser()
                    .Build();

                options.AddPolicy(AuthorizationPolicies.AzureAd, policy =>
                {
                    policy.AuthenticationSchemes.Add(AzureADDefaultsJwtBearerAuthenticationScheme);
                    policy.RequireAuthenticatedUser();
                });

                // do not define a Fallback policy
            });

[udlose]

Here is the controller endpoint that is only supposed to use the AzureAD scheme/policy and NOT the custom jwt default policy:

        [ProducesResponseType(typeof(TokenResponse), StatusCodes.Status200OK)]
        [Authorize(Policy = AuthorizationPolicies.AzureAd)]
        [HttpPost("")]
        public async Task<IActionResult> GetTokenAsync()
        {
            // elided for brevity
        }

The endpoints that are supposed to use the Default Custom JWT scheme/policy ONLY are just decorated with [Authorize]. It seems like all auth schemes are challenged regardless of what is specified on the Policy.

[brentschmaltz]

@udlose ill have to debug through this ... give me a bit of time.

[udlose]

@brentschmaltz Hi Brent, have you had any time to look at this yet?