Kibana: [New platform] Security plugin integration

[elasticmachine]

Pinging @elastic/kibana-platform

[restrry]

Right now legacy platform defines 2 auth strategies:

• session. used by default https://github.com/elastic/kibana/blob/master/x-pack/plugins/security/index.js#L117
• security-cookie, which is never set as default. https://github.com/elastic/kibana/blob/master/x-pack/plugins/security/server/lib/authentication/session.js#L125
  but it's called manually to decode and validate cookie https://github.com/elastic/kibana/blob/master/x-pack/plugins/security/server/lib/authentication/session.js#L56

[restrry]

@elastic/kibana-security would you mind to check everything here makes sense 😄?

[kobelb]

@restrry thanks for writing this up! Given @azasypkin's experience with the new platform, and his knowledge of our various authentication providers, I'd like to hear his opinion on this as well.

Were you intending for this extension point to be utilized for both authentication and authorization, or would we have two different "lifecycle methods" that the new platform exposes?

[restrry]

Were you intending for this extension point to be utilized for both authentication and authorization, or would we have two different "lifecycle methods" that the new platform exposes?

Does the authorization mechanism in the current state require incoming request interception?

As I understand Security plugin provides tools for other plugins to check user privileges but doesn't provide user data & roles directly.
https://github.com/elastic/kibana/blob/master/x-pack/plugins/security/index.js#L124-L125
Those tools heavily relay on request headers, which we don't expose right now in NP route handler. We are going to solve this problem in https://github.com/elastic/kibana/issues/33783, for now we can provide raw request if needed

[kobelb]

Does the authorization mechanism in the current state require incoming request interception?

Spaces does currently in the master branch https://github.com/elastic/kibana/blob/master/x-pack/plugins/spaces/server/lib/space_request_interceptors.ts#L41

With the introduction of Feature Controls we're using the following code to intercept some API calls https://github.com/elastic/kibana/blob/granular-app-privileges/x-pack/plugins/security/server/lib/authorization/api_authorization.ts#L14 and we're doing something similar to block certain applications from being loaded https://github.com/elastic/kibana/blob/granular-app-privileges/x-pack/plugins/security/index.js#L231

[azasypkin]

Or the platform can provide the ability to register a request middleware. Middleware specifies to what request headers it wants to have access to

Do we have any plans on defining framework-agnostic Kibana request lifecycle with lifecycle event based hooks similar to what Hapi has or there will be express-js-like mechanism?

With the introduction of Feature Controls we're using the following code to intercept some API calls https://github.com/elastic/kibana/blob/granular-app-privileges/x-pack/plugins/security/server/lib/authorization/api_authorization.ts#L14 and we're doing something similar to block certain applications from being loaded https://github.com/elastic/kibana/blob/granular-app-privileges/x-pack/plugins/security/index.js#L231

It _looks like_ we can group authc + all authz based interceptors internally and just register one security interceptor/pre-request-handler with the core, so that security does everything in one go before any other app route handler is invoked. It's kind of what we do already with multiple onPostAuth handlers.

Also it may depend on whether core will be treating API calls as a separate group of requests, IIRC there was a conversation about having something like core.http.regiterAPIRoute and core.http.registerSomethingElse and hence potentially two different extension points for interceptors (so that we can get rid of this a bit fragile !request.path.startsWith('/api/') check), but don't quote me on that.

[restrry]

Sorry for a long comment, I spent several days wrapping my head around current implementation and want to structure my thoughts.

Right now we rely on 3 different stages for every request (in execution order in hapi framework, more info about hapi request lifecycle)

• onRequest step, where plugins can control a request, say check headers or URL and a result to fail/ to redirect/ to continue request. I'm able to find only one usage in the plugin code base.
• auth (when enabled) where Security plugin authenticates a user, enhance a request with user information (scopes, roles) and restricts access to resources if authentication fails.
• onPostAuth, where Security plugin, given credentials from auth step, checks if user authorized to perform an operation. At the same time, there is a case when another plugin(s) (namely dashboard_mode) also uses user credentials to restrict access to dashboard content. Thus we have a situation when not only Security plugin operates with auth credentials. I talked to @azasypkin and we came to the conclusion that could be an old solution and we want to provide a mechanism similar to Feature Controls, where Security Plugin is the only source of authorization level. Probably @elastic/kibana-security can provide more information here.
  And yeah, there are a couple of cases that aren't directly related to authorization and we might want to move them from onPostAuth to onRequest step. @elastic/kibana-platform WDYT?
  https://github.com/elastic/kibana/blob/master/src/legacy/server/http/version_check.js
  https://github.com/elastic/kibana/blob/4aa2a4c05f128f09e113d29f8a31557c1875717f/src/legacy/server/http/xsrf.js
  Here I did and integration tests passed https://github.com/elastic/kibana/pull/34281

As a bottom line there are 2 main mechanics:

• intercept a request and implement the context-dependent operation, interceptors do not share any information between them and do not have access to user credentials., that makes them closer to event handlers than middlewares.
  Possible implementation:

interface InterceptorToolkit {
   next: () => ({ type: 'next' }),
   redirected: (url: string) => ({ type: 'redirected', url }),
   rejected: (error?: Error) => ({ type: 'rejected', error }),
}
core.http.registerResourceInterceptor((req: KibanaRequest, interceptorToolkit: InterceptorToolkit){
   if(req.method === 'GET') return interceptorToolkit.redirect('/');
   return interceptorToolkit.next();
});

• intercept a request and perform authc/authz operation. Those 2 are dependent on order and have different meaning and interfaces. The very first step performs authentication and provides a result to the next authorization steps, which have only read access to given user credentials. Potentially we can merge them into one step, but we need to decide where 3rd party plugins (dashboard_mode e.g.) register access to authz - via HTTP server or via security plugin (I'm inclined to this option). But want to hear from @elastic/kibana-security
  An option where we merge authc/authz steps.

// 3rd party plugin, for example dashboard_mode
plugins.security.addAuthorizationInterceptor(async function(req: KibanaRequest, interceptorToolkit: InterceptorToolkit, readonly user: UserStore, scopedTools){
   if(user.roles.includes(...) || scopedTools.getSpaces(...)){
      return interceptorToolkit.redirected(...)
   }
});
// plugins/security.js
// in following steps we can restrict an access only to required auth headers instead of whole hapi request
core.http.registerAuthorizationInterceptor((req: HapiRequest, interceptorToolkit: InterceptorToolkit) => {
 const authenticationResult = await authenticate(req);
 if(authenticationResult.succeeded()){ 
   for (const interceptor of this.getAuthorizationInterceptors()){
     const authorizationResult = await interceptor(KibanaRequest.from(req), interceptorToolkit, authenticationResult.user, scopeToolsToRequest(req));
     if(authorizationResult.redirected()) {
        return interceptorToolkit.redirected(authorizationResult.redirectURL)
     }
});

Or the implementation option where we separate authc/authz steps.

interface AuthToolkit {
   authenticated: () => ({ type: 'authenticated' }),
   unauthenticated: (error?) => ({ type: 'unauthenticated', error }),
   reject: (error: Error) => ({ type: 'reject', error }),
}
// in following steps we can restrict an access only to required auth headers instead of whole hapi request
core.http.registerAuthenticationInterceptor((req: HapiRequest, authToolkit: AuthToolkit, user: UserStore){
   const authenticationResult = authenticate(req);
   if(authenticationResult.succeeded()){
      user.set(authenticationResult);
   }
   return authToolkit.authenticated();
});

core.http.registerAuthorizationInterceptor((req: HapiRequest, interceptorToolkit: InterceptorToolkit, readonly user: UserStore){
  // user is scoped to the request. on the very first step we can support old approach with plugin.secuity.checkPriveledgesFor(request, user);
   const result  = plugin.secuity.checkPriveledgesFor(user);
   if(result) {
      return interceptorToolkit.next();
   } else {
      return interceptorToolkit.reject(new Error('not found'));
   }
});

And note if we migrate Security plugin to NP we should change dependent parts (like dashboard_mode) in legacy platform to restrict an access to hapi request object. Because request objects in New and legacy platform are different objects with different lifecycles.

[kobelb]

Thanks for giving this so much thought, and a clear description of the current situation @restrry.

I think it makes sense for the security plugin itself to handle all authentication and authorization, and provide extension points to the various plugins to "add" their own authorization. This allows the security plugin to determine when authentication/authorization should be performed, without requiring other plugin authors to understand the details of when security is enabled and auth should be performed.

[epixa]

I'm comfortable with the merged approach through the security plugin as well. It's worth noting that one way or another the underlying ability to build this auth stuff through the http service is there and available to plugins, so the security plugin just makes it more explicit.

[restrry]

I'm back to the problem. After talking with the Security team we figured out that several lifecycle stages are required.

• a stage that is performed before auth. Specificity: can perform request forwarding to another URL, before auth is performed and request hits a route handler. Analog in the LP - server.ext('onRequest',...). There is only one consumer as of now. But some extension points in LP can be placed here. one, two
• auth stage. Specificity: can perform authc/authz logic and store user credentials in memory.
• a stage that is performed after auth. Specificity: has access to auth status. Analog in the LP - server.ext('onPostAuth',...).

[restrry]

I'm going to work on restriction access to raw hapi request object in registerAuth((req: hapi.Request, ... and want Security team to clarify some moments

Authorization header

Kibana relays on Elasticsearch to do all heavy lifting for user authentication. Security plugin tries to apply different providers and authenticate a user by means of extending request headers with a custom authorization header.

// some provider specific logic
request.headers.authorization = providerSpecificHeader;
try {
  esClient.callWithRequest(request, 'shield.authenticate');
  ...
} catch(e) {
  delete request.headers.authorization;
}

Headers are mutated in order to retain authorization header for downstream plugins interacting with Elasticsearch. It means Elasticsearch server expects this header to set on KibanaRequest, Legacy.Request or whatever.
https://github.com/restrry/kibana/blob/f753474423c973aba2b9c35a68a22b031991e6e6/src/core/server/elasticsearch/cluster_client.ts#L155

public asScoped(req: { headers?: Headers } = {}) {...}

Here I have a question: how critical if we expose authorization header as a part of KibanaRequest abstraction? Should be able any plugin (including 3rd party ones) to have read/write access to this header? Depending on an answer we either use KibanaRequest in registerAuth or introduce our own abstraction (KibanaRequestSecurity?) which is only one that has access to read/write this authz header. Introducing additional abstraction also will require ClusterClient refactoring, because asScoped is a separate plugin and won't have direct access to request headers anymore.

Request body & query validation

SAML provider expects a request to contain some provider specific information.
https://github.com/restrry/kibana/blob/f753474423c973aba2b9c35a68a22b031991e6e6/x-pack/plugins/security/server/lib/authentication/providers/saml.ts#L80-L92
The new platform does try to enforce input validation on system boundaries and requires schema declaration for request body, query, params. Can Security plugin use router API to enforce request validation? https://github.com/restrry/kibana/blob/f753474423c973aba2b9c35a68a22b031991e6e6/src/core/server/http/router/route.ts#L61
Should we enforce request validation for Security plugin at all? It shouldn't be a problem as we already know the schema
https://github.com/restrry/kibana/blob/f753474423c973aba2b9c35a68a22b031991e6e6/x-pack/plugins/security/server/lib/authentication/providers/saml.ts#L46-L56

[kobelb]

Here I have a question: how critical if we expose authorization header as a part of KibanaRequest abstraction? Should be able any plugin (including 3rd party ones) to have read/write access to this header?

I would absolutely prefer that they don't. The way that we do this today has always bothered me, and as far as I'm aware it's only being done this way because security was introduced after a significant portion of Kibana was already written. I'm open to reconsidering this entire approach, if you are :) Would it be possible for the registerAuth implementation itself to respond with the headers that can be used to "authenticate against Elasticsearch as the end-user" and then the new platform could store this somewhere, perhaps in a weakmap with the request and the authentication headers, and then the Elasticsearch plugin could use this weakmap? I'd love to be able to rename callWithRequest to something like callImpersonatingAuthenticatedEndUser.

The new platform does try to enforce input validation on system boundaries and requires schema declaration for request body, query, params. Can Security plugin use router API to enforce request validation?

I believe we should be able to as we have this route where I believe we could perform this.

@azasypkin you mind double-checking everything I've said here for correctness and to ensure you agree with my recommendations?

[restrry]

I believe we should be able to as we have this route where I believe we could perform this.

In this case we might need to change api slightly:

await server.plugins.security.authenticate(request, {
   type: 'saml',
   SAMLRequest: query.SAMLRequest,
   SAMLResponse: body.SAMLResponse,
});

it could be useful, because we can use the same pattern for LoginAttempt

const { username, password } = request.body;
await server.plugins.security.authenticate(request, {
  type: 'basic',
  credentials: { username, password }
});

[kobelb]

@restrry sounds reasonable to me!

[azasypkin]

I believe we should be able to as we have this route where I believe we could perform this.

++, the validation schema should allow us to expect "unknown" keys as well (see OIDC route with Joi.object.unknown), don't remember if kbn/config-schema supports this already or not.

@azasypkin you mind double-checking everything I've said here for correctness and to ensure you agree with my recommendations?

Yes, I agree, thanks!

In this case we might need to change api slightly:

Exactly! That's what we're discussing for some time already: second argument for authenticate should be an optional free form LoginAttempt that will consolidate all these query string/body parameters and be created at the route handler level (e.g. api/security/v1/saml).

[restrry]

Authentication mechanism was migrated to the New Platform