Sensu-go: Agent authentication

[palourde]

All agents currently authenticate to the backend using the same credentials, and they are granted virtually all permissions, which is not desired.

We need to determine the desired behaviour around agent authentication. Some discussion tracks:

• Should agents across the same environments share the same authentication? (e.g. SSL certificates https://github.com/sensu/sensu-go/issues/203)
• How should agents authenticate against the backend?
• What about an unverified workflow for agents authentication (https://github.com/sensu/sensu-go/issues/201)
• Agents permissions (e.g. https://github.com/sensu/sensu-go/issues/1003)

[jaredledvina]

I wanted to drop some notes from my experience deploying Sensu 1.x regarding agent authentication. Most of these requirements stem from how I've ended up locking down and restricted the sensu-client to RabbitMQ in the 1.x world and they might be helpful to have in mind when looking at re-implementing client auth for Sensu Go.

• Each agent must be configured w/ a unique TLS client certificate signed by the internal CA.
  
  
  
  • In my case, I provide the CA cert chain, and client key/cert pair. I think it's completely fair to ask administrators to provide this and not have Sensu ship with a built in CA setup.
  
  
• The client certificates themselves should be restricted to only being used for the Sensu client process.
• Clients must authenticate via their provided client certificate. The backend service (previous this was just RabbitMQ in my case), must expose when clients fail to authenticate via this method.
  
  
  
  • This allows for administrators to keep track of clients who are currently unable to be monitored by Sensu due to their authentication failing.
  
  
• Agents must be restricted to the absolute minimum required permissions on the backend.
  
  
  
  • For RabbitMQ this was https://docs.sensu.io/sensu-core/1.6/guides/securing-rabbitmq/#client-perms-strict but, my goal was simply to ensure that as a client, I couldn't influence the check scheduling/results of another client without gaining more privileged credentials.
  
  
  • With 1.x via https://github.com/rabbitmq/rabbitmq-auth-mechanism-ssl, this ends up having RabbitMQ users that are the same as either the "Common Name" or the "Distinguished Name" from the certificate. As an administrator, this setup was very useful for hunting down which client was doing something terrible, and if needed, revoking that certificate to ensure the client was blocked from connecting.
  
  
• The cipher set used for agent <--> backend communication should be configurable and ideally support TLS 1.2 w/ a cipher leveraging perfect forward secrecy.
  
  
  
  • Notably, Sensu 1.x is pretty limited here (https://github.com/sensu/sensu-transport/issues/58).
  
  
• The backend service must verify the OCSP and CRL status for all provide agent certificates.
• The agent should verify when connecting to the backend that the certificate used for that endpoint is trusted, valid, etc.
  
  
  
  • Currently, 1.x does not do this (https://github.com/sensu/sensu/issues/1310) and it's very painful to discuss this limitation w/ compliance/infosec teams.
  
  

[gbolo]

Hi @jaredledvina, excellent points you made here. I will add some thoughts (about server auth too)

• The agent can further validate the backend endpoint with an SKI (cert pinning). In this case, we can use leverage x509 certificates signed by a standard authority (instead of an internal one) and simply provide the agent the expected cert pin.

• I like the idea of OCSP and CRL status for agent certificates. Maybe we can look at the possibility of utilising a certificate transparency log as an alternative? Also we can use SKI of agent certificates to further authenticate an agent. The benefit of using SKI here is that we can easily revoke a cert without the need for any other infrastructure mentioned above.

• In terms of ciphers, since we write both the backend and agent, there is no need to make them configurable. TLS 1.2 with only PFS ciphers #2700

[nikkictl]

The mTLS agent authentication draft can be found here and the proposal can be found here. Discussion scheduled for Friday, Aug 9th.

[portertech]

mTLS Agent authentication

[portertech]

We are proceeding with the proposed mTLS approach linked to above. This will be an Enterprise feature.