Tailing is a very important part of work with centralized logs. Loki looks really great but missing of tailing via UI is a bit sad. Loki Log CLI might be an option but it doesn't work for us as we have google auth and having separate credentials for tailing access sounds insecure.
Describe the solution you'd like
The first option is to add proper tailing on UI while another option is to add okta, google auth, etc in Loki Log CLI to use it in the same way as main Grafana UI.
Alternative solution
Make tailing to directly via Grafana but via promtail which is not exposed with auth but instead can be protected by networking (VPN etc).
This problem is not solved as easily as it might seem, because there are multiple places of authentication involved in here:
The Grafana web interface is authenticated by default, using the built-in login flow or using oauth2 (Google, Okta, GitHub, etc). This is only relevant for the browser side (frontend).
loki is the actual server, which processes and stores the log streams it is being sent by log shippers. By default, loki does not have authentication. Users are expected to run an auth layer on top of loki (nginx, oauth2_proxy, etc.).
When this happens, all loki clients (Grafana, promtail, logcli) will need to send the required auth information to operate properly:
Grafana is easy, just set up the required auth info on the datasource page:
forward oauth, to forward the oauth token the user used to log into Grafana to the loki server. Loki's auth proxy needs to be set up properly to honor this.As promtail's client is based on prometheus, this one is easy as well. Basic Auth, mTLS and oauth2 (bearer token) are fully supported (https://github.com/grafana/loki/blob/master/docs/promtail-setup.md#client_option)
As you've pointed out correctly, logcli has only partly support for the authentication methods:
--username and --passwordCurrently, there is no support for oauth2! Neither a Bearer Token can be supplied, nor the full oauth flow is implemented. See Proposal below.
(https://github.com/grafana/loki/blob/master/docs/logcli.md)
To get up and running with logcli SSO, oauth2 support is required.
Authentication: Bearer xxxx header to be set with an already obtained bearer token.logcli auth command, implementing the whole oauth2 flow using a temporary built-in server for the redirect-url: Basically, the user's browser is opened, after the auth is done, logcli is notified using the redirect and extracts the token. Job done, much like gcloud CLI does. See https://github.com/nmrshll/oauth2-noserver for reference.Thoughts on this @tomwilkie @gouthamve?
This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.
Most helpful comment
Current Situation
This problem is not solved as easily as it might seem, because there are multiple places of authentication involved in here:
Grafana
The Grafana web interface is authenticated by default, using the built-in login flow or using
oauth2(Google, Okta, GitHub, etc). This is only relevant for the browser side (frontend).Loki
lokiis the actual server, which processes and stores the log streams it is being sent by log shippers. By default, loki does not have authentication. Users are expected to run an auth layer on top of loki (nginx,oauth2_proxy, etc.).When this happens, all
lokiclients (Grafana,promtail,logcli) will need to send the required auth information to operate properly:Grafana
Grafana is easy, just set up the required auth info on the datasource page:
forward oauth, to forward the oauth token the user used to log into Grafana to the loki server. Loki's auth proxy needs to be set up properly to honor this.promtail
As promtail's client is based on prometheus, this one is easy as well. Basic Auth, mTLS and oauth2 (bearer token) are fully supported (https://github.com/grafana/loki/blob/master/docs/promtail-setup.md#client_option)
logcli
As you've pointed out correctly,
logclihas only partly support for the authentication methods:--usernameand--passwordCurrently, there is no support for
oauth2! Neither aBearer Tokencan be supplied, nor the full oauth flow is implemented. See Proposal below.(https://github.com/grafana/loki/blob/master/docs/logcli.md)
Proposal
To get up and running with
logcliSSO, oauth2 support is required.Authentication: Bearer xxxxheader to be set with an already obtained bearer token.logcli authcommand, implementing the wholeoauth2flow using a temporary built-in server for theredirect-url: Basically, the user's browser is opened, after the auth is done, logcli is notified using the redirect and extracts the token. Job done, much likegcloudCLI does. See https://github.com/nmrshll/oauth2-noserver for reference.Thoughts on this @tomwilkie @gouthamve?