Loki: s3_secret_access_key does not allow slashes

Created on 29 Jan 2020  路  6Comments  路  Source: grafana/loki

Describe the bug
Loki fails to start when an s3 bucket is configured as storage and the secret key contains a slash. I have generated a different access key pair whose secret key doesn't contain a slash character and it works fine.

To Reproduce
Steps to reproduce the behavior:

  1. Configure s3 storage with a secret key that contains a slash. E.g.
storage_config:
  boltdb:
    directory: /tmp/loki/index
  filesystem:
    directory: /tmp/loki/chunks
  aws:
    s3: s3://ASDFGHJIQWETTYUI:Jkasduahdkjh213kj1h31+lkjaflkjzvKASDOasofhjafaKFAF/GoQd@region/bucket_name
  1. Start Loki (master-765b34d)

Expected behavior
Loki should correctly parse the secret key and don't interpret it as a port

Environment:

  • Infrastructure: Docker running in a VM
  • Deployment tool: Docker-compose

Screenshots, Promtail config, or terminal output

failed parsing config: /etc/loki/local-config.yaml: parse s3://ASDFGHJIQWETTYUI:Jkasduahdkjh213kj1h31+lkjaflkjzvKASDOasofhjafaKFAF/GoQd@region/bucket_name invalid port ":Jkasduahdkjh213kj1h31+lkjaflkjzvKASDOasofhjafaKFAF" after 
host
keepalive

Most helpful comment

I can confirm that URL encoding the character works. In my opinion there should be better error handling or at least the documentation should warn you. I can provide a PR for documentation if you think this is useful

All 6 comments

@PabloCastellano you should encode the URL. Try Like below
s3://ASDFGHJIQWETTYUI:Jkasduahdkjh213kj1h31+lkjaflkjzvKASDOasofhjafaKFAF%2FGoQd@region/bucket_name

/ is replaced with %2F

FYR: Check the comment here https://github.com/weaveworks/common/blob/master/aws/config.go#L16

Nice digging @adityacs. I do think we should have nicer error handling for this (or transparently handle this case)

I can confirm that URL encoding the character works. In my opinion there should be better error handling or at least the documentation should warn you. I can provide a PR for documentation if you think this is useful

Definitely! Feel free to stop by #loki-dev on grafana.slack.com if you have any questions.

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@PabloCastellano are you still available to add some documentation around this, that would be very helpful!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

Horkyze picture Horkyze  路  5Comments

shkmaaz11 picture shkmaaz11  路  5Comments

gouthamve picture gouthamve  路  4Comments

arnitolog picture arnitolog  路  6Comments

cyriltovena picture cyriltovena  路  4Comments