Authelia: Authelia is failing silently with insufficient resources

Created on 19 Oct 2020  Â·  5Comments  Â·  Source: authelia/authelia

After a few hours of debugging I've managed to discover that Authelia is failing silently when it's assigned insufficient resources in Docker Swarm.
The story is that I had assigned 1 CPU core and 1G memory to it, and in the consequence all login requests were timing out with no logs outputted, not even with the debug logging level.
This seems like a bug to me

Most helpful comment

There's probably three logical things we can do in an effort to communicate this, all on startup, all involving checking available memory in /sys/fs/cgroup/memory/memory.limit_in_byte if using argon2id and running in a container.

  1. Check memory and if insufficient log a warning.
  2. Check memory and if insufficient log a warning and alter the memory property in argon2id (probably bad for security).
  3. Check memory and if insufficient log a critical and exit.

All 5 comments

I had this issue too. Basically as far as I can tell 1GB isn't enough due to how how the go cypto library works. https://github.com/authelia/authelia/issues/1044#issuecomment-672864011 It seems you need a minimum of 2GB. I think maybe there should be more warnings given around this? It's a frustrating issue to debug.

I don't have a lot of experience with Swarm but I expect what is likely occurring is that the Swarm manager is going to be restarting the container when it hits the memory limit as opposed to Authelia itself OOMing, this is likely what is resulting in "failing silently".

The documentation regarding the password hashing explains the parameters in some detail. Realistically you should be tuning said parameters to fit your deployment and available resources.

Just to rehash the docs:

MEMORY
Value Type: Int
Possible Value: at least 8 times the value of parallelism
Recommended: 1024‬‬ (1GB) or as much RAM as you can afford to give to hashing
What it Does: Sets the amount of RAM used in MB for hashing

There's not really much we can do if your Orchestration tool is killing Authelia before it has the opportunity to OOM and dump you a stack trace.

There's not really much we can do if your Orchestration tool is killing Authelia before it has the opportunity to OOM

Would it be possible to log a warning message if Authelia is started with one gig or less of RAM? Having a memory recommendation in the docs is good but usually application logs are the first place someone would look when experiencing problems like application crashes. And this seems to be an issue which pops up on GitHub not terribly frequently but relatively regularly.

Presumably scanning /sys/fs/cgroup/memory/memory.limit_in_byte could reveal the limit from inside the Authelia container and if it's lower than 1G I'd say the container could even gracefully crash then, logging something out

There's probably three logical things we can do in an effort to communicate this, all on startup, all involving checking available memory in /sys/fs/cgroup/memory/memory.limit_in_byte if using argon2id and running in a container.

  1. Check memory and if insufficient log a warning.
  2. Check memory and if insufficient log a warning and alter the memory property in argon2id (probably bad for security).
  3. Check memory and if insufficient log a critical and exit.
Was this page helpful?
0 / 5 - 0 ratings

Related issues

J7mbo picture J7mbo  Â·  4Comments

RafalMaleska picture RafalMaleska  Â·  7Comments

modem7 picture modem7  Â·  3Comments

greenpau picture greenpau  Â·  4Comments

zjzeit picture zjzeit  Â·  3Comments