Authelia: Support using environment variables in configuration

[J7mbo]

Hi. #569 led me to creating a new task to discuss this.

Authelia's use of a configuration file currently requires all configuration to be hardcoded. Examples can be the domain under access_control, the user under authentication_backend.ldap etc.

Are there any reservations for a PR using the template package or similar to utilise environment variables in authelia configuration? Preferably without any unnecessary prefixes or suffixes like terraform does (these require users to use extra scripts just to use vendor-specific environment variables).

ldap:
        url: {{ LDAP_URL }}

No complexity with defaults - if the environment variable exists then place it otherwise it's an empty string.

[james-d-elliott]

We're actively discussing this. There are a few factors which affect this.. we're looking towards a management UI which would introduce in database configuration. Additionally we are about to deprecate our current use of env variables to replace values in config. I think though based on your description it sounds like you want to add it so you can replace the contents of the YAML prior to reading it into a struct?

I think we'd consider this (I can't speak for @clems4ever or @nightah so lets wait for them to comment) provided it had a CLI argument associated with it.

I think a good design is to have an argument like --templates="/var/folder/file1.yaml,/var/folder/file2.yaml" - an array of files to replace using the template package prior to reading configuration. We could theoretically utilize this with a database preload in the future with little to no adjustment.

Alternatively maybe --templates=/var/folder --templates-out=/var/folder2. Where --templates is a directory of *.tpl files, and --templates-out is the same file names with tpl removed (defaulting to either the /etc/authelia/ dir, or the same folder as the templates?

[J7mbo]

If there were CLI arguments as an options, I would just use --authentication_backend.ldap.url=$LDAP_URL in docker-compose command:. Were this to exist for every configurable option, this would negate the need for a YAML file being copied over and mounted to be used in the first place. But I think that's a different kettle of fish...

When parsing the YAML file to convert it into a configuration struct, for each option, if the option is in the format {{ envVarName }}, then instead of using the string use os.GetEnv() instead.

Looking at: reader.go, after this, my implementation for a PR would be: loop through all properties of the config struct (they're all public), if any {{ someString }} are found with, say, a regex, pull out the someString, do an os.GetEnv() lookup with a default value of an empty string (and in the default case, a warning log), and replace the value in the config struct with the value from the environment.

For docker-compose/docker this would work fine with the environment: key or -e in docker run which passes the env vars to the containers.

[clems4ever]

Hello @J7mbo , there is no reservation on community PRs, they are actually very welcome and you did the right thing by coming and discuss the design first. I'd be interested to know the use case first though. That's important to decide whether we would accept that change because as mentioned by @james-d-elliott , this change can conflict with the migration of the configuration values in database if we decide to do so eventually. It's still under discussion because I'm wondering if doing so wouldn't make setup automation harder.

Also please note that with viper, setting those config variables with env vars would be trivial if https://github.com/spf13/viper/issues/761 was resolved. That would be the proper solution in order to delegate the config-specific complexity to the library instead of Authelia. In the meantime and for the reasons mentioned above, I'd like to hear more about the actual use case.

Can you elaborate on your situation right now?

[J7mbo]

@clems4ever Sure, happy to discuss the use-case.

As a user, I don't want to hardcode my domain, password or other configuration options such as the port, sensitive or otherwise, in configuration.yml. I use $DOMAIN when spinning up many services and currently have to hardcode the domain in authentication_backend.ldap.base_dn, authentication_backend.ldap.user, authentication_backend.ldap.password, totp.issuer etc.

All my environment variables are stored in a .env file which I read out and pass during deployment. Authelia can't handle this right now so my alternative is to place the placeholders in and script an on-the-fly replacement.

Perhaps irrelevant, but other tools that work well alongside authelia such as traefik allow the passing in of configuration options on the command line, which by proxy enables environment variables to be used and it's a popular feature for 100% setup automation which is my goal. OpenLDAP has this feature. Elasticsearch and Kibana too. Seems to be quite common to allow for dynamic setups.

Configuration values in a database (with the UI?) sounds great. I haven't looked at the PR and discussion for it, how having this addition would conflict with it? If configuration values were specified in a database, they will be used, with environment variables taking precedence if present, or the other way around.