Victoriametrics: Add option to load configuration file

[KawaiDesu]

Hello!

I would like to run VictoriaMetrics like a service via systemd (I'm packaging victoriametrics in the fact), but i don't want to change systemd service file every time i want to change some options. I could use unit override but it doesn't look very convenient. So it would be nice to add an option which loads targeted configuration file in simple format, like ini or maybe yaml.

[oOHenry]

Hi @KawaiDesu,
I packaged VictoriaMetrics too, you can solve the problem by using the systemd EnvironmentFile option:

cat /lib/systemd/system/victoria-metrics.service
[Unit]
Description=Prometheus :: victoria-metrics
Requires=basic.target network.target
After=basic.target network.target
Documentation=https://github.com/VictoriaMetrics/VictoriaMetrics
[Service]
User=nobody
Group=nogroup
EnvironmentFile=/etc/default/victoria-metrics-prod
ExecStart=/usr/bin/victoria-metrics-prod $ARGS
ExecStop=/bin/kill -s SIGTERM $MAINPID

[Install]
WantedBy=default.target

cat /etc/default/victoria-metrics-prod
ARGS=-retentionPeriod=24 -storageDataPath /var/lib/victoria-metrics/data

Just edit the options in /etc/default/victoria-metrics-prod and restart the service.

[valyala]

I would like to run VictoriaMetrics like a service via systemd (I'm packaging victoriametrics in the fact)

There are package scripts for deb and rpm inside package folder. These scripts are called from Makefile build rules defined in app/victoria-metrics/Makefile. Probably, you could base your packaging scripts on the code inside package folder.

i don't want to change systemd service file every time i want to change some options. I could use unit override but it doesn't look very convenient. So it would be nice to add an option which loads targeted configuration file in simple format, like ini or maybe yaml.

Agreed. Will think about the implementation for this feature. Something like accepting -configFile arg with path to ini file where command-line flags may be set.

[valyala]

@oOHenry , thanks for the solution! I'm unfamiliar with systemd and packaging for various Linux distros, so your insights are very valuable. Could you inspect the pacakge folder contents and help maintaining it together with the original authors of this folder @patsevanton and @kmansoft ?

[valyala]

@KawaiDesu , just a reference on how to use EnvironmentFile option for loading command-line args - https://coreos.com/os/docs/latest/using-environment-variables-in-systemd-units.html#environmentfile-directive .

[kmansoft]

Why not implement a config file like /etc/victoria-metrics.conf?

Also it makes admin tasks a lot easier if you also support config file snippets in a directory like /etc/victoria-metrics.d which take priority over the .conf file.

This way the packages can include the .conf with a "standard" configuration - and users can override what they want / need using the .d directory. Upgrades then will overwrite the .conf because it won't be modified but the .d files if any will persist.

[patsevanton]

@valyala where store config file?
/etc/default/victoria-metrics-prod or /etc/default/victoria-metrics.conf or /etc/victoria-metrics.conf?

[valyala]

@valyala where store config file?

/etc/victoria-metrics.conf looks fine to me

[KawaiDesu]

Personally, I wouldn't recommend to use /etc/.
Archwiki states the following:

/etc | System-essential configuration files
/etc/{pkg} | Configuration files for {pkg}

https://wiki.archlinux.org/index.php/Arch_package_guidelines#Directories

FHS 3.0 statement about /etc/ is close to Archs one, but more general:

The /etc hierarchy contains configuration files. A "configuration file" is a local file used to control the operation of a program; it must be static and cannot be an executable binary. [2]
It is recommended that files be stored in subdirectories of /etc rather than directly in /etc.

https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s07.html

So, please, use /etc/victoriametrics/victoriametrics.conf or /etc/victoria-metrics/victoria-metrics.conf. Use one that matches package name with or without hypen. I use without because of repository name.

[KawaiDesu]

@oOHenry thanks for suggestion with EnvironmentFile. This is nice workaround while cli option is not yet implemented.
@valyala thanks for reference link.

[patsevanton]

@valyala /etc/victoria-metrics.conf or /etc/victoriametrics/victoriametrics.conf

[micoq]

@valyala A dedicated configuration file might be useful to hide the passwords displayed in the command line (for example with the option httpAuth.password).

[valyala]

FYI, VictoriaMetrics accepts config args via command-line flags starting from v1.34.0. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/311 and these docs for details.

[Allineer]

Извините, я по-русски опишу.

Многие пользователи, я уверен, уже сейчас используют файлы с параметрами запуска компонентов VM, прописанные в параметре EnvironmentFile соответствующих systemd *.service файлов. И я так делаю - пример одного из моих файлов параметров для vmagent будет ниже.

У нас уже сейчас есть конфиги -rule для vmalert, -promscrape.config и -remoteWrite.urlRelabelConfig для vmagent и это действительно большие, важные, сложные файлы конфигурации.

В то же время, у нас есть большое число параметров командной строки для каждого компонента.
Да, я помню ваш аргумент, о том, что все они не нужны большинству пользователей - им достаточно прописать retention для хранилища и список таргетов.

Но, как показывает практика (https://github.com/VictoriaMetrics/VictoriaMetrics/issues/576), параметры -remoteWrite.* тоже могут быть большим и сложным набором ключей, в котором можно запутаться. В то же время, я считаю костылём-полумерой делать отдельный конфиг только лишь для параметров -remoteWrite.*.

Я считаю, что имеет смысл и пришло время все параметры конфигурации всех компонентов vm* вынести в отдельный файл и запускать каждый компонент с единственным ключом --config=.
Это позволит структурировать параметры внутри файла, сделать их более понятными и наглядными, избежать множества ошибок.

И да, у одних пользователей содержимое конфига будет состоять из двух-трех параметров, а у кого-то будут простыни. Но конфиги будут понятными и логичными и для тех и для других.

У меня, практически домашнего пользователя, конфиг vmagent состоит из 15 параметров (пример ниже), конфиг vmalert - из 6 параметров, конфиги нод - из 9 параметров каждый.

OPTIONS="                                                 \
  -httpListenAddr=127.0.0.1:8429                          \
  -loggerOutput=stdout                                    \
  -memory.allowedPercent=10                               \
  -promscrape.config.strictParse=1                        \
  -promscrape.config=/.../targets.yaml                    \
  -promscrape.disableCompression=1                        \
  -remoteWrite.maxDiskUsagePerURL=10000000000             \
  -remoteWrite.showURL=1                                  \
  -remoteWrite.tmpDataPath=/.../vmagent                   \
  -remoteWrite.url=http://127.0.0.1:8428/api/v1/write     \
  -remoteWrite.url=http://127.0.0.1:8427/api/v1/write     \
  -remoteWrite.url=http://127.0.0.1:8426/api/v1/write     \
  -remoteWrite.urlRelabelConfig=/.../filters/system.yaml  \
  -remoteWrite.urlRelabelConfig=/.../filters/etc.yaml     \
  -remoteWrite.urlRelabelConfig=/.../filters/testing.yaml \
"