Victoriametrics: Documentation Addition for Creating a service

[MaxDiOrio]

First - VM is amazing so far. Thank you.

I'd love to see you add documentation on running VM as a service in systemd. Here's what I did to make it work on CentOS 7, but this should work on any systemd OS.

Create a directory to hold the PID file:

mkdir /run/victoriametrics

Create /etc/systemd/system/victoriametrics.service. Copy and paste the below into this new file, adjusting the ExecStart line as needed.

[Unit]
Description=VictoriaMetrics
After=network.target

[Service]
Type=simple
StartLimitBurst=5
StartLimitInterval=0
Restart=on-failure
RestartSec=1
PIDFile=/run/victoriametrics/victoriametrics.pid
ExecStart=/usr/local/bin/victoriametrics -storageDataPath /data -retentionPeriod 6
ExecStop=/bin/kill -s SIGTERM $MAINPID

[Install]
WantedBy=multi-user.target

Set the file limits for the service:
mkdir /etc/systemd/system/victoriametrics.service.d
In this folder create a file ulimit.conf with the following:

[Service]
LimitNOFILE=32000
LimitNPROC=32000

Enable the service to start automatically:

systemctl enable victoriametrics

Start the service:

systemctl start victoriametrics

[valyala]

@MaxDiOrio , thanks for these instructions! I prettified your message and linked it from the README.md. Feel free updating this info as needed.

[mihaibalaci]

Quick question: if we deploy it in aks/eks is it possible to store data in something like s3 or adls?

[valyala]

Quick question: if we deploy it in aks/eks is it possible to store data in something like s3 or adls?

Currently the data may be stored in network block storage such as Amazon EBS or Google compute disks. There are plans for adding blob storage support such as S3 or GCS in the future - see this issue for details.

[kmansoft]

If anyone is interested, I can add build system scripts to create .rpm packages (did this recently for my project, went from "know nothing about RPM" to "know enough to be dangerous").

This would cover CentOS / Fedora, at least those versions that use systemd.

[valyala]

If anyone is interested, I can add build system scripts to create .rpm packages (did this recently for my project, went from "know nothing about RPM" to "know enough to be dangerous").

This sounds good. Please, coordinate your work with @patsevanton regarding his yum build for VictoriaMetrics.

[kmansoft]

@patsevanton I'm thinking of adding new code to the makefiles / build scripts.

It would dynamically generate the .spec file with the right paths (which do vary by version and by actual built path) and then use rpmbuild -bb --target to package.

Here is some code from my own build scripts:

            mkdir -p "${TEMPDIR_RPM}/${WHAT}/SPECS"
            cat > "${TEMPDIR_RPM}/${WHAT}/SPECS/clearview-${WHAT}.spec" <<EOF
Summary: Clearview agent
Name: clearview-agent
Version: ${VERSION}
Release: ${BUILD}
License: n/a
URL: https://clearview.rocks
Group: System
Packager: Kostya Vasilyev <[email protected]>
Requires: libpthread
Requires: libc

%description
Clearview is an easy to install, easy to use performance monitoring for
your Linux web servers. Inspired by Linode Longview.

%files
%attr(0744, root, root) /usr/sbin/*
%attr(0644, root, root) /lib/systemd/system/*

%prep
echo "BUILDROOT = \$RPM_BUILD_ROOT"

mkdir -p \$RPM_BUILD_ROOT/usr/sbin/
mkdir -p \$RPM_BUILD_ROOT/lib/systemd/system/

cp ${PWD}/package/temp-deb/agent/usr/sbin/${OUT_EXE} \$RPM_BUILD_ROOT/usr/sbin/
cp ${PWD}/package/agent/clearview-agent.service \$RPM_BUILD_ROOT/lib/systemd/system/
EOF

            cat "${TEMPDIR_RPM}/${WHAT}/SPECS/clearview-${WHAT}.spec"

            rpmbuild -bb --target "${RPM_ARCH}" \
                "${TEMPDIR_RPM}/${WHAT}/SPECS/clearview-${WHAT}.spec"

            cp "${HOME}/rpmbuild/RPMS/${RPM_ARCH}/${OUT_RPM}" "${PACKDIR}/${OUT_RPM}"

[patsevanton]

@kmansoft Can you create PR to https://github.com/patsevanton/victoriametrics-rpm
Thanks!

[kmansoft]

@patsevanton Hmmmm... I was thinking adding it to the existing build script(s) which build .deb packages, and therefore need to fork it off the complete Victoria repository.

[kmansoft]

Also @patsevanton can we drop support for non-systemd systems?

[patsevanton]

@kmansoft i think first merge https://github.com/VictoriaMetrics/VictoriaMetrics/pull/121

I dont know about drop support for non-systemd systems. Question to @valyala

[kmansoft]

RHEL 7 (has systemd) came out in 2014

For Debian, it's systemd only so that's "wheezy" and newer (I _think_), 2016 release date.

[kmansoft]

@kmansoft i think first merge #121

Got it, great, once that's done I'll fork the main repo again.

[dswarbrick]

Is there some particular reason why the LimitNOFILE and LimitNPROC are not part of the main service file?

Also, maybe it's not a great idea to run the service as root, if it's not really necessary.

[kmansoft]

Is there some particular reason why the LimitNOFILE and LimitNPROC are not part of the main service file?

If by "main" you mean the one for Debian - no reason, just oversight on my part.

Also, maybe it's not a great idea to run the service as root, if it's not really necessary.

Agreed. Maybe already fixed in the RPM packages, but not for DEB.

The install script will need to chown any already existing data files.

[dswarbrick]

A common pattern used by the various Prometheus packages on Debian, which require the ability to override command line settings, is to source variables from an external config file.

For example:

$ cat /etc/default/prometheus
# Set the command-line arguments to pass to the server.
ARGS="--storage.tsdb.retention.time=30d --query.max-samples=10000000"

$ systemctl cat prometheus
# /lib/systemd/system/prometheus.service
[Unit]
Description=Monitoring system and time series database
Documentation=https://prometheus.io/docs/introduction/overview/

[Service]
Restart=always
User=prometheus
EnvironmentFile=/etc/default/prometheus
ExecStart=/usr/bin/prometheus $ARGS
ExecReload=/bin/kill -HUP $MAINPID
TimeoutStopSec=20s
SendSIGKILL=no
LimitNOFILE=8192

The EnvironmentFile in the systemd unit, and passing $ARGS to the ExecStart allows you to use a one-size-fits-all unit, whilst still allowing customization of options.

[agarthetiger]

There is also an Ansible Role on GitHub which installs a single VictoriaMetrics instance as a service which I came across today along with this issue. It is at https://github.com/dreamteam-gg/ansible-victoriametrics-role in case it's useful to anyone else who lands here.

[valyala]

There is also https://github.com/VictoriaMetrics/ansible-playbooks