Netbox: Replace supervisord with systemd

[candlerb]

Change Type

[ ] Addition
[ ] Correction
[X] Deprecation
[ ] Cleanup (formatting, typos, etc.)

Proposed Changes

I propose changing the installation instructions to remove supervisord and configure systemd instead.

supervisord depends on python2, which is soon to be end of life. Installing supervisord pulls in python2 as a dependency, introducing two python versions on a previously python3-only system.

Furthermore, the mainstream modern Linux distros already use systemd as default (RedHat 7, Ubuntu 16.04+, Debian Stretch+) and there is no need to introduce another daemon manager into those systems. Anyone choosing to run a different system can work out for themselves how to supervise a daemon.

Detail

Since gunicorn is installed via pip rather than system packages, we need to provide the unit file. The following is tested under Ubuntu 16.04.

Create a unit file /etc/systemd/system/netbox.service:

[Unit]
Description=Netbox - IP address management (IPAM) and data center infrastructure management (DCIM) tool
After=network-online.target
Wants=network-online.target
Conflicts=shutdown.target

[Service]
User=www-data
ExecStart=/usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
WorkingDirectory=/opt/netbox/netbox
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Then activate it:

systemctl daemon-reload
systemctl start netbox
systemctl enable netbox

A similar file would be required for rqworker, if that's in use.

Notes

This relates to #2457. That was rejected because of the overhead of documenting systemd alongside supervisord. Here I propose getting rid of supervisord entirely.

2457 includes multiple files, including configuration for a special flavour of Nginx called "unit". I don't think that should be included. Out-of-the-box packages for Nginx and Apache2 will already have suitable systemd unit files, so only one is required for Netbox (plus optional second one for rqworker).

[kasimon]

FWIW, that's almost exactly what I do (with the one difference that our inhouse netbox debian package ships its own virtualenv environment and thus its own gunicorn binary).

supervisord depends on python2, which is soon to be end of life. Installing supervisord pulls in python2 as a dependency, introducing two python versions on a previously python3-only system.

Furthermore, the mainstream modern Linux distros already use systemd as default (RedHat 7, Ubuntu 16.04+, Debian Stretch+) and there is no need to introduce another daemon manager into those systems. Anyone choosing to run a different system can work out for themselves how to supervise a daemon.

Both very valid points. I would welcome this update to the documentation.

[goebelmeier]

Works perfect for me without the need of another daemon manager.

[DanSheps]

I think a better approach, would be to include a "contrib" directory and instead use envirovars to control certain settings in the unit file. Less overhead and less chance for a mistake.

Would streamline the install process too.

[candlerb]

I am only proposing a simple swapout of supervisord config file with systemd config file.

@DanSheps: sorry I don't understand what you're saying. What exactly would you put in the "contrib" directory? What environment variables would "control certain settings", and how would the user be expected to set them?

[DanSheps]

contrib would have:

contrib/[email protected]
contrib/netbox.env

In the netbox.env, you have:

#WorkingDir='/usr/local/netbox/'
#NBUser='www-data'
#NBGroup='www-data'
#PidPath='/var/run/netbox.pid'
#Bind='127.0.0.1:8001'
#Workers=3
#Timeout=120
#Log='/usr/local/netbox/netbox.log'

If @jeremystretch is willing, I can do this as a PR, but I will need someone to sanity it as I only run CentOS.

[DanSheps]

If someone could sanity check my last commit for ubuntu, I think that will work for most instances, I can submit a PR assuming this is approved.

[Grokzen]

My 2 cents on this is that you should not replace this "yet". But instead to provide both solutions in the documentation for a period of time, but recommend/prefer the systemd solution.

[candlerb]

Why? What's the benefit of supervisord?

[tb-killa]

Why doesn't provide multiple install choices on documentation side and let the user select the right ones themselve ?
I think it's a bit of overhead to build all this stuff up because if someone is using netbox the person should know what he is doing.

[candlerb]

Why doesn't provide multiple install choices on documentation side

That was suggested in #2457, and rejected because of the overhead of maintaining two parallel sets of documentation.

I repeat: why is supervisord a better choice than systemd? Why not just document systemd, which is the standard for all mainstream Linux distributions? Then if as you say "the person should know what he is doing" they can install something else if they choose.

[pobk]

I would avoid putting any recommendations on system level processes entirely.

In answer to the question "why is supervisord better than systemd?": because supervisord is trying to do one job very well... unlike systemd which is trying to do all the jobs, but it's doing them badly. Case in point: systemd DNS resolver subsystem is very broken and does not comply with standard operational tennants. Also, why do you want your init system doing DHCP or managing your network or fiddling with your system clock? Why do you need it managing ALL THE THINGS?

Systemd is trying to undertake things, which personally, I believe the authors have little experience of, whereas the authors of things like syslog-ng or rsyslog, in the case of logging for example, have a great deal of experience in handling that data and their core remit is to handle ONLY that data...You don't see syslog-ng suddenly taking over DHCP-Client functions?

But back to my original point here: I can see a point for having a separate repo of init-scripts and docs around other extras for those who wish to contribute to in order to facilitate the installation of NetBox... I do not think systemd implementations should be a core part of the distribution/installation.

I've ansiblised NetBox on FreeBSD in an iocage. Systemd is (very thankfully) irrelevant in my environment.

[Grokzen]

You can argue that since python2 still has one year left before EOL it can still be in but put into deprecated state and migrated out over time.

Another argument that can be done is that is not a good deprecation and migration strategy to rip the old out and replace it with something new right away within one major release cycle.

[candlerb]

Another argument that can be done is that is not a good deprecation and migration strategy to rip the old out and replace it with something new right away within one major release cycle.

I am not suggesting that users should remove supervisord: I said to remove supervisord from the netbox documentation. Existing installations can continue to use supervisord without any changes.

In any case, neither supervisord nor systemd is part of netbox.

[pobk]

I am not suggesting that users should remove supervisord: I said to remove supervisord _from the netbox documentation_. Existing installations can continue to use supervisord without any changes.

In any case, neither supervisord nor systemd is part of netbox.

Agreed, although in defence of supervisord... It does abstract the process supervision away very nicely from any operating system requirements. Supervisord works on most *nix based operating systems I've worked with and the overall init system is abstracted away from the applications.

I've just checked... supervisord does have support for Py3 in development... From the github repo:

Supervisor is intended to work on Python 3 version 3.4 or later and on Python 2 version 2.7.

And also, from the supervisor issues list #1060 Python 3 support?

Looks like it is very much in progress and this issue is more of a non-issue.

[kasimon]

@pobk your systemd ranting is mostly unrelated to what this issue is about, please refrain from doing so. The systemd dns resolver for example has nothing to do with running netbox on a system.

[kasimon]

Let me add my 0.02 $currency: Currently, the setup documentation is specific to ubuntu 18.04 (which is using systemd). So I don't really see a point in installing additional software and running another service when what the system already contains is more than enough. This would certainly make the current documentation simpler and the resulting system a little bit easier to manage (the same management commands can be used that are already used for any other service on the system, no need to learn how to use supervisord).

One point that speaks for using supervisord in the documentation would be that it would make the documentation a little easier to be translated to other operating systems. But every freebsd (or non-linux/non-systemd) admin I know would have no problem to translate that simple systemd service description to something working on his/her system. And using a systemd service description in the documentation would make it easily translatable to almost any current mainstream linux distribution.

Summary: A systemd based setup would simplify the instructions without losing any features relevant to the scope of the document.

[DanSheps]

I would avoid putting any recommendations on system level processes entirely.

Unfortunately, this is going to be part of the installation of almost any system, look at httpd, sql, etc. Most of them handle the automatic creation of the systemd or initd scripts. Netbox has no automatic installation

In answer to the question "why is supervisord better than systemd?": because supervisord is trying to do one job very well... unlike systemd which is trying to do all the jobs, but it's doing them badly. Case in point: systemd DNS resolver subsystem is very broken and does not comply with standard operational tennants. Also, why do you want your init system doing DHCP or managing your network or fiddling with your system clock? Why do you need it managing ALL THE THINGS?

I think there is a mis-understanding of what systemd is, and what it isn't.

systemd is a system configurator and service bootstrapper. It actually does very well with managing services and system configuration.
>

Systemd ... ... DHCP-Client functions?

systemd is not a single binary, it is actually composed of separate binaries in one project. So, when you refer to the "resolver", you are actually referring to "resolved". systemd-resolved it is a daemon made by the maker of systemd, and it is not enabled by default on CentOS7 (not sure about Ubuntu or any other flavours).

But back to my original point here: I can see a point for having a separate repo of init-scripts and docs around other extras for those who wish to contribute to in order to facilitate the installation of NetBox... I do not think systemd implementations should be a core part of the distribution/installation.

If you feel that way, I don't think supervisord should be included either. It is still a service bootstrapper.

I've ansiblised NetBox on FreeBSD in an iocage. Systemd is (very thankfully) irrelevant in my environment.

Your usage is most likely not mainstream. However, thankfully these instructions are not REQUIRED for netbox to function. If you want to go outside the setup docs, you are more or less on your own as far as getting it to work, which it sounds like you did anyways.

The main argument for using systemd, from what I can see, is it reduces a dependency from Netbox. Systemd is in most major Linux versions around now, so apart from BSD, systemd will be available on most systems running the software.

By removing supervisord, you also reduce the chance that there is a failure somewhere along the line. supervisord is also started by systemd on almost all systems covered by the installation docs. So all that is being done is removing a middleware that could potentially fail or introduce failures along the way.

[DanSheps]

@candlerb Thanks for taking a look and sanity checking the commit, I did a few more updates.

[pobk]

@pobk your systemd ranting is mostly unrelated to what this issue is about, please refrain from doing so.

@kasimon: I would ask that you refrain grandstanding and throwing around admonishments at anyone as you have done to me. My systemd ranting, while very ranty and verbose (for which I will offer an apology), is related to the discussion and very much on-topic, whereas admonitions are irrelevant.

WIth respect to "FreeBSD admins will be able to translate systemd"... That's horrifically short sighted and exclusionist: Yes, why not make the documentation impenetrable to new engineers or those that aren't as experienced. Lets write documentation that focuses on a single OS.

The documentation already says:

The installation instructions provided here have been tested to work on Ubuntu 18.04 and CentOS 7.5. The particular commands needed to install dependencies on other distributions may vary significantly. Unfortunately, this is outside the control of the NetBox maintainers. Please consult your distribution's documentation for assistance with any errors.

Focusing the documentation even more on a singular init system is a poor idea.

Documenting the deployment process on Supervisord keeps things agnostic. You don't need Py3 support for Supervisor for netbox to function and you don't exclude non-linux OS users.

The alternative, as suggested elsewhere is to write specific integration guides elsewhere. I would be happy to write a BSD flavour if this is a viable alternative.

[kasimon]

@kasimon: I would ask that you refrain grandstanding and throwing around admonishments at anyone as you have done to me. My systemd ranting, while very ranty and verbose (for which I will offer an apology), is related to the discussion and very much on-topic, whereas admonitions are irrelevant.

I was a bit harsh, I'm sorry for that. Please let us all try to stay as close as possible to what this issue is about, there is more than enough room elsewhere on the net to have general systemd discussions.

WIth respect to "FreeBSD admins will be able to translate systemd"... That's horrifically short sighted and exclusionist: Yes, why not make the documentation impenetrable to new engineers or those that aren't as experienced. Lets write documentation that focuses on a single OS.

Well, netbox is not a very complicated program to install and to run. Providing an example documentation that will fit probably >90% of those needing help with their setup seems not a bad idea to me.

The documentation already says:

The installation instructions provided here have been tested to work on Ubuntu 18.04 and CentOS 7.5. The particular commands needed to install dependencies on other distributions may vary significantly. [...]

I was referring to https://netbox.readthedocs.io/en/stable/installation/3-http-daemon/:

For the sake of brevity, only Ubuntu 18.04 instructions are provided here, but this sort of web server and WSGI configuration is not unique to NetBox. Please consult your distribution's documentation for assistance if needed.

Which I think is a good way to handle this.

[pobk]

I was a bit harsh, I'm sorry for that. Please let us all try to stay as close as possible to what this issue is about, there is more than enough room elsewhere on the net to have general systemd discussions.

Accepted and agreed.

Well, netbox is not a very complicated program to install and to run. Providing an example documentation that will fit probably >90% of those needing help with their setup seems not a bad idea to me.

It's not very complicated to us. Asking others might give you a different perspective.

I was referring to https://netbox.readthedocs.io/en/stable/installation/3-http-daemon/:

For the sake of brevity, only Ubuntu 18.04 instructions are provided here, but this sort of web server and WSGI configuration is not unique to NetBox. Please consult your distribution's documentation for assistance if needed.

Which I think is a good way to handle this.

For HTTPDs, I would agree entirely. What we are discussing here, however, is the supervisord/gunicorn side. My contention is:

1. describing a purely systemd implementation is a bad move.
2. supervisord is a useful container/runner which applies to all platforms
3. If you're going to document the systemd side of things, do so as a separate contrib docs which are not OS dependant.

[candlerb]

2. supervisord is a useful container/runner which applies to all platforms

Each OS will have its own standard mechanism for starting/stopping processes, which happens to be systemd for those OSes mentioned in the documentation.

But I wouldn't recommend supervisord to anyone, for this reason alone:

Supervisor is known to work with Python 2.4 or later but will not work under any version of Python 3.

In other words, supervisord is abandonware. And pulling in legacy python2 as a dependency causes debugging headaches.

[pobk]

1. supervisord is a useful container/runner which applies to all platforms

Each OS will have its own standard mechanism for starting/stopping processes, which happens to be systemd for those OSes mentioned in the documentation.

So remove it, instead of bleating on about some other poor software simply because a majority of users might be using an operating system that utilises it.

You're complaining about one "poor" piece of software and suggesting replacing it with another "poor" piece of software.

In other words, supervisord is abandonware. And pulling in legacy python2 as a dependency causes debugging headaches.

But as has been mentioned previously, they are working on that exact problem. It is very much not abandonware. Now you're just being sensationalist and obtuse.

And also, from the supervisor issues list #1060 Python 3 support?

[mnaberez]

Supervisor 4.0.0 (PyPI package; supports Python 3.4+ and 2.7)

[davidc]

Ranting aside (I don't like systemd either), it is simply a fact that a lot of OSes ship with systemd as standard these days, whereas none that I know of ship with supervisord. Changing the default installation instructions to systemd would make sense - creating less work for the sysadmin installing Netbox and resulting in fewer packages installed. As has already been mentioned, a competent sysadmin could use whatever they wanted anyway, so this just makes the standard installation use-case simpler.

(Perhaps keep the old supervisor file and instructions in a contrib directory as already suggested).

[DanSheps]

Closed in #3017