Nixpkgs: nginx: can't change cache and log directories

[koslambrou]

Following this PR #85862, the cache and log directories of nginx were hardcoded to var/cache and var/log respectively.

Because those locations are read-only for non-sudo users, I can't run nginx as a local user and I can't seem to change those paths.

To Reproduce

Here is my shell.nix:

let
  pkgs = import (builtins.fetchGit {
    name = "nixos-unstable-2020-08-24";
    url = "https://github.com/nixos/nixpkgs-channels/";
    ref = "refs/heads/nixos-unstable";
    rev = "c59ea8b8a0e7f927e7291c14ea6cd1bd3a16ff38";
  }) {};

in
  pkgs.mkShell {
    buildInputs = [
      pkgs.nginx
    ];
}

Commands to run in the same directory as the shell.nix:

$ nix-shell
$ nginx
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (2: No such file or directory)
2020/08/24 12:35:16 [emerg] 6270#6270: mkdir() "/var/cache/nginx/client_body" failed (2: No such file or directory)
$ nginx -p . # I would expect the paths to point in my local directory
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (2: No such file or directory)
2020/08/24 12:38:16 [emerg] 12264#12264: open() "./conf/nginx.conf" failed (2: No such file or directory)

Expected behavior

The path ./conf/nginx.conf is correctly changed, but /var/log/nginx/error.log should be ./logs/error.log.

Notify maintainers
@thoughtpolice @raskin @fpletz @globin

Metadata
nix-shell -p nix-info --run "nix-info -m" result:

 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.43, NixOS, 20.03.2015.e7752db2fb6 (Markhor)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.5`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos

[aanderse]

ping @Izorkin @Mic92 @flokli

[Izorkin]

@Mic92 @flokli @aanderse is it possible to add this variant?

diff --git a/pkgs/servers/http/nginx/generic.nix b/pkgs/servers/http/nginx/generic.nix
index 6ec5b0a7851..ac7620b91b8 100644
--- a/pkgs/servers/http/nginx/generic.nix
+++ b/pkgs/servers/http/nginx/generic.nix
@@ -5,6 +5,7 @@
 , withStream ? true
 , withMail ? false
 , withPerl ? true
+, withoutRootless ? true
 , modules ? []
 , ...
 }:
@@ -69,6 +70,7 @@ stdenv.mkDerivation {
     "--with-http_stub_status_module"
     "--with-threads"
     "--with-pcre-jit"
+  ] ++ optionals withoutRootless [
     "--http-log-path=/var/log/nginx/access.log"
     "--error-log-path=/var/log/nginx/error.log"
     "--pid-path=/var/log/nginx/nginx.pid"
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 619f74c9602..13e7dad4307 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -16438,6 +16438,10 @@ in
     modules = [ nginxModules.rtmp nginxModules.dav nginxModules.moreheaders nginxModules.shibboleth ];
   };

+  nginxRootless = nginxStable.override {
+    withoutRootless = false;
+  };
+
   libmodsecurity = callPackage ../tools/security/libmodsecurity { };

   ngircd = callPackage ../servers/irc/ngircd { };

./result/bin/nginx -V 2>&1 | sed -e 's|--|\n--|g'

nginx version: nginx/1.18.0
built by gcc 9.3.0 (GCC)
built with OpenSSL 1.1.1g  21 Apr 2020
TLS SNI support enabled
configure arguments:
--prefix=/nix/store/kkvxhll0jgxfzk48byykrysadb14imsk-nginx-1.18.0
--with-http_ssl_module
--with-http_v2_module
--with-http_realip_module
--with-http_addition_module
--with-http_xslt_module
--with-http_geoip_module
--with-http_sub_module
--with-http_dav_module
--with-http_flv_module
--with-http_mp4_module
--with-http_gunzip_module
--with-http_gzip_static_module
--with-http_auth_request_module
--with-http_random_index_module
--with-http_secure_link_module
--with-http_degradation_module
--with-http_stub_status_module
--with-threads
--with-pcre-jit
--with-stream
--with-stream_geoip_module
--with-stream_realip_module
--with-stream_ssl_module
--with-stream_ssl_preread_module
--with-http_image_filter_module
--with-file-aio
--add-module=/nix/store/dr6n543igdhj589qirfh36m5a5fcg47d-rtmp
--add-module=/nix/store/6pb7j6kymf3y4xs5blp3g8mwin2j22kk-dav
--add-module=/nix/store/y39g23fn8ikzcd1iy3b1bclqwjk2qmxd-moreheaders

[Mic92]

@Izorkin Would it be feasible to not hard-code those directories at all and just set them in the nixos module by default? Is there anything we would loose here? I think some configuration option can be passed via command line? The only issue I potentially see is that syntax errors in the configuration don't get logged properly to /var/log/nginx.

[Mic92]

I would prefer to not have another variant of nginx in all-packages.nix just for this use case.

[Izorkin]

I don't see any other variant but changing "--http-log-path=/var/log/nginx/access.log"
https://trac.nginx.org/nginx/ticket/147
https://trac.nginx.org/nginx/ticket/2011

[Mic92]

@Izorkin The ticket mentioned that it is possible to recompile nginx with --error-log-path=stderr, which logs everything to stderr by default. Is this not something we want anyway because we log everything to journald in our services?

[Izorkin]

With

diff --git a/pkgs/servers/http/nginx/generic.nix b/pkgs/servers/http/nginx/generic.nix
index 6ec5b0a7851..6dc115a1335 100644
--- a/pkgs/servers/http/nginx/generic.nix
+++ b/pkgs/servers/http/nginx/generic.nix
@@ -70,7 +70,7 @@ stdenv.mkDerivation {
     "--with-threads"
     "--with-pcre-jit"
     "--http-log-path=/var/log/nginx/access.log"
-    "--error-log-path=/var/log/nginx/error.log"
+    "--error-log-path=stderr"
     "--pid-path=/var/log/nginx/nginx.pid"
     "--http-client-body-temp-path=/var/cache/nginx/client_body"
     "--http-proxy-temp-path=/var/cache/nginx/proxy"

error create cache dirs:

2020/08/26 22:39:07 [notice] 57365#57365: ModSecurity-nginx v1.0.1 (rules loaded inline/local/remote: 0/0/0)
2020/08/26 22:39:07 [emerg] 57365#57365: mkdir() "/var/cache/nginx/client_body" failed (2: No such file or directory)

Need remove --http-...-temp-path options

[Izorkin]

ANd return stateDir

[Mic92]

@Izorkin if we remove those, does it try to create those in the installation directory instead or where does it put them? Also can the temp path not be overwritten via command line or configuration?

[Izorkin]

If remove --http-...-temp-path error:

2020/08/26 22:45:35 [emerg] 61749#61749: mkdir() "/nix/store/7qqpa9s9jlb2lmc1firniv4bkjxafj1l-nginx-1.19.2/client_body_temp" failed (30: Read-only file system)

Need running nginx with -p /custom_folder

[Mic92]

A different alternative would be to use client_body_temp_path in nginx.conf, which sounds also reasonable.

[flokli]

Isn't all that these flags in the derivation about configuring default paths, that are used when nginx is configured without these parameters?

I think it's sane to have nginx default to behaviour as a system daemon, which we currently do.

If people want to use nginx outside of the module system, can't they just set these values to their value, either by providing a nginx.conf, or by setting things with -g?

[aanderse]

On the flip side to that - if we can configure a program in the module to work while keeping defaults that work for a regular user then the program works out of the box in both scenarios, unlike the current situation which requires configuration from a user (configuration which may not be intuitive).

[flokli]

Sure, we can configure some things in then module, but I guess nginx should behave similarly to how it behaves on other distros (if that's not fundamentally incompatible with NixOS). Where are we differing here?

[aanderse]

That's a good point and question @flokli. Anyone happen to know what behavior you get if you run nginx on something like Debian, Arch, or RedHat?

[Izorkin]

@koslambrou check work nginx with this nginx configuration:

daemon off;
pid /home/user/nginx_test/logs/nginx.pid;
events {
  use epoll;
  worker_connections  128;
}
error_log /home/user/nginx_test/logs/error.log info;
http {
  client_body_temp_path /tmp/nginx_cache_client_body;
  proxy_temp_path /tmp/nginx_cache_proxy;
  fastcgi_temp_path /tmp/nginx_cache_fastcgi;
  uwsgi_temp_path /tmp/nginx_cache_uwsgi;
  scgi_temp_path /tmp/nginx_cache_scgi;

  server {
    listen 0.0.0.0:8080;
    root /home/user/nginx_test;
    access_log /home/user/nginx_test/logs/site_access.log;
    error_log /home/user/nginx_test/logs/site_error.log;
    location / {
      index index.html;
    }
  }
}

nginx -c /home/user/nginx_test/test.conf -p /home/user/nginx_test

nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (2: No such file or directory)
2020/08/27 09:29:28 [notice] 112061#112061: ModSecurity-nginx v1.0.1 (rules loaded inline/local/remote: 0/0/0)
^C%
Time: 0h:01m:48s

[koslambrou]

@Izorkin I tried your configuration. At first, I'm getting the following error:

nginx: [emerg] open() "/home/user/nginx_test/logs/error.log" failed (2: No such file or directory)

But after creating the directory with mkdir /home/user/nginx_test/logs, it seems to work.

[Mic92]

@koslambrou is this good enough for your use case?

[koslambrou]

Yes, thanks for your time!