Unit: Nginx Unit --state saving multiple json configs on service restarts

Created on 24 Sep 2017  路  17Comments  路  Source: nginx/unit

I source compiled Nginx Unit on CentOS 7.4 64bit using configure options

./configure --prefix=/opt/unit --pid=/run/unitd.pid --log=/var/log/unitd.log --modules=modules --user=nginx --group=nginx --state=state

I assumed the --state option would save all loaded configurations as when I tested with just a single /root/tools/unitconfigs/php5631start.json loaded, a unitd service restart reloaded the config ok. But when I loaded more php configurations listed below, on unitd service restart, only the last loaded config was restored for /root/tools/unitconfigs/php720start.json ?

5.6.31

/root/tools/unitconfigs/php5631start.json

{
     "listeners": {
         "*:8300": {
             "application": "php56domaincom"
         }
     },
     "applications": {
         "php56domaincom": {
             "type": "php 5.6.31",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php5631start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

curl -X PUT -d @/root/tools/unitconfigs/php5631start.json --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "success": "Reconfiguration done."
}

7.0.24

/root/tools/unitconfigs/php7024start.json

{
     "listeners": {
         "*:8400": {
             "application": "php70domaincom"
         }
     },
     "applications": {
         "php70domaincom": {
             "type": "php 7.0.24",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php7024start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

curl -X PUT -d @/root/tools/unitconfigs/php7024start.json --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "success": "Reconfiguration done."
}

7.1.10

/root/tools/unitconfigs/php7110start.json

{
     "listeners": {
         "*:8500": {
             "application": "php71domaincom"
         }
     },
     "applications": {
         "php71domaincom": {
             "type": "php 7.1.10",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php7110start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

{
        "success": "Reconfiguration done."
}

7.2.0

/root/tools/unitconfigs/php720start.json

{
     "listeners": {
         "*:8600": {
             "application": "php72domaincom"
         }
     },
     "applications": {
         "php72domaincom": {
             "type": "php 7.2.0",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php720start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

curl -X PUT -d @/root/tools/unitconfigs/php720start.json --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "success": "Reconfiguration done."
}

but only the last config was restored on unitd service restart

curl --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "listeners": {
                "*:8600": {
                        "application": "php72domaincom"
                }
        },

        "applications": {
                "php72domaincom": {
                        "type": "php 7.2.0",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                }
        }
}
picture centminmod

All 17 comments

This state directory isn't a configuration storage. It was introduced just for runtime configuration persistence in order to allow Unit to survive server reloads without additional tricks. It's up to you where to store and how to manage your configurations.

VBart picture VBart on 24 Sep 2017

i see but even if i do not restart unitd service, i can only list the last added php720start.json configuration ? the previous 3 configs are gone ?

curl --unix-socket /opt/unit/control.unit.sock http://localhost/                                                     
{
        "listeners": {
                "*:8600": {
                        "application": "php72domaincom"
                }
        },

        "applications": {
                "php72domaincom": {
                        "type": "php 7.2.0",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                }
        }
}

unitd.log

2017/09/24 01:04:42 [info] 18356#18356 discovery started
2017/09/24 01:04:42 [notice] 18356#18356 module: php 5.6.31 "/opt/unit/modules/php5631.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: python 2.7.5 "/opt/unit/modules/python.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: php 5.6.31 "/opt/unit/modules/remiphp5631.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 ignoring /opt/unit/modules/remiphp5631.unit.so module with the same application language version php 5.6.31 as in /opt/unit/modules/php5631.unit.so
2017/09/24 01:04:42 [notice] 18356#18356 module: php 7.0.24RC1 "/opt/unit/modules/remiphp7023.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: php 7.1.10RC1 "/opt/unit/modules/remiphp719.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: php 7.2.0RC2 "/opt/unit/modules/remiphp720.unit.so"
2017/09/24 01:04:42 [info] 18357#18357 controller started
2017/09/24 01:04:42 [info] 18358#18358 router started
2017/09/24 01:04:42 [crit] 18358#18358 epoll_ctl(3, 3, 9) failed (2: No such file or directory)
2017/09/24 01:04:42 [crit] 18357#18357 epoll_ctl(3, 3, 9) failed (2: No such file or directory)
2017/09/24 01:04:42 [notice] 18355#18355 process 18356 exited with code 0
centminmod picture centminmod on 24 Sep 2017

They are gone not because of the reload. When you do PUT command to the root of the configuration / it's replaced by the provided object. The PUT has create or replace semantic.

VBart picture VBart on 24 Sep 2017

hmm the docs at isn't exactly clear on this http://unit.nginx.org/docs-configuration.html#creating-configuration-objects we have to separately add the listener and applications from separate json files ?

centminmod picture centminmod on 24 Sep 2017

There are 3 methods supported at the moment: GET, PUT, DELETE. You can operate over the configuration root / as well as any sublayers, like /listeners, /listeners/IP:PORT, /listeners/IP:PORT/application, /applications, /applications/NAME, /applications/NAME/workers, etc...

VBart picture VBart on 24 Sep 2017
👍1

You can add an application and then a listener, thus with two requests. For now, there's only one way to add both a listener and an application with just one request: by uploading the full resulting configuration to the root /.

Support for the MERGE/PATCH methods in conformity with RFC 7396 is planned.

VBart picture VBart on 24 Sep 2017
👍1

I also can suggest a nice and very powerful utility for operating with JSON:

% cat conf.json                     
{
        "listeners": {
                "127.0.0.1:8081": {
                        "application": "test"
                }
        },

        "applications": {
                "test": {
                        "type": "python",
                        "module": "wsgi",
                        "path": "/home/vbart/Development/tests"
                }
        }
}
% <conf.json | jq .applications.test
{
  "type": "python",
  "module": "wsgi",
  "path": "/home/vbart/Development/tests"
}
% <conf.json | jq '.listeners[]' 
{
  "application": "test"
}

https://stedolan.github.io/jq/

VBart picture VBart on 24 Sep 2017
👍1

And you can merge JSON files with it:

% cat conf.json
{
        "listeners": {
                "127.0.0.1:8081": {
                        "application": "test"
                }
        },

        "applications": {
                "test": {
                        "type": "python",
                        "module": "wsgi",
                        "path": "/home/vbart/Development/tests"
                }
        }
}
% cat conf_t.json
{
     "listeners": {
         "*:8300": {
             "application": "php56domaincom"
         }
     },
     "applications": {
         "php56domaincom": {
             "type": "php",
              "workers": 10,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "vbart",
              "group": "users",
              "index": "index.php"
         }
     }
}
% jq -s '.[0] * .[1]' conf.json conf_t.json
{
  "listeners": {
    "127.0.0.1:8081": {
      "application": "test"
    },
    "*:8300": {
      "application": "php56domaincom"
    }
  },
  "applications": {
    "test": {
      "type": "python",
      "module": "wsgi",
      "path": "/home/vbart/Development/tests"
    },
    "php56domaincom": {
      "type": "php",
      "workers": 10,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "vbart",
      "group": "users",
      "index": "index.php"
    }
  }
}
VBart picture VBart on 24 Sep 2017
👍1

cheers but don't we need to define the actual listener *:8300 too ?

cat /root/tools/unitconfigs/php5631start.json 
{
     "listeners": {
         "*:8300": {
             "application": "php56domaincom"
         }
     },
     "applications": {
         "php56domaincom": {
             "type": "php 5.6.31",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

if contents of /root/tools/unitconfigs/php5631start-app.json is

cat /root/tools/unitconfigs/php5631start.json | jq .applications.php56domaincom
{
  "type": "php 5.6.31",
  "workers": 20,
  "root": "/home/nginx/domains/domain.com/public",
  "user": "nginx",
  "group": "nginx",
  "index": "index.php"
}

for 

curl -X PUT -d @/root/tools/unitconfigs/php5631start-app.json --unix-socket /opt/unit/control.unit.sock http://localhost/applications/php56domaincom

and for /root/tools/unitconfigs/php5631start-listener.json contents would be

cat /root/tools/unitconfigs/php5631start.json | jq '.listeners[]'
{
  "application": "php56domaincom"
}

for

curl -X PUT -d @/root/tools/unitconfigs/php5631start-listener.json --unix-socket /opt/unit/control.unit.sock http://localhost/listeners

??

centminmod picture centminmod on 24 Sep 2017

Yes, you have to define it in URI: /listeners/*:8300

VBart picture VBart on 24 Sep 2017
👍1

cheers so

curl -X PUT -d @/root/tools/unitconfigs/php5631start-listener.json --unix-socket /opt/unit/control.unit.sock http://localhost/listeners/*:8300

lots to digest.. thanks

Hope this info makes it into official documentation manual too :)

centminmod picture centminmod on 24 Sep 2017

Merging with jq also works fine:

% jq -s '.[0] * .[1]' conf.json conf_t.json | curl -X PUT -d@-  http://127.0.0.1:8443
{
        "success": "Reconfiguration done."
}
% curl http://127.0.0.1:8443
{
        "listeners": {
                "127.0.0.1:8081": {
                        "application": "test"
                },

                "*:8300": {
                        "application": "php56domaincom"
                }
        },

        "applications": {
                "test": {
                        "type": "python",
                        "module": "wsgi",
                        "path": "/home/vbart/Development/tests"
                },

                "php56domaincom": {
                        "type": "php",
                        "workers": 10,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "vbart",
                        "group": "users",
                        "index": "index.php"
                }
        }
}
VBart picture VBart on 24 Sep 2017
👍1

@VBart thanks merging seems easier to do as I can script it to read all .json files within a config directory for merging :)

jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json 
{
  "listeners": {
    "*:8300": {
      "application": "php56domaincom"
    },
    "*:8400": {
      "application": "php71domaincom"
    },
    "*:8600": {
      "application": "php72domaincom"
    },
    "*:8500": {
      "application": "php71domaincom"
    }
  },
  "applications": {
    "php56domaincom": {
      "type": "php 7.1.9",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php70domaincom": {
      "type": "php 7.0.24",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php72domaincom": {
      "type": "php 7.2.0",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php71domaincom": {
      "type": "php 7.1.10",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    }
  }
}

though getting 405 not allowed error

 jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json  | curl -X PUT -d@-  http://localhost
<html>
<head><title>405 Not Allowed</title></head>
<body bgcolor="white">
<center><h1>405 Not Allowed</h1></center>
<hr><center>nginx</center>
</body>
</html>
centminmod picture centminmod on 24 Sep 2017

That error comes from nginx, not unit. =)

Note, that I used curl command without unix socket, because I run my Unit with control socket set to 127.0.0.1:8443.

VBart picture VBart on 24 Sep 2017
👍1

thanks for the hint added the control socket back

jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json | curl -X PUT -d@- --unix-socket /opt/unit/control.unit.sock http://localhost/ 
{
        "error": "Invalid configuration."
}

jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json
{
  "listeners": {
    "*:8300": {
      "application": "php56domaincom"
    },
    "*:8400": {
      "application": "php71domaincom"
    },
    "*:8600": {
      "application": "php72domaincom"
    },
    "*:8500": {
      "application": "php71domaincom"
    }
  },
  "applications": {
    "php56domaincom": {
      "type": "php 7.1.9",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php70domaincom": {
      "type": "php 7.0.24",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php72domaincom": {
      "type": "php 7.2.0",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php71domaincom": {
      "type": "php 7.1.10",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    }
  }
}
centminmod picture centminmod on 24 Sep 2017

If you have the modules collection that you mentioned in https://github.com/nginx/unit/issues/47, then php 7.1.9 is missing.

VBart picture VBart on 24 Sep 2017
👍1

ah ha thanks @VBart needed that second set of eyes :)

jq -s '.[0] * .[1] * .[2] * .[3]' php5631start.json php7024start.json php720start.json php7110start.json  | curl -X PUT -d@- --unix-socket /opt/unit/control.unit.sock http://localhost
{
        "success": "Reconfiguration done."
}

curl --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "listeners": {
                "*:8300": {
                        "application": "php56domaincom"
                },

                "*:8400": {
                        "application": "php70domaincom"
                },

                "*:8600": {
                        "application": "php72domaincom"
                },

                "*:8500": {
                        "application": "php71domaincom"
                }
        },

        "applications": {
                "php56domaincom": {
                        "type": "php 5.6.31",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                },

                "php70domaincom": {
                        "type": "php 7.0.24",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                },

                "php72domaincom": {
                        "type": "php 7.2.0",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                },

                "php71domaincom": {
                        "type": "php 7.1.10",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                }
        }
}
centminmod picture centminmod on 24 Sep 2017
Was this page helpful?
0 / 5 - 0 ratings

Related issues

Madhu1512 picture Madhu1512  路  4Comments
hongzhidao picture hongzhidao  路  3Comments
lolgab picture lolgab  路  3Comments
AkulovNV picture AkulovNV  路  4Comments
Izorkin picture Izorkin  路  4Comments