Docker-mailserver: [documentation] Reload/restart after accounts/virtual change?

[strarsis]

Because no postmap files are used - is it required to reload/restart the server daemons after
changing the 'postfix-account.cf' and/or 'postfix-virtual.cf' file?
Do both daemons (postfix + dovecot) need a restart or can a softer reload be used after config change?

Config files are changed without setup.sh (synced/uploaded), hence no setup.sh is involved
(that may perform additional action to ensure the new config takes effect).

[voldedore]

I didn't take much time to view the setup.sh

But I don't think that the server daemons will know to update user list when you just change the postfix-account.cf.

As for me, I used setup.sh script to add new user and had to restart container to take effect, although I think we just need to restart ~dovecot~ postfix service.

[strarsis]

Correct, according to documentation, process will not detect changes to texthash type:
http://www.postfix.org/DATABASE_README.html#types

Postfix has to be reloaded for using the new config:
http://www.postfix.org/DATABASE_README.html#detect

postfix reload takes away performance each time - it is recommended in
documentation using a config type that doesn't require reloading - like hash (postmap).

[tomav]

https://github.com/tomav/docker-mailserver/pull/382 - cc @tyranron

This can be changed (again). We can talk about that here.

[tyranron]

@strarsis why not just restart whole container on config change? Docker containers are rarely designed in a way to reload stuff inside it. I suppose there can appear more issues related to it.

[strarsis]

@tyranron: During container restart mail clients fail to connect and retrieve mail.
Also I had issues in the past where a restart fails because of incorrect ownership/permissions while the container still runs - resulting in downtime until it is discovered and manually fixed.

[tomav]

Reloading postfix != Restart container
Reloading is enough and much faster.

[strarsis]

So when the accounts/aliases configuration files have been changed,
postfix has to be reloaded because the currently used configuration file format
requires a reload after each change.

[tomav]

texthash on aliases does not need reload/restart if done in /etc/postfix folder of your running container.
hash needs to, with a postmap before.

[tyranron]

I understand your concerns.

But again, in most cases HA problem in Docker world is solved outside container by orchestration stuff (rolling restarts, multiple instances, etc). There are few exceptions I've seen related to situations when application represents some kind of controller to work with dynamic services discovery.

The #382 was made from that point of view:

As far as Postfix setup inside Docker container does not require to re-apply configuration on-fly, and reconfiguration is done with container restarts

However, as practice show it actually does.

I don't see any problems with returning back to hash format instead texthash if it's required.

[tomav]

And we'll have to document that part too. This is a recurring question.
@strarsis?

[strarsis]

@tyranron: For a full restart of the mail docker container some kind of LB for all the
mail protocols is needed and the restart would consist of starting a wholly new container instance -
so services are not interrupted for a lengthy time that it takes for the mail daemons to startup.
Also the daemons have to cope with other instances of themselves accessing
the mounted data (Dovecot and Maildir).
In other container based implementations, e.g. https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion, the daemon inside the container is just reloaded, too.

Either the user has to manually restart / reload the daemons, or
another config format has to be used (it requires to run postmap after edit).
For restarting / reloading the daemons after config or cert/key change a convenience script could
be offered to the user - or the ACME client to invoke automatically after cert renewal.

[johansmitsnl]

Any suggestions on how to solve it?
Or change to a postmap file is the best way forward?

[strarsis]

Changing to postmap and providing a shell script for running postmap over it seems to be the best alternative to a full database.

[johansmitsnl]

Adding a watch function to the file that generates the postmap might also work.
Are you willing to contribute in a PR?

[strarsis]

In the rare case of someone using this over network (e.g. samba share),
the watch feature may not work and silently fail though.

[johansmitsnl]

True, could you create a pull request?

[strarsis]

I can try but I can't promise anything time-wise. 🐱

[kamuri]

Ok make a script what does this job and run as a background application.
Test it on my system and works fine. Would like to push it to a branch so you can check it out.

[johansmitsnl]

@kamuri Looks good!, added some comments that needs to be changed.
Could you also add a test for this to ensure it will not break in the future?

[kamuri]

Ok add the changes and test it locally. Works.
What kind of test you mean?

[johansmitsnl]

Changes are good to me, we add tests to ensure it does not break in the future.
My suggestion would be to create one test that include these steps and asserts:

• Test a login failure for a new user
• Add user
• Wait 2 seconds (the script waits every second)
• Test a login success for the new user
• Send a email to the new user
• Wait 10 seconds (suggestion you have to check if this is enough)
• Verify that the email has arrived

Here are some parts of the tests

test login failure for dovecot and postfix
https://github.com/tomav/docker-mailserver/blob/12f0b5f0335814858f4b73760d3b033b425a8f52/test/tests.bats#L223-L226
https://github.com/tomav/docker-mailserver/blob/12f0b5f0335814858f4b73760d3b033b425a8f52/test/tests.bats#L252-L255

Test login success for dovecot and postfix
https://github.com/tomav/docker-mailserver/blob/12f0b5f0335814858f4b73760d3b033b425a8f52/test/tests.bats#L218-L221
https://github.com/tomav/docker-mailserver/blob/12f0b5f0335814858f4b73760d3b033b425a8f52/test/tests.bats#L247-L250

Vhost changes
https://github.com/tomav/docker-mailserver/blob/12f0b5f0335814858f4b73760d3b033b425a8f52/test/tests.bats#L375-L381

Should be 1 email received
https://github.com/tomav/docker-mailserver/blob/12f0b5f0335814858f4b73760d3b033b425a8f52/test/tests.bats#L300-L304

The command line I use to run tests is:
make clean && git checkout test/config && make build && make generate-accounts run fixtures tests

Any questions please let me know

[kamuri]

Ok running while testing in this error.

Oct  8 18:18:46 mail postfix/postfix-script[2241]: fatal: the Postfix mail system is already running
Oct  8 18:19:35 mail postfix/postfix-script[3471]: fatal: the Postfix mail system is already running
Oct  8 18:23:18 mail crontab[4809]: (root) LIST (root)
Oct  8 18:23:53 mail postfix/postfix-script[5917]: fatal: the Postfix mail system is already running

which cause the test to fail for the checks.

 ✗ checking logs: mail related logs should be located in a subdirectory
   (from function `assert_output' in file test/test_helper/bats-assert/src/assert.bash, line 239,
    in test file test/tests.bats, line 250)
     `assert_output 3' failed    

   -- output differs --
   expected : 3
   actual   : 4
   --

and this

 ✗ checking system: postfix should not log to syslog
   (from function `assert_failure' in file test/test_helper/bats-assert/src/assert.bash, line 140,
    in test file test/tests.bats, line 879)
     `assert_failure' failed

   -- command succeeded, but it was expected to fail --
   output (2 lines):
     Oct  8 18:18:46 mail postfix/postfix-script[2241]: fatal: the Postfix mail system is already running
     Oct  8 18:19:35 mail postfix/postfix-script[3471]: fatal: the Postfix mail system is already running
   --

Change the supervisorctl restart postfix back to postfix reloadsolve it.

[johansmitsnl]

First good work! My only concern is the postfix reload. I added a comment could you look into this to see if this solves without a reload?

[johansmitsnl]

I see now that the restart script of postfix does not work, trying to fix it

[johansmitsnl]

Traced down the issue. The service script used uses a check where the image needs more privileges.
In the compose file you should have the SYS_PTRACE added. When this is added it works as expected.

   cap_add:
      - SYS_PTRACE

Could you change the run command with --cap-add=SYS_PTRACE in it and see if this solves the warning and passes the tests?

[kamuri]

When i try to start the container with this option i run in an other Problem while testing.

# Sending test mails
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-spam.txt"
make: *** [fixtures] Error 1

Solve this by adding a sleep 20 here

    docker exec mail /bin/sh -c "maildirmake.dovecot /var/mail/localhost.localdomain/user1/.INBOX.spam"
    docker exec mail /bin/sh -c "chown 5000:5000 -R /var/mail/localhost.localdomain/user1/.INBOX.spam"
    sleep 20
    # Sending test mails
    docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-spam.txt"

dont know why but the container didnt resonse after the docker exec mail /bin/sh -c "chown 5000:5000 -R /var/mail/localhost.localdomain/user1/.INBOX.spam" for a few seconds. 5 was not enought.

But anyway now it works as it should. I add the --cap-add=SYS_PTRACE in the .dist and readme.MD too in my last request.