Harbor: rsyslogd, log service stuck in restarting state - .pid file remains after docker restart.

[pmihalcin]

After some time of using harbor, I can see that log service got stuck in Restarting state (after my restart):

harbor-log /bin/sh -c crond && rsyslo ... Restarting 0.0.0.0:1514->514/tcp

Looking at log container logs I can see the following:
If you want to run multiple instances of rsyslog, you need to specify
different pid files for them (-i option).
rsyslogd: run failed with error -3000 (see rsyslog.h or try http://www.rsyslog.com/e/3000 to learn what that number means)
rsyslogd: pidfile '/var/run/rsyslogd.pid' and pid 9 already exist.

Do you have any idea what the problem is about?

docker-compose -version

docker-compose version 1.7.1, build 0a9ab35

docker --version

Docker version 1.12.1, build 23cf638

[reasonerjt]

this happens because when you restart the containers the pid file remains, you can run "docker-compose down" to remove the containers and "docker-compose up" to recreate the containers to workaround.

[pmihalcin]

thanks.. this worked.. it'd still be good to have it restartable somehow without the need to remove container

[reasonerjt]

yep, will keep this issue open, and investigate later, but probably low prty.

[dejwsz]

I had successfully resolved this issue. I had just taken the rsyslog container definition and changed the CMD line to:
CMD crond && rm -f /var/run/rsyslogd.pid && rsyslogd -n
After this change it is working well and always restarting. Later I had just added new image name to my docker-compose.yml script of harbor. It would be good to have this change in harbor repo.

[hainingzhang]

Thanks for your solution. We will verify this. You are welcome to submit a PR if you like. @dejwsz

@reasonerjt @ywk253100

[reasonerjt]

@dejwsz PR #1197 has been merged, closing the issue.