Fail2ban: Fail2ban fails to run after upgrade to Ubuntu 16.04

[enoch85]

Environment:

• Fail2Ban version (including any possible distribution suffixes): iF fail2ban 0.9.3-1 all
• OS, including release name/version: Ubuntu Server 16.04 updated from Ubuntu Server 14.04
• [X] Fail2Ban installed via OS/distribution mechanisms
• [X] You have not applied any additional foreign patches to the codebase
• [ ] Some customizations were done to the configuration (provide details below is so)
  

  The issue:

After I upgraded to Ubuntu 16.04, fail2ban won't start. I tried to reinstall and reboot but it didn't work. I ran this config before and it worked: http://stuffphilwrites.com/2013/03/permanently-ban-repeat-offenders-fail2ban/

Steps to reproduce

1. Upgrade to Ubuntu 16.04
   

   Expected behavior

Fail2ban should start

Observed behavior

Fail2ban doesn't start

Any additional information

Everything worked before the upgrade, now it fails

Configuration, dump and another helpful excerpts

Any customizations done to /etc/fail2ban/ configuration

Relevant parts of /var/log/fail2ban.log file:

_preferably obtained while running fail2ban with loglevel = 4_

Relevant lines from monitored log files in question:

Apr 24 11:20:24 NGNIX systemd[1]: Failed to start Fail2Ban Service.
Apr 24 11:20:24 NGNIX systemd[1]: fail2ban.service: Unit entered failed state.
Apr 24 11:20:24 NGNIX systemd[1]: fail2ban.service: Failed with result 'exit-code'.
Apr 24 11:20:24 NGNIX systemd[1]: fail2ban.service: Service hold-off time over, scheduling restart.
Apr 24 11:20:24 NGNIX systemd[1]: Stopped Fail2Ban Service.
Apr 24 11:20:24 NGNIX systemd[1]: fail2ban.service: Start request repeated too quickly.
Apr 24 11:20:24 NGNIX systemd[1]: Failed to start Fail2Ban Service.

root@NGNIX:~# apt-get install fail2ban 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  mailx monit
The following NEW packages will be installed:
  fail2ban
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/227 kB of archives.
After this operation, 1,180 kB of additional disk space will be used.
Selecting previously unselected package fail2ban.
(Reading database ... 94758 files and directories currently installed.)
Preparing to unpack .../fail2ban_0.9.3-1_all.deb ...
Unpacking fail2ban (0.9.3-1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu4) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up fail2ban (0.9.3-1) ...
Job for fail2ban.service failed because the control process exited with error code. See "systemctl status fail2ban.service" and "journalctl -xe" for details.
invoke-rc.d: initscript fail2ban, action "start" failed.
dpkg: error processing package fail2ban (--configure):
 subprocess installed post-installation script returned error exit status 1
Processing triggers for systemd (229-4ubuntu4) ...
Processing triggers for ureadahead (0.100.0-19) ...
Errors were encountered while processing:
 fail2ban
E: Sub-process /usr/bin/dpkg returned an error code (1)

root@NGNIX:/etc/fail2ban# service fail2ban restart 
Job for fail2ban.service failed because the control process exited with error code. See "systemctl status fail2ban.service" and "journalctl -xe" for details.
root@NGNIX:/etc/fail2ban# cat /var/log/fail2ban.log 
root@NGNIX:/etc/fail2ban# fail2ban -f start
fail2ban: command not found
root@NGNIX:/etc/fail2ban# service fail2ban start -f
Job for fail2ban.service failed because the control process exited with error code. See "systemctl status fail2ban.service" and "journalctl -xe" for details.
root@NGNIX:/etc/fail2ban# journalctl -xe
Apr 24 11:24:43 NGNIX systemd[1]: Stopped Fail2Ban Service.
-- Subject: Unit fail2ban.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has finished shutting down.
Apr 24 11:24:43 NGNIX systemd[1]: Starting Fail2Ban Service...
-- Subject: Unit fail2ban.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has begun starting up.
Apr 24 11:24:44 NGNIX fail2ban-client[1255]: ERROR  Failed during configuration: While reading from '/etc/fail2ban/jail.local' [line 146]: op
Apr 24 11:24:44 NGNIX systemd[1]: fail2ban.service: Control process exited, code=exited status=255
Apr 24 11:24:44 NGNIX systemd[1]: Failed to start Fail2Ban Service.
-- Subject: Unit fail2ban.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has failed.
-- 
-- The result is failed.
Apr 24 11:24:44 NGNIX systemd[1]: fail2ban.service: Unit entered failed state.
Apr 24 11:24:44 NGNIX systemd[1]: fail2ban.service: Failed with result 'exit-code'.
Apr 24 11:24:44 NGNIX systemd[1]: fail2ban.service: Service hold-off time over, scheduling restart.
Apr 24 11:24:44 NGNIX systemd[1]: Stopped Fail2Ban Service.
-- Subject: Unit fail2ban.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has finished shutting down.
Apr 24 11:24:44 NGNIX systemd[1]: fail2ban.service: Start request repeated too quickly.
Apr 24 11:24:44 NGNIX systemd[1]: Failed to start Fail2Ban Service.
-- Subject: Unit fail2ban.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has failed.
-- 

[yarikoptic]

What do you have there?

Failed during configuration: While reading from '/etc/fail2ban/jail.local' [line 146]: op

[enoch85]

Yes it says that port already exists in pam_generic Issue is solved if I comment that out, plus like everything expect what I use in jail.local. But still, the packages shouldn't require the user to do that.

[yarikoptic]

Regarding package file bug in Ubuntu unless you got it directly from me on Debian or NeuroDebian.
I still haven't seen your jail.local, which seems to be too big, so not exactly sure what you commented out there

[enoch85]

@yarikoptic Here you go on my newly updated server. Been doing 7 servers today, and they all were the same, except this one that totally destroyed my old jail.local. :/ http://pastebin.com/KN6dLm3c

[yarikoptic]

Who destroyed your .local file? Installation process shouldn't touch it... Your local is a copy of original jails, whenever it should have carried only the customizations you needed... Some man page described it

On April 24, 2016 12:00:14 PM EDT, Daniel Hansson [email protected] wrote:

@yarikoptic Here you go on my newly updated server. Been doing 7
servers today, and they all were the same, except this one that totally
destroyed my old jail.local. :/ http://pastebin.com/KN6dLm3c

---

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/fail2ban/fail2ban/issues/1396#issuecomment-213989683

[enoch85]

Well, It was there before the update, and not there after. So you tell me :)

Anyway, issue is solved by commenting out the pam_generic section, feel free to close if you don't want to hunt this bug.

[yarikoptic]

Not sure if the is any action for us to take here, so will close
As for disappearing files, of somewhat a helper etckeeper could be. If you could replicate and skills that upgrade of fail 2ban removes some .local file that would indeed be a grave issue but so far I don't see any evidence ;-)

[irobayna]

BTW, same exact problem as @enoch85 on an upgrade to Ubuntu 16.04 from a working Ubuntu 14.04

[jauderho]

Same issue here.

[enoch85]

@yarikoptic Maybe you should investigate this after all? :smile:

[yarikoptic]

may be... or may be Ubuntu MOTUs should... was issue reported in ubuntu? I haven't received a similar report in Debian, where I maintain Fail2Ban. And again my summary was:

• installation of Fail2Ban was configured without following recommended practice of minimal customizations in .local files, instead full .conf was copied into .local and then tuned.
• pam_generic : you have further misconfigured it by providing duplicate pam-generic section and then double specification of port (from your http://pastebin.com/KN6dLm3c)

[pam-generic]

enabled  = false
# pam-generic filter can be customized to monitor specific subset of 'tty's
filter   = pam-generic
# port actually must be irrelevant but lets leave it all for some possible uses
port     = all
banaction = iptables-allports
port     = anyport
logpath  = /var/log/auth.log
maxretry = 6

• disappearance of .local file is a mystery to me which shouldn't happen and noone provided a reproducible example/demonstration
• "same here" is as descriptive as "I have a bug"

Overall, I have no clue on what to "investigate" further here besides recommending users to follow recommended practices

[enoch85]

Just to confirm. Installed on a new server, with a small jail.local file - everything works.

[voxnonecho]

Same issue here.

[bri-n]

FYI
I had the same error when following the instructions on DigitalOcean.com, which included the following
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
As you see, the suggestion here was to copy jail.conf and alter it. I gather from the comments here that jail.local should only have those sections that were added or changed. Possibly others have also started with a copy of jail.conf...

[yarikoptic]

Thanks for the confirmation. If someone straitens out the instructions at digital ocean, would be appreciated

[voxnonecho]

@bri-n I confirm I also used the instructions on DigitalOcean.com - I'm pretty sure this is the root cause of the issue.

[mitchellkrogza]

Also had initial problems using the guide at Digital Ocean. They seem to come up first on Google for just about anything but a few of their guides are not 100% accurate.

Sent from BlueMail

On 31 Jul 2016, 21:53, at 21:53, "Steven Beaupré" [email protected] wrote:

Same issue here.

---

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/fail2ban/fail2ban/issues/1396#issuecomment-236452758

[yarikoptic]

digital ocean

• [x] https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-centos-7 it provides correct recommendation.
• [ ] https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-ubuntu-14-04 (and probably others) incorrect Could anyone having an account there leave a comment recommending to adjust the page?

linode

• [ ] https://www.linode.com/docs/security/using-fail2ban-for-security?gclid=* incorrect
  Filed an issue https://github.com/linode/docs/issues/518

[kLOsk]

I suppose if anyone reads this the problem seems to be always the same.

line 146 of /etc/fail2ban/jail.local calls port for the second time. You can simply comment out everything if you don't use pam. Otherwise it should also help to simply comment out one of the ports.

line 146 of /etc/fail2ban/jail.local
# [pam-generic]

# enabled  = false
# pam-generic filter can be customized to monitor specific subset of 'tty's
# filter   = pam-generic
# port actually must be irrelevant but lets leave it all for some possible uses
# port     = all
# banaction = iptables-allports
# port     = anyport
# logpath  = /var/log/auth.log
# maxretry = 6

[AurelioDeRosa]

I had the same issue after updating and I solved it by commenting line 146.

[sebres]

@kLOsk, @AurelioDeRosa

I had the same issue after updating and I solved it by commenting line 146.

The problem is thus not really "solved", it was rather workarounded resp. translocated to the future.
Because the correct way - to hold jail.local so clean as possible (and for God's sake don't copy jail.conf in jail.local), so only your local changes should be made in jail.local (e.g. enabled = true).

So again: copying of jail.conf in jail.local is just a bad way

It's all been said in this thread above, so I'll appreciate the community spares this issue from comments like abovementioned, meaning please leave this issue by correct conclusion of @yarikoptic.
PS. I'll delete such comments in the future (or even lock this conversation to prevent possible confusion of other people).

[benmcnix]

WHOOPS! Just saw this is closed. Apologize.

I received a similar fault, with a different exit reason:

benmctee@xubuntu-server:~$ sudo systemctl start fail2ban
Job for fail2ban.service failed because the control process exited with error code. See "systemctl status fail2ban.service" and "journalctl -xe" for details.
benmctee@xubuntu-server:~$ journalctl -xe
Sep 10 10:46:13 xubuntu-server systemd[1]: fail2ban.service: Unit entered failed state.
Sep 10 10:46:13 xubuntu-server systemd[1]: fail2ban.service: Failed with result 'exit-code'.
Sep 10 10:46:13 xubuntu-server systemd[1]: fail2ban.service: Service hold-off time over, scheduling restart.
Sep 10 10:46:13 xubuntu-server systemd[1]: Stopped Fail2Ban Service.
-- Subject: Unit fail2ban.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has finished shutting down.
Sep 10 10:46:13 xubuntu-server systemd[1]: Starting Fail2Ban Service...
-- Subject: Unit fail2ban.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has begun starting up.
Sep 10 10:46:14 xubuntu-server fail2ban-client[14424]: ERROR  No file(s) found for glob /var/log/apache*/*error.log
Sep 10 10:46:14 xubuntu-server fail2ban-client[14424]: ERROR  Failed during configuration: Have not found any log file for apache-noscript jail
Sep 10 10:46:14 xubuntu-server systemd[1]: fail2ban.service: Control process exited, code=exited status=255
Sep 10 10:46:14 xubuntu-server systemd[1]: Failed to start Fail2Ban Service.
-- Subject: Unit fail2ban.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has failed.
-- 
-- The result is failed.
Sep 10 10:46:14 xubuntu-server systemd[1]: fail2ban.service: Unit entered failed state.
Sep 10 10:46:14 xubuntu-server systemd[1]: fail2ban.service: Failed with result 'exit-code'.
Sep 10 10:46:14 xubuntu-server systemd[1]: fail2ban.service: Service hold-off time over, scheduling restart.
Sep 10 10:46:14 xubuntu-server systemd[1]: Stopped Fail2Ban Service.
-- Subject: Unit fail2ban.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has finished shutting down.
Sep 10 10:46:14 xubuntu-server systemd[1]: Starting Fail2Ban Service...
-- Subject: Unit fail2ban.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has begun starting up.
Sep 10 10:46:14 xubuntu-server fail2ban-client[14433]: ERROR  No file(s) found for glob /var/log/apache*/*error.log
Sep 10 10:46:14 xubuntu-server fail2ban-client[14433]: ERROR  Failed during configuration: Have not found any log file for apache-noscript jail
Sep 10 10:46:14 xubuntu-server systemd[1]: fail2ban.service: Control process exited, code=exited status=255
Sep 10 10:46:14 xubuntu-server systemd[1]: Failed to start Fail2Ban Service.
-- Subject: Unit fail2ban.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit fail2ban.service has failed.
-- 
-- The result is failed.
Sep 10 10:46:14 xubuntu-server systemd[1]: fail2ban.service: Unit entered failed state.
Sep 10 10:46:14 xubuntu-server systemd[1]: fail2ban.service: Failed with result 'exit-code'.

After creating the directory /var/log/apache (sudo mkdir /var/log/apache) and then
sudo touch /var/log/apache/error.log
I was able to start the service using
sudo systemctl start fail2ban

This happened after an upgrade of Xubuntu to 16.04. I wanted to post this here since your troubleshooting method led me to finding my issue. So, thank you!

[sebres]

@benmctee

After creating the directory ... and touch file ... able to start the service

I suspect, all the issues (contain Ubuntu 16) will land here...

No, that is not the correct point, for your request (it is not why this origin issue was opened).
Yes, this issue is closed (but as already said, it is not really your issue), so should open a new one.
No, your suggested "solution" is wrong, because your fail2ban will never receive any failure over this apache-jail, so:

• if you don't use apache, just deactivate this jail with enabled=false (instead of mockup it this way)
• if your apache does not write in to a file anymore (keyword systemd), you can switch backend of this jail to systemd (or even your whole fail2ban backend, if all your services write to systemd journals);
• if your apache writes however the log file, but somewhere else, you should specify correct log-path in your jail.local.

And last but not least, if you installed your fail2ban via OS/distribution mechanisms, possibly you can better seek support from the distribution you obtained Fail2Ban from.