Fail2ban

Fail2Ban Installation from GitHub

In case an existing Fail2Ban server is running:

sudo service fail2ban stop

Download Fail2Ban from GitHub:

export VERSION=0.10.0
wget https://github.com/fail2ban/fail2ban/archive/"${VERSION}".tar.gz -O fail2ban-"${VERSION}".tar.gz

Unpack and install:

sudo tar -zxpvf fail2ban-"${VERSION}".tar.gz

cd fail2ban-"${VERSION}"

sudo python setup.py install

This will install Fail2Ban into the python library directory. The executable scripts are placed into /usr/local/bin and configuration under /etc/fail2ban.

Enable fail2ban as an automatic service:

sudo cp files/debian-initd /etc/init.d/fail2ban

sudo update-rc.d fail2ban defaults

sudo service fail2ban start

Changes in configuration

Avoid banning IPs from local network (file /etc/fail2ban/jail.conf)

# "ignoreip" can be a list of IP addresses, CIDR masks or DNS hosts. Fail2ban
# will not ban a host which matches an address in this list. Several addresses
# can be defined using space (and/or comma) separator.
#ignoreip = 127.0.0.1/8 ::1
#>>>2019-04-13 Frank Wulf
ignoreip = 127.0.0.1/8 192.168.141.0/24
#<<<2019-04-13 Frank Wulf

Adjust regular expression for SSH rules (file /etc/fail2ban/filter.d/sshd.conf)

cmnfailre = ^[aA]uthentication (?:failure|error|failed) for <F-USER>.*</F-USER> from <HOST>( via \S+)?%(__suff)s$
            ^User not known to the underlying authentication module for <F-USER>.*</F-USER> from <HOST>%(__suff)s$
            ^Failed publickey for invalid user <F-USER>(?P<cond_user>\S+)|(?:(?! from ).)*?</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
            ^Failed \b(?!publickey)\S+ for (?P<cond_inv>invalid user )?<F-USER>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
            ^<F-USER>ROOT</F-USER> LOGIN REFUSED FROM <HOST>
            ^[iI](?:llegal|nvalid) user <F-USER>.*?</F-USER> from <HOST>%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because not listed in AllowUsers%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because listed in DenyUsers%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because not in any group%(__suff)s$
            ^refused connect from \S+ \(<HOST>\)
            ^Received <F-MLFFORGET>disconnect</F-MLFFORGET> from <HOST>%(__on_port_opt)s:\s*3: .*: Auth fail%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because a group is listed in DenyGroups%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because none of user's groups are listed in AllowGroups%(__suff)s$
            ^<F-NOFAIL>%(__pam_auth)s\(sshd:auth\):\s+authentication failure;</F-NOFAIL>(?:\s+(?:(?:logname|e?uid|tty)=\S*)){0,4}\s+ruser=<F-ALT_USER>\S*</F-ALT_USER>\s+rhost=<HOST>(?:\s+user=<F-USER>\S*</F-USER>)?%(__suff)s$
            ^(error: )?maximum authentication attempts exceeded for <F-USER>.*</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?%(__suff)s$
            ^User <F-USER>.+</F-USER> not allowed because account is locked%(__suff)s
            ^<F-MLFFORGET>Disconnecting</F-MLFFORGET>(?: from)?(?: (?:invalid|authenticating)) user <F-USER>\S+</F-USER> <HOST>%(__on_port_opt)s:\s*Change of username or service not allowed:\s*.*\[preauth\]\s*$
            ^<F-MLFFORGET>Disconnecting</F-MLFFORGET>: Too many authentication failures(?: for <F-USER>.+?</F-USER>)?%(__suff)s$
            ^<F-NOFAIL>Received <F-MLFFORGET>disconnect</F-MLFFORGET></F-NOFAIL> from <HOST>%(__on_port_opt)s:\s*11:
            ^<F-NOFAIL>Connection <F-MLFFORGET>closed</F-MLFFORGET></F-NOFAIL> by%(__authng_user)s <HOST><mdrp-<mode>-suff-onclosed>
            ^<F-MLFFORGET><F-MLFGAINED>Accepted \w+</F-MLFGAINED></F-MLFFORGET> for <F-USER>\S+</F-USER> from <HOST>(?:\s|$)
#>>>2018-09-11 Frank Wulf
            ^Disconnected from <HOST>%(__on_port_opt)s\s+\[preauth\]\s*$
#<<<2018-09-11 Frank Wulf

Using IP sets instead of iptables rules

Difference between iptables and IP sets

By default Fail2Ban uses iptables rules to block IP addresses. These rules are processed sequentially and can therefore result in slow response time due to a large number of entries.

Example:

iptables -S

Result:

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N f2b-sshd
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A f2b-sshd -s 120.52.56.124/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 116.193.161.242/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 14.215.237.205/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 118.244.238.18/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 155.133.82.12/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 49.4.6.132/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 118.244.206.22/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 61.132.29.162/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 192.160.102.169/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 162.247.72.213/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 149.56.223.241/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 27.255.79.82/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 211.104.171.220/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 187.252.208.82/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -s 116.6.49.126/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -j RETURN

Since Linux Kernel 2.6 there is an option to use so-called IP sets to hold big amount of IP addresses in the memory. This technique uses hashtables to store and search IP adresses and is therefore much more efficient than parsing sequentially through the iptables rules.

Shell script fwfail2ban

The following shell script fwfail2ban moves IP addresses from the iptables rules to an IP set:

#!/bin/bash
#
# Author:  Frank Wulf
# Version: 1.3 (2022-03-01)
#
# This program moves iptables entries created by fail2ban to
# an IP set in the Linux Kernel. Advantage is that ipset uses
# a hashtable to store/fetch IP addresses and thus the IP lookup
# is much more efficient and faster than sequentially parsing
# the iptables rules.
#
# Version history:
# 1.0   2017-10-04   Initial release
# 1.1   2019-04-19   Add DELETE command for new database table "bips"
# 1.2   2020-04-16   Replace "DELETE from db" logic by "fail2ban unban --all"
# 1.3   2022-03-01   Add logic to handle ipsets reaching maximum number of elements

# Get program name to be used as prefix
pfx=$(basename $0)

# Temporary files
chn=/tmp/$pfx.chn
out=/tmp/$pfx.out

# Get fail2ban database
db=`confget -f /etc/fail2ban/fail2ban.conf dbfile`

# Get all chains created by fail2ban
iptables -S|grep "^\-A f2b-"|awk '{print $2}'|sed "s/f2b-//"|sort -u >$chn

while read chain; do

  # Build the ipset if not exist
  ipset -exist create $pfx-$chain hash:ip timeout 0

  # Build the iptables rules to use ipset if not exist
  for i in INPUT FORWARD; do
    if ! `iptables -C $i -m set --match-set $pfx-$chain src -j DROP 1>/dev/null 2>&1`; then
      iptables -I $i -m set --match-set $pfx-$chain src -j DROP
    fi
  done

  # Get banned IP addresses from iptables
  iptables -L f2b-$chain -v -n|grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'|awk '{print $8}'|grep -v '0\.0\.0\.0' >$out

  # Get bantime from fail2ban configuration
  bantime=`confget -f /etc/fail2ban/jail.local -s $chain bantime`

  while read ipaddr; do
    sw_ipset=1
    # If bantime is not persistent then the timeout for ipset needs to be
    # adjusted depending on the time when fail2ban has banned the IP address.
    if [ "$bantime" == "" -o "$bantime" == "-1" ]; then
      timeout=0
    else
      timeofban=`sqlite3 -batch $db "SELECT timeofban FROM bans WHERE jail = '$chain' AND ip = '$ipaddr' LIMIT 1"`
      timeout=$(($bantime - (`date +%s` - $timeofban)))
      if [ $timeout -le 0 ]; then
        # Bantime has exceeded (should happen only if fail2ban is unbanning an IP address
        # right during the runtime of this program).
        sw_ipset=0
      fi
    fi
    if [ $sw_ipset -eq 1 ]; then
      # Check if ipset has reached maximum number of elements
      maxelem=`ipset list -t $pfx-$chain | grep "maxelem"|awk '{print $7}'`
      numelem=`ipset list -t $pfx-$chain | grep "Number of entries"|awk -F: '{print $2}'`

      if [ `echo "$maxelem - $numelem" | bc` -lt 1 ]; then
        # ipset is full => rename with date stamp and create new one

        # Remove iptables rules to enable renaming of ipset
        for i in INPUT FORWARD; do
          iptables -D $i -m set --match-set $pfx-$chain src -j DROP
        done

        oldset=$pfx-$chain-`date +'%Y%m%d'`
        ipset rename $pfx-$chain $oldset
        ipset create $pfx-$chain hash:ip timeout 0

        # Build the iptables rules to use old and new ipset
        for i in INPUT FORWARD; do
          iptables -I $i -m set --match-set $oldset src -j DROP
          iptables -I $i -m set --match-set $pfx-$chain src -j DROP
        done
      fi

      # Add IP address to ipset
      ipset add $pfx-$chain $ipaddr timeout $timeout 1>/dev/null 2>&1
    fi
  done <$out

  # Flush all rules in this chain created by fail2ban
  iptables -F f2b-$chain

done <$chn

# Remove temporary files
rm $out 1>/dev/null 2>&1
rm $chn 1>/dev/null 2>&1

# "Unban" all IP addresses to clear iptables rules as the entries are now in the IP set.
/usr/local/bin/fail2ban-client unban --all  1>/dev/null 2>&1

# Save IP set to enable restoring after reboot
ipset save -f /etc/iptables/rules.ipset

# Save iptables to enable restoring after reboot, the entries created
# by fail2ban are filtered, those will be restored by fail2ban itself.
iptables-save | grep -v "^\-A.*f2b-" >/etc/iptables/rules.v4

The script runs once a day via cron. During the day fail2ban bans IP addresses with iptables rules which are then moved at 0:00 o'clock to an IP set.

By default both iptables rules and IP sets are hold in the memory and get lost during a reboot. Therefore the script saves the data which then can be automatically restored after a reboot.