Apache – How to use fail2ban to ban all php and cgi-bin requests

Fail2ban doesn't have neat functionality to automatically block attacks from a whole subnet. It is possible to do, though, using a recent version of fail2ban (I use v0.11), some simple fail2ban scripts and a small, pure python3 script.

Note: The question refers to 'a whole subnet' (which I'll refer to as CIDR blocks or IP ranges). This is a difficult thing, because we don't know how large a block of addresses the attacker controls. It might even be that the attacker by chance controls a handful of addresses from the same block, with the in-between addresses being legitimate.

Step 1. Get CIDR of hosts

The log files that fail2ban monitors typically show hosts (e.g. 127.0.0.1) instead of CIDR blocks (127.0.0.0/24) or IP ranges (127.0.0.0 - 127.0.0.255).

A solution could be to first assume a small CIDR block and then grow it as logs report more misbehaving hosts. Obviously it should only grow the CIDR, if those hosts are from adjacent addresses. But this is complex and legitimate addresses might get caught up regardless of the sophistication of the algorithm.

Instead, we can also simply lookup the CIDR in whois. This incurs some latency for the whois lookup, and generates some traffic. But the script that parses whois, can just write the CIDR to syslog, which can then be caught by fail2ban again.

Note: don't forget to hook this script into the actionban of your preferred action.d/lorem-ipsum.conf script. Be aware that if its maxretry > 1, then you will not catch CIDR blocks where hosts only fail once!

#!/usr/bin/python

import sys, subprocess, ipaddress, syslog

def narrowest_cidr(cidrA, cidrB):
    _ip, subA = cidrA.split('/')
    _ip, subB = cidrB.split('/')

    if int(subA) > int(subB):
        return cidrA
    else:
        return cidrB

bad_ip = sys.argv[1]
cidrs = []
inetnums = []
ret = None

whois = subprocess.run(['whois', bad_ip], text=True,
        stdout=subprocess.PIPE, check=True)

for line in whois.stdout.split('\n'):
    if 'CIDR:' in line:
        cidrs.append(line.replace('CIDR:', '').strip())
    if 'inetnum:' in line:
        inetnums.append(line.replace('inetnum:', '').strip())

if len(cidrs) >= 1:
    if len(cidrs) == 1:
        cidr = cidrs[0]
    else:
        cidr = narrowest_cidr(cidrs[0], cidrs[-1])
elif len(inetnums) > 0:
    if len(inetnums) == 1:
        inetnum = inetnums[0]
        startip, endip = inetnum.split(' - ')
        cidrs = [ipaddr for ipaddr in ipaddress.summarize_address_range(ipaddress.IPv4Address(startip), ipaddress.IPv4Address(endip))]
        if len(cidrs) == 1:
            cidr = cidrs[0]
        else:
            cidr = narrowest_cidr(cidrs[0], cidrs[-1])
else:
    cidr = "no CIDR found"

syslog.openlog(ident="suspectrange")
syslog.syslog("{} has CIDR {}".format(bad_ip, cidr))

Step 2. Figure out when to block a CIDR

If we had the dynamic CIDR determination this might get a little elaborate, as we'd have to change what we're banning. But with the whois lookup we can simply ban the CIDR block we found, based on a maxretry and findtime that we deem appropriate. Here's the jail I use:

[fail2ban-cidr-recidive]
filter = fail2ban-cidr-recidive
action = nftables-common[name=BADRANGE]
logpath = /var/log/everything/current

#5 bans in 12hrs is 48hr ban
maxretry = 5
findtime = 12h
bantime = 2d

And accompanying filter

[Definition]

failregex = \[suspectrange\] .* has CIDR <SUBNET>

Step 3. Actually block the CIDR

As you may have noticed, I use action.d/nft-common.conf. nftables allows blocking CIDR blocks instead of single hosts. This requires a small change to the first line of the actionstart part of the action script:

actionstart = nft add set netdev f2b <set_name> \{ type ipv4_addr\; \}

Should be modified to:

actionstart = nft add set netdev f2b <set_name> \{ type ipv4_addr\; flags interval\; \}

fail2ban can be configured for permanent bans by setting bantine to -1

In jail.conf

bantime = -1 

These will be lost on a reboot, but that's not necessarily a bad thing because so many attempts will be transient from pwned home machines in a botnet...

If you want persistence, then https://arno0x0x.wordpress.com/2015/12/30/fail2ban-permanent-persistent-bans/ may give some guidance.

Essentially modifying the fail2ban config to create a persistent configuration file of all the banned IPs, and have iptables load this list on reboot...

So if you check your default jail.conf you may find the default action is iptables-multiport. This corresponds to the configuration file /etc/fail2ban/ction.d/iptables-multiport.conf

We can add the following entries:

[Definition]
# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart = iptables -N fail2ban-<name>
              iptables -A fail2ban-<name> -j RETURN
              iptables -I <chain> -p <protocol> -m multiport --dports <port> -j fail2ban-<name>
          cat /etc/fail2ban/persistent.bans | awk '/^fail2ban-<name>/ {print $2}' \
          | while read IP; do iptables -I fail2ban-<name> 1 -s $IP -j <blocktype>; done

# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop = iptables -D <chain> -p <protocol> -m multiport --dports <port> -j fail2ban-<name>
             iptables -F fail2ban-<name>
             iptables -X fail2ban-<name>

# Option:  actioncheck
# Notes.:  command executed once before each actionban command
# Values:  CMD
#
actioncheck = iptables -n -L <chain> | grep -q 'fail2ban-<name>[ \t]'

# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    See jail.conf(5) man page
# Values:  CMD
#
actionban = iptables -I fail2ban-<name> 1 -s <ip> -j <blocktype>
        echo "fail2ban-<name> <ip>" >> /etc/fail2ban/persistent.bans

Now, when fail2ban flags an entry it will add a line to /etc/fail2ban/persistent.bans (via the actionban configuration). When fail2ban starts up it calls actionstart which reads this file and builds the iptables rules necessary.

Of course, fail2ban needs restarting after any of the configuration files are changed.

All credit to "arno0x0x" and his wordpress site for this recipe.