DAVBlack -- Automatically Blacklist Website Attackers

DAVBlack -- Automatically BLACKLIST Web Site Attackers

I believe this script has been SIGNIFICANTLY deprecated by the ModSecurity addition to Apache. I leave it up here for the historical value and for those people who are not using Apache. If you still need something like this, please look at sshblack as it is very similar and much more up to date than this code.

This is Version 2.0 of the popular dav-black script. It provides variable aggressiveness by allowing the administrator to adjust the number of 'hits' required to make the blacklisting fire. Additionally, it will remove addresses from the blacklist after a user-defined time period. The problem of duplicate table listings in the old version is also fixed...I think.

DAVBlack is based on the work of Julian Haight. His mailmgr program was a bit more than I needed. Yet I liked the idea of tailing a log file.

After I noticed a large quantity of "Microsoft-WebDAV-MiniRedir/" entries in my Apache logs I started Googling around to find out what these were. All indications seem to point to these being harmless and possibly unintentional "attacks." However, I was tired of seeing them in my logs. I would get hundreds of these entries a day -- particularly on my DSL line -- and it made my log difficult to read.

Ideally these could have been handled with blockme.cgi except all the entries were using HTTP methods of OPTIONS and PROPFIND instead of GET. Additionally, there are indications that certain blacklists like SORBS may list you as an "exploitable" web server if you respond to certain attacks with an HTTP response code of 200 (which blockme.cgi does). Certainly I could have had Apache block these methods but this would not have stopped the "attacking" sites from continually showing up in my logs. They needed to be blacklisted.

You will note that this also will blacklist people attempting IIS exploits such as default.ida, msdac, and fp30reg.dll. Obviously this code is intended for people running Apache or a web server other than Microsoft IIS. If you actually have a legitimate "scripts" directory, you will need to modify this code. Let me state that again: You need to look at the line of $REASONS in the beginning of the code and make sure you are not going to blacklist people for incorrect reasons.
Also note that this is a fairly aggressive posture. If you are not prepared to monitor your logs and keep an eye on things, you may end up blacklisting someone you shouldn't have. Our experience shows that unintentional blacklisting is highly unlikely in the real world, probably something you shouldn't lose any sleep over. Just as long as you understand what is going on here. You also need to think about the effect on dynamic IP addresses such as dial-up, cable and ADSL.
I store the database in /var/tmp/ but you should know that this is not a protected directory. If you don't want people to see you pending/active blacklist, put the database somewhere else.
See the first few lines of code for user-defined variables.
The version shown here can not be set lower than two hits (because of my poor coding). I have a one-hit version that will only work for one-hit applications. It is available here. It still uses the database concept but it fires on the first illegal string instead of waiting for MAXHITS.
Note that this code is very similar to SSHBlack. It is included here for completeness.

The Code [Click to download text version or copy/paste from below]

#!/usr/bin/perl -w
#
# dav-black.pl Version 2.0
#
# based on mailmgr (c) 2003, Julian Haight, All Rights reserved under GPL license:
# http://www.gnu.org/licenses/gpl.txt
# Modified on 02AUG04 as provided under GNU GPL licensing
# no rights reserved under this modification
#
#  See http://www.pettingers.org/code/davblack.html for details on this
#
# This is a script which tails web log files and dynamically blocks
# connections from hosts which meet certain criteria, using
# command-line kernel-level firewall configuration tools provided by
# underlying operating system (iptables)
# As the script is modifying iptables, it will need root access to do so.
#
# Note: this script can also be modified to monitor ANY log file
# including access (secure) logs and sendmail (mail) logs.  The
# aggressiveness can be adjusted by setting the variables in the
# first few lines.  It will probably work well right out of the box.
# Modifications will also be required for use of ipchains instead of
# iptables.
#
# Setup:  You need to create the initial chain that dav-black will work with.
# For iptables, you would do this:
# iptables -N WEB ##      Create a new chain called WEB
# Then you would do this:
# iptables -A INPUT -p tcp -m tcp --dport 80 --syn -j WEB
##      Send all TCP port 80 packets through the chain.  We will be adding
##      REJECT jumps to this chain with the program below.
#
# The easiest way to run the script is in the background with a shell script,
# you can then put the shell script into /etc/rc.d/rc.local to run at start-up.
# Something like:
#    #! /bin/bash
#    /root/utils/dav-black.pl >>/var/log/web-blacklisting 2>&1 &
#
# This will create a nice log of your activities at /var/log/web-blacklisting

##############################################################################

use strict;
use Socket;
# The log file you want to monitor
my($LOG) = '/var/log/httpd/access_log';
# The cache file to keep track of attackers
my($CACHE) = '/var/tmp/web-blacklist-pending';

# regex for whitelisted IPs - never blacklist these addresses
my($LOCALNET) = '^(?:127\.0\.0\.1|192\.168\.0)';
# your kernel-firewall, see "man iptables" to redefine params used
my($IPTABLES) = '/sbin/iptables';
my($ADDRULE) = '-I'; # cmdline for insert rule
my($DELRULE) = '-D'; # cmdline for delete rule

# Regex of reasons to get firewalled. Separate with pipe (|).
# This VARIES BASED ON THE VERSION OF SOFTWARE YOU ARE RUNNING
# Look at your logs and adjust as necessary.
#
my($REASONS) = '(OPTIONS|PROPFIND|CONNECT|SEARCH|fp30reg|tickerbar|scripts|vti_bin|mem_bin|\
			|system32|msadc|sumthin|default.ida|prxjdg|userstat.pl|NULL.printer|formmail)';
#
# Maximum time (sec) before they are removed from the database
# unless they are already blacklisted
my($AGEOUT) = 86400;
# Time delay (day) before they are released from the blacklist in DAYS!
my($RELEASEDAYS) = 3;
# Time dealy (sec) to check the database for cleanup
my($CHECK) = 600;
# Maximum number of booboos before they get listed
my($MAXHITS) = 2;
#
#
#
########### No user defined paramters below ################
#
my($OCT) = '(?:25[012345]|2[0-4]\d|1?\d\d?)';
my($IP) = $OCT . '\.' . $OCT . '\.' . $OCT . '\.' . $OCT;

$RELEASEDAYS *= 86400; # Lots of seconds!
# $RELEASEDAYS = 130;     #For testing

print "\nInitializing...\n";

# Poor man's touch command
open (TOUCH, ">> $CACHE"); close (TOUCH);

# Start the monitoring
taillog();

sub taillog {
   my($offset, $name, $line, $ip, $reason, $stall, $ind) = '';
   my (@loser, @buildlist) = ();

   $offset = (-s $LOG); # Don't start at begining, go to end

   while (1==1) {
       sleep(1);
       $| = 1;
       $stall += 1;
       if ((-s $LOG) < $offset) {
           print "Log shrunk, resetting..\n";
           $offset = 0;
       }
       open(TAIL, $LOG) || print STDERR "Error opening $LOG: $!\n";

        if (seek(TAIL, $offset, 0)) {
           # found offset, log not rotated
       } else {
           # log reset, follow
           $offset=0;
           seek(TAIL, $offset, 0);
       }
       while ($line = <TAIL>) {
           chop($line);
           if (($REASONS) && ($line =~ m/$REASONS/)) {
               $reason = $1;
               if ($line =~ m/($IP)/) {
                 $ip = $1;

                 open(LIST, $CACHE) || print STDERR "Error opening $CACHE: $!\n";
                 $ind = 0;
                 @buildlist = <LIST>;
                 foreach $line(@buildlist) {
                   @loser = split(/,/, $line);
                   # [0] is IP, [1] is time, [2] is hits
                   if ($loser[0] eq $ip) {
                     # Already listed, increase count
                     $loser[2] += 1;

                     if ($loser[2] == $MAXHITS) {
                     # See ya!
                       print "$ip being blocked because of $reason, \n";
                       blockIp($ip);
                       print "     Killed ", scalar localtime, "\n";
                       print "-------------------------------------------\n";
                       $loser[2] += 1; # Avoid double listings (???)
                     }
                     $line = join(',', @loser); # put back together for saving
                     $line .= "\n";
                     $buildlist[$ind] = $line;
                     $ip = 'logged';
                   } # End if already listed
                   $ind += 1;
                 } # End foreach read
                 close (LIST);
                 if ($ip ne 'logged') {
                    $line = $ip . ',' . time() . ',' . 1 . "\n";
#                    $line = join(',', $ip time() '1' "\n");
                    push (@buildlist, $line);
                 }

                 open (LIST, ">$CACHE") || print STDERR "Error opening $CACHE: $!\n";
                 print LIST @buildlist;
                 close (LIST)
               } # End if IP
               next;
           } # End if match reasons
       } # End while read line
       $offset=tell(TAIL);
       close(TAIL);

       if ($stall >= $CHECK) {
        # Time to do cleanup
        $stall = 0;
        @buildlist = ();
        open(LIST, $CACHE) || print STDERR "Error opening $CACHE: $!\n";
        while ($line = <LIST>) {
           @loser = split(/,/, $line);
           # [0] is IP, [1] is time, [2] is hits
           if ($loser[2] >= $MAXHITS) {
                # already blacklisted
                if (($loser[1] + $RELEASEDAYS) > time()) {
                   push (@buildlist, $line);
                }
                else {
                   iptables($DELRULE, $loser[0]);
                   print "Freeing $loser[0]", " on ", scalar localtime, "\n";
                   print "-------------------------------------------\n";
                } #set free after $RELEASEDAYS

           }
           elsif (($loser[1] + $AGEOUT) > time()){
                # Not listed and not aged out
                   push (@buildlist, $line);
           }
        } # End while reading
        close (LIST);
        # open for writing
        open (LIST, ">$CACHE") || print STDERR "Error opening $CACHE: $!\n";
        print LIST @buildlist;
        close (LIST);
        @buildlist = ();
       } # End cleanup check

   } # End while endless loop
} # End sub taillog


sub blockIp {
   my($ip) = @_;
   if ($ip =~ m/$LOCALNET/) {
   print "WHITELISTED HOST - NOT BLOCKING \n";
       return;
   }

   iptables($ADDRULE, $ip);
   return;

} # End sub blockIp

sub iptables {
   my($action, $ip) = @_;
   my(@args) = ($IPTABLES,
                $action,
                'WEB', '--source',
                $ip,
               '-j', 'REJECT', '--reject-with', 'icmp-host-prohibited');
   system(@args);
   return;

} # End sub iptables

Prerequisites

iptables. For those of you with older kernels, you can easily modify it to work with ipchains.
Existing chain in iptables for the blacklist. This is currently configured as WEB but you can name it whatever you want. Also make sure you actually run packets through the chain in your firewall config or it won't do much good!
Root (superuser) access. Required for the iptables calls and access to the logs.

Be sure to visit the Geek Page for more geeky stuff.