Code:HTTP Request Packet Inspection

The following script, http-request-filter.pl uses tcpdump for "deep packet inspection" and logs all HTTP requests.

#!/usr/bin/env perl

# http-request-filter
# Filters the output og tcpdump -A to show HTTP requests

# run as follows:
# tcpdump -i en0 -nn -A dst port http | http-request-filter

use strict;

my $url;
my $host;
my $protocol = 'http';

while (<>) {
    if (/^\d\d?:\d\d?:\d\d?\.\d+ /) {
        if ($url) {
            print "$protocol://$host$url\n";
        }
        $url = "";
        $host = "";
    }
    if (/(GET|HEAD|POST|PUT|DELETE) (\/\S*) HTTP\/1\.\d/) {
        $url = $2;
    } elsif (/^Host: *([a-zA-Z0-9_\.]+)/) {
        $host = $1;
    } elsif (/^ IP6? \d[\d:\.]+\.\d+ > (\d[\d:\.]+)\.\d+: Flags/) {
        $host = $1;
    }
}

As commented in the script, it should be run as

tcpdump -i eth0 -nn -A dst port http | http-request-filter.pl

Where eth0 is to be replaced by the active network interface (-i any might work too).

Note that this does not log any https requests, nor requests to non-http ports (webservices running on port 8080 are ignored). This script is intended to quickly check for requested URLs by scripts. If you intend to use this script for security logging, you must be nuts or a politician, or both.