Howto : Guide to SNORT IDS in Debian based system with Barnyard2, Mysql and Pullpork

Last Update: 01/07/2013 
Major changes: 
01/07/2013 - Ignoring local traffic
11/11/2012 - Initial Version



If you are interested in advanced Opensource security tools, you probably already know about Snort Intrusion Prevention System, and if you don't let's follow my guide to help easily getting a full working Snort installation running in Intrusion Prevention System mode.

I will begin with some explanations:

- Snort can operate in various mode, as a Network sniffer, as an Intrusion Detection System (IDS) and as an Intrusion Prevention System (IPS)

- Snort IDS is Opensource but rules are not (anymore) and can be delivered in 2 ways by Sourcefire :

* For free if you register to Snort, you will get 1 month old rules version and an Oinckcode if you want to use third party tools to automatically update rules

* With charge if you want to get a subscription and get always up to date verified rules

- Snort does not support anymore database logging, all activity will logged to files, so you need a third party tool to insert Snort events into a database (Mysql recommended) : Barnyard2

- Snort does not offer itself any frontend or GUI, you will need a third party tool : most known "Base" (Basic And Security Engine)

- Snort won't update itself its detection rules, using the 3rd party tool "Pulledpork" will automatically update your rules. (you need to register on Snort site and generate an Oinckcode, see later)


My guide is based on following documentation, with some adaptations:
http://www.snort.org/assets/167/deb_snort_howto.pdf
http://www.snort.org/assets/158/snortinstallguide293.pdf


I will assume you already have a working Linux installation, which is really far way from the objective of this guide :-)



Summary of steps: 

Step 1: Prepare your system            
            1.1: Install Apache2, Mysql and system dependencies

            1.2: Install Snort dependencies (libpcap, libdnet, daq)

            1.3: Update Library Path

Step 2: Install and configure Snort

            2.1: Installation

            2.2: Test Snort

Step 3: Install Barnyards

Step 4: Setup Mysql Databases

            4.1: Connection to Mysql Server

            4.2: Databases and Privileges creation

            4.3: Populate database with Snort Structure

            4.4: Launch Snort and Barnyard for second testing
            4.5: OPTIONAL - Ignoring local Traffic

Step 5: Init script

Step 6: Cleanup and start services

Step 7: Install Pullpork and update your rules

            7.1: Register to Snort and get your Oinckcode

            7.2: Install and configure Pullpork

            7.3: Install Pullpork and update your rules

            7.4: Plan auto updates using cron

Step 8: Configure Apache2 and BASE

            8.1: Configure php.ini

            8.2: Install Perl requirements

            8.3: Activate Apache SSL

            8.4: Install and configure BASE

Step 9: Recommended alternative Frontend for Snort: Splunk for Snort
Step 10: Testing Snort IDS

Step 1: Prepare your system

The ideal configuration would be to have 2 networking interfaces, one used as the management interface and an other that will act as the collector interface. (eg. eth0: Management Int, eth1: Collector Int)

Therefore you can off course install and use Snort with only one networking interface system that will be used for both normal networking traffic and data collecting.

I will assume you only have one Networking Interface for all traffic, but this can be easily adapted to your needs.


Step 1.1: Configure your Network Interfaces

First, ensure your Network IP Address are statically fixed, or reserved in your DHCP Server.

Disable ”Large Receive Offload” and ”Generic Receive Offload” on the collector interface

# ethtool -K eth0 gro off
# ethtool -K eth0 lro off

Explanation from Snort guide, chapter 1.5:

Some network cards have features named ”Large Receive Offload” (lro) and ”Generic Receieve Offload”
(gro). With these features enabled, the network card performs packet reassembly before they’re processed by
the kernel.
By default, Snort will truncate packets larger than the default snaplen of 1518 bytes. In addition, LRO and
GRO may cause issues with Stream5 target-based reassembly. We recommend that you turn off LRO and
GRO. On linux systems, you can run:
$ ethtool -K eth1 gro off
$ ethtool -K eth1 lro off

Step 1.1: Install Apache2, Mysql and system dependencies

Update:

As always, update apt:

# sudo apt-get update

Install Apache2 Web Server:

If you don't already have Apache2 installed and running:

# sudo apt-get install apache2 apache2-utils apache2.2-bin apache2.2-common libapache2-mod-php5

Install Mysql server:

Your Mysql server can be local or remote.

Local Mysql-server installation:

# sudo apt-get install mysql-client mysql-common mysql-server

Note: Installation wizard will ask to enter your mysql-server admin password

In case of remote mysql-server, just install mysql client and some dependencies:

# sudo apt-get install mysql-client

Install Dependencies:

Install various dependencies:

# sudo apt-get install g++ make autoconf automake libtool flex bison gcc libnet1 libnet1-dev libapache2-mod-php5 libcrypt-ssleay-perl libpcre3 libpcre3-dev libmysqlclient-dev libphp-adodb libssl-dev libtool libwww-perl libmysqlclient-dev mysql-common mysql-client ntp php5-cli php5-gd php5-mysql php-pear

Step 1.2: Install Snort dependencies : (libpcap, libdnet, daq)

Install libpcap: 

# cd /usr/src && wget http://www.tcpdump.org/release/libpcap-1.3.0.tar.gz 
# tar -zxf  libpcap-1.3.0.tar.gz && cd libpcap-1.3.0 
# ./configure --prefix=/usr --enable-shared && make && make install

Install libdnet: 

# cd /usr/src && wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz 
# tar -zxf libdnet-1.12.tgz && cd libdnet-1.12 
# ./configure --prefix=/usr --enable-shared && make && make install

Install libdnet: 

# cd /usr/src && wget http://www.snort.org/dl/snort-current/daq-1.1.1.tar.gz 
# tar -zxf daq-1.1.1.tar.gz && cd daq-1.1.1 
# ./configure && make && make install

Step 1.3: Update library path

Do:

# echo >> /etc/ld.so.conf /usr/lib 
# echo >> /etc/ld.so.conf /usr/local/lib && ldconfig 

Step 2: Install Snort


Step 2.1: Installation

Download, configure compile and install Snort:

# cd /usr/src && wget http://labs.snort.org/snort/2931/snort.conf -O snort.conf
# wget http://www.snort.org/dl/snort-current/snort-2.9.3.1.tar.gz -O snort-2.9.3.1.tar.gz
# tar -zxf snort-2.9.3.1.tar.gz && cd snort-2.9.3.1
# ./configure --enable-sourcefire && make && make install
# mkdir /etc/snort /etc/snort/rules /var/log/snort /var/log/barnyard2 /usr/local/lib/snort_dynamicrules
# touch /etc/snort/rules/white_list.rules /etc/snort/rules/black_list.rules
# groupadd snort && useradd -g snort snort
# chown snort:snort /var/log/snort /var/log/barnyard2
# cp /usr/src/snort-2.9.3.1/etc/*.conf* /etc/snort
# cp /usr/src/snort-2.9.3.1/etc/*.map /etc/snort
# cp /usr/src/snort.conf /etc/snort

Edit "/etc/snort/snort.conf": (using vi to search for a line : esc then ":N" where N is the line number)

Line #45 - ipvar HOME_NET 172.26.12.0/22 – make this match your internal (friendly) network
Line #48 - ipvar EXTERNAL_NET !$HOME_NET
Line #104 - var RULE_PATH ./rules
Line #113 - var WHITE_LIST_PATH ./rules
Line #114 - var BLACK_LIST_PATH ./rules
Line #297 - add this to the end after “decompress_depth 65535” max_gzip_mem 104857600
Line #538 - add this line output unified2: filename snort.log, limit 128
Line #554 - delete or comment out all of the “include $RULE_PATH” lines except “local.rules”

Step 2.2: First test of Snort

Temporarily and for testing purposes, add this to "/etc/snort/rules/local.rules" 

alert icmp any any -> $HOME_NET any (msg:"ICMP test"; sid:10000001;)

This will be a local testing rule using ICMP request, when running Snort in console and pinging our host from another computer, Snort shall detect it:

Start Snort:

# /usr/local/bin/snort -A console -q -u snort -g snort -c /etc/snort/snort.conf -i eth0

Ping your Snort host, you should see Snort logging it:

02/09-11:29:43.450236 [**] [1:10000001:0] ICMP test [**] [Priority: 0] {ICMP} 172.26.12.1 -> 172.26.12.2
02/09-11:29:43.450251 [**] [1:10000001:0] ICMP test [**] [Priority: 0] {ICMP} 172.26.12.2 -> 172.26.12.1

If you see this kind of output, then your Snort installation is functional ^^

If not, you may have a configuration issue, stop all and carefully restart from the beginning!

Enter crtl+c to stop Snort.



Step 3: Install Barnyard2


Download, configure, compile and install:

# cd /usr/src && wget https://nodeload.github.com/firnsy/barnyard2/tarball/master
# tar -zxf master && cd firnsy-barnyard2-*
autoreconf -fvi -I ./m4 && ./configure --with-mysql && make && make install
# mv /usr/local/etc/barnyard2.conf /etc/snort
# cp schemas/create_mysql /usr/src

Edit "/etc/snort/barnyard2.conf":

Line #215 change to output alert_fast

At the end of the file, add the Database configuration line, adapt:

- "<mypassword>" to the value of the database snort password you want to set (not the admin password database, but the password you are planing to use for Snort database)

- "localhost" : If your Mysql-server will local, then let localhost, if your Mysql server is remote then enter its IP Address

output database: log, mysql, user=snort password=<mypassword> dbname=snort host=localhost

Step 4: Setup Mysql Databases


Here comes the Mysql configuration, we will:

- Create a main database called "snort" where Snorts events are going to be logged by the barnyards daemon
- Create a secondary database called "archive" to rotate Snort events
- Create a user "snort" with all privileges to these databases


Step 4.1: Connect to your Mysql server

If Mysql-server local, enter:

# mysql -u root -p

If Mysql-server remote, enter: (adapt IPAddress)

# mysql -u root -p -h IPAddress

Then enter your admin password, you should get the mysql prompt:

mysql>

Step 4.2: Create Databases and privileges

Create Databases:

mysql>

create database snort;
create database archive;

Create Snort user and privileges: (adapt <mypassword> to the Snort database user password value of your choice)


If Mysql-server local: (adapt IPAddress with Mysql Server value)

mysql>

grant usage on snort.* to snort@192.168.1.104;
grant usage on archive.* to snort@192.168.1.104;
set password for snort@192.168.1.104=PASSWORD('snort');
grant all privileges on snort.* to snort@192.168.1.104;
grant all privileges on archive.* to snort@192.168.1.104;
flush privileges;

If Mysql-server Remote: (adapt IPAddress with Mysql Server value)

mysql>

grant usage on snort.* to snort@IPAddress;
grant usage on archive.* to snort@IPAddress;
set password for snort@IPAddress=PASSWORD('<mypassword>');
grant all privileges on snort.* to snort@IPAddress;
grant all privileges on archive.* to snort@IPAddress;
flush privileges;

Step 4.3: Populate Database with Snort structure

Do:

mysql>

use snort;
source /usr/src/create_mysql
show tables; # you should see the list of new tables you just imported.
exit

Step 4.4: Launch Snort and Barnyes for second testing purposes

Launch Snort and Barnyard :

Do:

# /usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 &

# /usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.log -w /etc/snort/bylog.waldo -G /etc/snort/gen-msg.map -S
/etc/snort/sid-msg.map -C /etc/snort/classification.config &

If Barnyard successfully connects to the Snort database, you should get such an output:

database: compiled support for (mysql)                                                                                        
database: configured to use mysql                                                                                             
database: schema version = 107                                                                                                
database:           host = 192.168.1.100                                                                                      
database:           user = snort                                                                                              
database:  database name = snort                                                                                              
database:    sensor name = raspberrypi:NULL                                                                                   
database:      sensor id = 1                                                                                                  
database:     sensor cid = 12275                                                                                              
database:  data encoding = hex                                                                                                
database:   detail level = full                                                                                               
database:     ignore_bpf = no                                                                                                 
database: using the "log" facility                                                                                            
                                                                                                                              
        --== Initialization Complete ==--                                                                                     
                                                                                                                              
  ______   -*> Barnyard2 <*-                                                                                                  
 / ,,_  \  Version 2.1.10 (Build 313)                                                                                         
 |o"  )~|  By Ian Firns (SecurixLive): http://www.securixlive.com/                                                            
 + '''' +  (C) Copyright 2008-2012 Ian Firns <firnsy@securixlive.com>                                                         
                                                                                                                              
Using waldo file '/etc/snort/bylog.waldo':                                                                                    
    spool directory = /var/log/snort                                                                                          
    spool filebase  = snort.log                                                                                               
    time_stamp      = 1352226216                                                                                              
    record_idx      = 24326                                                                                                   
Opened spool file '/var/log/snort/snort.log.1352226216'                                                                       
Closing spool file '/var/log/snort/snort.log.1352226216'. Read 24326 records                                                  
Opened spool file '/var/log/snort/snort.log.1352301110'                                                                       
Waiting for new data

Any error preventing Barnyard2 from connecting to the Mysql Database will be shown in ouput messages (bad username/password, host unreacheable, etc...)

If successful, then congratulation you have a working Snort and Barnyard2 installation ^^

Let's continue!


Step 4.5: OPTIONAL - Ignoring local Traffic


In most situations you won't be really interested in monitoring your local networking traffic as you can consider it as a trust zone.

If so, you can easily ask Snort to ignore any local traffic:


Edit "/etc/snort/snort.conf", search for "config bpf_file", comment it out and change to

config bpf_file: /etc/snort/ignore.bpf

Create the file "/etc/snort/ignore.bpf" with the following content: (adapt to your local Network CIDR address)

!(src net 192.168.100.0/24)

And save, at next reboot any local traffic will be ignored.

This configuration may also be more complex to ignore different Networks, Hosts and even specific ports.
Google is your friend ^^

Let's continue!




Step 5: Init script


Add and enable an init script for Snort and Barnyard2.

NOTE: Change "eth0" to your interface name if your collector Interface is different from eth0.

Create the new file "/etc/init.d/snortbarn" and add this content:

#! /bin/sh
#set -x
#
### BEGIN INIT INFO
# Provides: snortbarn
# Required-Start: $remote_fs $syslog mysql
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# X-Interactive: true
# Short-Description: Start Snort and Barnyard
### END INIT INFO
. /lib/init/vars.sh
. /lib/lsb/init-functions
do_start()
{
log_daemon_msg "Starting Snort and Barnyard" ""
/sbin/ifconfig eth0 up
/usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 &
/usr/local/bin/barnyard2 -q -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.log -w /etc/snort/bylog.waldo -G /etc/snort/gen-msg.map –S /etc/snort/sid-msg.map -C /etc/snort/classification.config 2> /dev/null &
log_end_msg 0
return 0
}
do_stop()
{
log_daemon_msg "Stopping Snort and Barnyard" ""
kill $(pidof snort) 2> /dev/null
kill $(pidof barnyard2) 2> /dev/null
log_end_msg 0
return 0
}
case "$1" in
start)
do_start
;;
stop)
do_stop
;;
restart)
do_stop
do_start
;;
*)
echo "Usage: snort-barn {start|stop|restart}" >&2
exit 3
;;
esac
exit 0

Authorisation:

# chmod +x /etc/init.d/snortbarn

Activate:

# update-rc.d snortbarn defaults

Step 6: Cleanup and start services


Stop existing processes and clean up various files:

# rm /var/www/index.html
# chmod 755 /var/www/base
# pkill snort && pkill barnyard2
# rm -rf /var/log/snort/* /var/log/barnyard2/*

Deactivate the local test rules and activate standard rules:

Edit "etc/snort/rules/local.rules" and comment out the test rule line
Edit "/etc/snort/snort.conf" – Line 553: add: include $RULE_PATH/snort.rules

Start Snort and Barnyard2:

# sudo service snortbarn start

Step 7: Install Pullpork and update your rules


Step 7.1: Register to Snort and get your Oinckcode

First, go to www.snort.org and register (for free).

When your accound had been created, go in:

- "My Account"
- Then "Subscriptions and Oinckcodes"
- "Oinkcodes"

And generate your Oinckcode, copy it somewhere it will be required to set up Pulleporked.


Step 7.2: Install and configure Pulledporked

Do:

# cd /usr/src && wget http://pulledpork.googlecode.com/files/pulledpork-0.6.1.tar.gz
# tar -zxf pulledpork-0.6.1.tar.gz && cd pulledpork-0.6.1
# cp pulledpork.pl /usr/local/bin && cp etc/*.conf /etc/snort

Edit "/etc/snort/pulledpork.conf":

Comment out lines 22 & 26

Line 20: enter your “oinkcode” where appropriate or comment out the line if you didn’t get one above
Line 23: leave alone (uncommented) to use the Emerging Threats rule set
Line 71: change to: rule_path=/etc/snort/rules/snort.rules
Line 86: change to: local_rules =/etc/snort/rules/local.rules
Line 89: change to: sid_msg=/etc/snort/sid-msg.map
Line 112: change to: config_path=/etc/snort/snort.conf

Line 124: change to: distro=Debian-Lenny

Line 171: Uncomment and change to: enablesid=/etc/snort/enablesid.conf
Line 173: Uncomment and change to: disablesid=/etc/snort/disablesid.conf
Line 174: Uncomment and change to: modifysid=/etc/snort/modifysid.conf

Disable Blocking rules:

Do:

# echo pcre:fwsam >> /etc/snort/disablesid.conf # disables all block (fwsam) rules

Step 7.3: Start Pulledporked and update your rules set

Do:

# /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -T -l

If everything is fine, you should get this kind of output:

# /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -T -l                                          
                                                                                                                              
    http://code.google.com/p/pulledpork/                                                                                      
      _____ ____                                                                                                              
     `----,\    )                                                                                                             
      `--==\\  /    PulledPork v0.6.1 the Smoking Pig <////~                                                                  
       `--==\\/                                                                                                               
     .-~~~~-.Y|\\_  Copyright (C) 2009-2011 JJ Cummings                                                                       
  @_/        /  66\_  cummingsj@gmail.com                                                                                     
    |    \   \   _(")                                                                                                         
     \   /-| ||'--'  Rules give me wings!                                                                                     
      \_\  \_\\                                                                                                               
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                               
                                                                                                                              
Checking latest MD5 for snortrules-snapshot-2931.tar.gz....                                                                   
        They Match                                                                                                            
        Done!                                                                                                                 
Prepping rules from snortrules-snapshot-2931.tar.gz for work....                                                              
        Done!
Reading rules...                                                                                                              
Reading rules...                                                                                                              
Processing /etc/snort/enablesid.conf....                                                                                      
        Modified 0 rules                                                                                                      
        Done                                                                                                                  
Processing /etc/snort/disablesid.conf....                                                                                     
        Modified 0 rules
        Done                                                                                                                  
Setting Flowbit State....                                                                                                     
        Enabled 10 flowbits                                                                                                   
        Enabled 1 flowbits
        Done
Writing /etc/snort/rules/snort.rules....                                                                                      
        Done
Generating sid-msg.map....                                                                                                    
        Done
Writing sid_msg....                                                                                                           
        Done
Writing /var/log/sid_changes.log....                                                                                          
        Done                                                                                                                  
Rule Stats....                                                                                                                
        New:-------0                                                                                                          
        Deleted:---0                                                                                                          
        Enabled Rules:----3511                                                                                                
        Dropped Rules:----0                                                                                                   
        Disabled Rules:---11787                                                                                               
        Total Rules:------15298                                                                                               
        Done                                                                                                                  
Please review /var/log/sid_changes.log for additional details                                                                 
Fly Piggy Fly!

Step 7.4:Plan auto updates using cron

Edit your crontab and add a line for Pullpork, example with an auto update every Sunday at 2.30 AM:

Edit your crontab:

# crontab -e

Add:

30 2 0 * * /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -T -l >/var/log/snort/pulledpork_crontab.log

The auto update result will be available in the log file.



Step 8: Configure Apache2 and install BASE

To use BASE as a Snort front-end which is Web GUI you off course need a running Web Server, as installed before we will use Apache2.

Configuring Apache is far away from this guide objective, the default configuration will allow us to access to BASE with no issue.


Step 8.1: Configure php.in

Edit "/etc/php5/apache2/php.ini", look for the line "error_reporting" and change it to:

error_reporting = E_ALL & ~E_NOTICE

Step 8.2: Install Perl requirements

Do:

# pear config-set preferred_state alpha && pear channel-update pear.php.net && pear install --alldeps Image_Color Image_Canvas Image_Graph

Step 8.3: If you are using Apache2 default configuration, activate SSL

Do:

# a2enmod ssl
# service apache2 restart

Step 8.4: Install and configure BASE

Do:

# cd /usr/src && wget http://sourceforge.net/projects/secureideas/files/BASE/base-1.4.5/base-1.4.5.tar.gz
# tar -zxf base-1.4.5.tar.gz && cp -r base-1.4.5 /var/www/base
# chown -R www-data:www-data /var/www/base

Open your Web Browser and navigate to "https://IPAddress/base", you should get the BASE configuration page.

Do as follows:

Path to adodb: /usr/share/php/adodb
Click Continue
Database Name: snort
Database Host: localhost or your Mysql Server IP if remote
Database Port: leave blank
Database User Name: snort
Database Password: <mypassword>

Activate Database Archive and configure as above. (database name "archive")

Put in values for the authentication system and click submit.
Click "create baseag" which extends the DB to support BASE.


And you should get access to BASE ^^

Step 9: Recommended alternative frontend for Snort

BASE is quite limited and obsolete, as a Snort Frontend i personally recommend the Snort plugin for Splunk.

Splunk is an extremely powerful knowledge tool that will aggregate and manipulate any data, an application is available for Snort and provides a very great frontend for Snort!

A full guide coming soon :-) 



Step 10: Testing SNORT IDS

Here are some tools i recommend:

1. Nessus

Nessus is a well known security scan tool, you can get for a free a personal user key and use the software to scan your sites or hosts.

See:
http://www.tenable.com/


Installation is really simple, then simply launch a Web Application scan and wait for the report, this should generate a lot of events in Snort!

2. Nikto

Also very powerful and simple to use, Nikto will carefully scan and launch various attacks/exploit against your Web Server.

See:
http://cirt.net/

Example of use:

Launching a scan in http:

perl nikto.pl -h HOST -p PORT

Launching a scan in https:

perl nikto.pl -h HOST -p PORT -ssl

Testing Web Server with Nikto should also generate new events in Snort.