Setting up a lab with ModSecurity, Apache and DVWA

• Home
• Blog
• ModSecurity Lab

Thurs 15th Aug 13

I've been meaning to build a ModSecurity lab for a while and seeing as I had some free time I decided it was about time to do it and to document it for everyone to share.

The lab I built uses an up-to-date version of ModSecurity with a rule set taken from the SpiderLabs github repo and, so there is something to attack, I've included DVWA.

This is my first install of ModSecurity so it may not be perfect but it works and is based on the guides I've read so is likely to be close to what you'd find on servers where the admin isn't an expert and just went for a default install.

The lab is built on a Debian testing (jessie) server, I initially went for stable (wheezy) but found that the version of ModSecurity on stable is quite old and doesn't work with the current rule sets so, to play along, you need to get yourself a working jessie install. I did mine from fresh and on the software selection part of the install I only chose standard packages and ssh server, I didn't go for web server as I don't know what packages it installs and what it configures for you and I prefer to do that kind of thing myself.

After the install is complete we need to install the following basic packages:

apt-get install apache2 php5 mysql-server libapache2-modsecurity git php5-mysql

I also added vim, screen and sudo but they aren't strictly required just nice to have.

Lets start the configuration. First, install the default ModSecurity configuration file:

cd /etc/modsecurity/
cp modsecurity.conf-recommended modsecurity.conf

Next, we need the Core Rule Set (CRS). When you install the Debian package it comes with a copy of this but I've chosen to get a copy from the SpiderLabs github repository. This means it can be easily kept up-to-date with the latest greatest rules.

Still in the modsecurity directory clone the repo:

git clone https://github.com/SpiderLabs/owasp-modsecurity-crs

This now needs configuring. First, as with ModSecurity itself, the initial config file needs enabling:

cd owasp-modsecurity-crs/
cp modsecurity_crs_10_setup.conf.example modsecurity_crs_10_setup.conf

This turns the config file on but without including it in some way it isn't actually being used yet. A quick aside is needed here to explain the layout the CRS uses. The repo contains five rules directories:

• base_rules
• experimental_rules
• optional_rules
• slr_rules
• activated_rules

The first four contain the actual rules that make up the rule set, the last one, activated_rules, is where you put the rules you chose to run. Think of the way Apache on Debian runs it sites-available and sites-enabled system or how init.d works.

So, to enable the actual rules you need to symlink them from their resting places into the activated_rules directory. You also need to add the main config file.

cd activated_rules/
ln -s ../modsecurity_crs_10_setup.conf .
ln -s ../base_rules/* .
ln -s ../optional_rules/* .

Most of the examples use a for loop to do the above but I find the wildcard works fine, this is probably shell dependant though so if it fails you can use:

cd ..
for f in `ls base_rules/` ; do sudo ln -s /etc/modsecurity/owasp-modsecurity-crs/base_rules/$f activated_rules/$f ; done
for f in `ls optional_rules/ | grep comment_spam` ; do sudo ln -s /etc/modsecurity/owasp-modsecurity-crs/optional_rules/$f activated_rules/$f ; done

Either way, you should now have an activated_rules directory full of symlinks to files in the base_rules and optional_rules directories. There is nothing here stopping you from also linking in the other two rule directories but I'm following what all the guides I've read say to do so is what I think will be the configuration of most of the servers out there.

Even after doing all of this, Apache won't acknowledge any of it yet as it isn't being actively included in its configuration. To do this you need to include the active_rules directory using an Include statement.

At the moment, the ModSecurity config file /etc/apache2/mods-enabled/security2.conf contains the following line:

IncludeOptional /etc/modsecurity/*.conf

So you could just add another line in here to include the activated_rules directory but as this file is likely to be upgraded as the package is upgraded this is a bad idea. Instead I've added the include in the modsecurity.conf file we created at the start of this process.

The line needed is:

Include /etc/modsecurity/owasp-modsecurity-crs/activated_rules/*conf

While you are in this file add the following line as well:

SecDisableBackendCompression On

Without it when the log file is created, any requests which come in compressed are logged as compressed data which makes the file unreadable.

A final update is needed to Apache to enable the headers module, this allows ModSecurity to control and modify the HTTP headers for both requests and responses.

a2enmod headers

All that is left now is to restart Apache and you should have a working install.

service apache2 restart

Testing

So, lets see if all the configuration paid off, start tailing the log file:

tail -f /var/log/apache2/modsec_audit.log

Now, browse to the site by IP and you should see some entries like the following:

--d5274346-H--
Message: Warning. Pattern match "^[\\d.:]+$" at REQUEST_HEADERS:Host. [file "/etc/modsecurity/owasp-modsecurity-crs/activated_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "98"] [id "960017"] [rev "2"] [msg "Host header is a numeric IP address"] [data "192.168.0.68"] [severity "WARNING"] [ver "OWASP_CRS/2.2.8"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/IP_HOST"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [tag "http://technet.microsoft.com/en-us/magazine/2005.01.hackerbasher.aspx"]
Message: Warning. Pattern match "^(?i:0|allow)$" at RESPONSE_HEADERS. [file "/etc/modsecurity/owasp-modsecurity-crs/activated_rules/modsecurity_crs_55_application_defects.conf"] [line "151"] [id "981405"] [msg "AppDefect: X-FRAME-OPTIONS Response Header is Missing or not set to Deny."] [data "X-FRAME-OPTIONS: "] [tag "WASCTC/WASC-15"] [tag "MISCONFIGURATION"] [tag "http://websecuritytool.codeplex.com/wikipage?title=Checks#http-header-x-frame-options"]
Message: Warning. Operator LT matched 5 at TX:inbound_anomaly_score. [file "/etc/modsecurity/owasp-modsecurity-crs/activated_rules/modsecurity_crs_60_correlation.conf"] [line "33"] [id "981203"] [msg "Inbound Anomaly Score (Total Inbound Score: 3, SQLi=0, XSS=0): Host header is a numeric IP address"]
Stopwatch: 1376483251361382 39821 (- - -)
Stopwatch2: 1376483251361382 39821; combined=26192, p1=477, p2=24676, p3=85, p4=400, p5=437, sr=135, sw=117, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.4 (http://www.modsecurity.org/); OWASP_CRS/2.2.8.
Server: Apache/2.4.6 (Debian)
Engine-Mode: "DETECTION_ONLY"

--d5274346-Z--

As well as other debug information, there are three messages here, each starting with Message: . The first and last ones are commenting on the fact that the site has been accessed by IP, the middle one that there is no X-FRAME-OPTIONS value set.

If you are going to be accessing this lab through an IP most of the time then this can become annoying and clog your log files so you can disable the rules. The easiest way to do this is to simply delete them from the original rules file, the message tells you which file contains the rules:

[file "/etc/modsecurity/owasp-modsecurity-crs/activated_rules/modsecurity_crs_55_application_defects.conf"]

and which line the rule was on:

[line "151"]

so you can just go in and delete or comment out that rule. The problem with doing this is that if you then upgrade the rules through a git pull you'll not get the latest version of that file as you have modified the local copy. The better way is to create your own rules file that overrides any rules you don't like. To do this you need to know the id of the rule you want to remove, this is also contained in the log file, for the first entry it is:

[id "960017"]

To do this, we first need to create a directory to store our custom rules and a file to put them in:

cd /etc/modsecurity/owasp-modsecurity-crs
mkdir custom_rules

The name of the rules file needs to such that it is the last to be picked up in alphbetical order when it joins all the others in the activated_rules directory. Currently the last file in the directory is modsecurity_crs_60_correlation.conf so calling ours modsecurity_crs_99_custom.conf ensures there is plenty of space between it and me.

Edit the file:

vi custom_rules/modsecurity_crs_99_custom.conf

add the following line:

SecRuleRemoveById 960017

and then link it into the actived_rules directory:

cd activated_rules
ln -s ../custom_rules/modsecurity_crs_99_custom.conf .

If you now restart Apache, tail the log file and visit the page again the message should no longer appear.

I found that I had to do a full page refresh to check things were working properly as a refresh that returned a 304 sometimes didn't trip the sensors.

By adding entries to this custom file you can disable any rules which are adding chaff to the testing you are trying to do, I'll suggest a couple more after the DVWA install.

DVWA

Installing DVWA is pretty straightforward, grab the files from the github repo, set the database connection details and then browse to it and let it create the database entries it needs.

cd /var
rm -rf www/
git clone https://github.com/RandomStorm/DVWA.git www
cd www
vi config/config.inc.php

Once you've set up your credentials you can then browse to the site and you should be presented with the DVWA welcome screen telling you the database doesn't exist. Follow its walkthrough and it will set it all up for you.

Once all is set up, all you need to do is to click one of the menu links and you'll be taken to the login page. Log in with the credentials admin:password and start hacking. Watch the ModSecurity log and see how many places you would have been caught - clue, its probably a lot!

You'll probably see a lot of the following messags in the log:

Message: Warning. Pattern match "^(?i:0|allow)$" at RESPONSE_HEADERS. [file "/etc/modsecurity/owasp-modsecurity-crs-git/activated_rules/modsecurity_crs_55_application_defects.conf"] [line "151"] [id "981405"] [msg "AppDefect: X-FRAME-OPTIONS Response Header is Missing or not set to Deny."] [data "X-FRAME-OPTIONS: "] [tag "WASCTC/WASC-15"] [tag "MISCONFIGURATION"] [tag "http://websecuritytool.codeplex.com/wikipage?title=Checks#http-header-x-frame-options"]
Message: Warning. Match of "contains no-store" against "RESPONSE_HEADERS:Cache-Control" required. [file "/etc/modsecurity/owasp-modsecurity-crs/activated_rules/modsecurity_crs_55_application_defects.conf"] [line "121"] [id "900046"] [msg "AppDefect: Cache-Control Response Header Missing 'no-store' flag."] [data "Cache-Control: no-cache, must-revalidate"] [tag "WASCTC/WASC-15"] [tag "MISCONFIGURATION"] [tag "http://websecuritytool.codeplex.com/wikipage?title=Checks#http-cache-control-header-no-store"]

As with the browse by IP address detection, these are messages you don't need to worry about in a lab environment so can be disabled by adding their IDs to the custom file:

SecRuleRemoveById 900046
SecRuleRemoveById 981054

Restart Apache and that is about it, you now have a lab with a deliberately vulnerable web app but a very good web app firewall protecting it.

One last thing, by default ModSecurity just reports on issues, it doesn't block them. If you want to enable blocking then you need to edit the main config file, /etc/modsecurity/modsecurity.conf and change the SecRuleEngine entry. Valid options are On, Off and DetectionOnly (just log it). This will enable blocking of bad requests:

SecRuleEngine On

Conclusions

This blog was written as I learned the set up process myself so may not contain best practice ways of doing things and is certainly not enough to set up a production environment from but, it should give you at least a place to start from.

Happy hacking and remember, if you find a way to bypass any of the rules, the ModSecurity or SpiderLabs teams would probably be very interested to hear how you did it.

Links

The following are links which helped me put this blog post together:

• http://www.root25.com/2012/11/how-to-install-modsecurity-on-apache-ubuntu12-stepbystep-tutorial.html
• http://notepad2.blogspot.co.uk/2012/11/ubuntu-1204-precise-lts-install.html
• https://www.owasp.org/index.php/OWASP_Securing_WebGoat_using_ModSecurity_Project
• http://spiderlabs.github.io/owasp-modsecurity-crs/
• https://library.linode.com/web-servers/apache/mod-security

If you don't have time to go through all this to build a lab, or you want a second, the OWASP Broken Web Applications Project is a VM which contains a number of vulnerable applications and also runs ModSecurity.

Thanks BruCON

The time to create this project was kindly sponsored by the BruCON 5x5 award.