headers
Stephen Craig Evans | 27 Jun 18:01

Re: problem with my regex and single line HTMLcomment in RESPONSE_BODY

Thanks a lot, Ryan. I'll try it out tomorrow morning and I'm 97% sure
that it will work, based on your previous success rate of suggestions.

I didn't mean to be offensive with the ModSecurity "choking" part; I
believe it uses the Apache PCRE engine.

In addition to those tools, I find the
http://www.regular-expressions.info/tutorial.html site useful and the
Friedl regex book is my current bathroom book :-)

Cheers,
Stephen

On Fri, Jun 27, 2008 at 10:45 PM, Ryan Barnett <Ryan.Barnett <at> breach.com> wrote:
> Stephen,
> First off, I am sending you a "Virtual High-Five" as you are
> demonstrating that you are using many of the various tools and
> techniques that we have been discussing for trouble-shooting!  Using
> RegEx Coach/Expresso and also leveraging the debug log is outstanding.
>
> To your exact issues, I don't believe that Mod was "choking" on anything
> but rather that your RegEx was not working correctly for the single line
> comment.  Here are a few comments -
>
> 1) While RegEx tools (such as Expresso) are invaluable for helping to
> create or trouble-shoot regex issues, beware that you can also run into
> issues if the tool is not configured identically to how ModSecurity's
> pcre/transformations will be used.  I just ran into this recently where
> Expresso had the option checked to "Ignore Whitespace" and it screwed up
> some of my tests.  In Expresso 3.0, click on the "Design Mode" tab and
> check at the bottom to see how it is handling some of these issues.
>
> 2) In order to account for both single or multi-line comments, I had to
> update your RegEx to add a few quantifiers to allow for the
> presence/absence of CR, LF, etc...  I also made a few changes to catch
> other "admin" or "password" variations.  One other item, in order to
> handle mixed-case issues you can use "t:lowercase" however keep in mind
> that transformation functions against the RESPONSE_BODY are rather
> expensive so if you can specify all of the variations in the RegEx you
> might want to for performance reasons.  You could also try and use the
> regex embedded ignore case flag (?i:).  Here is an updated rule that
> works for me -
>
> SecRule RESPONSE_BODY "<!--[
> \r\n\t]*?(.*)?(?i:adm(in)?|pwd|passw(or)?d)(.*)?[ \r\n\t]*?-->"
> "phase:4,t:none,log,auditlog,deny,status:503,msg:'Authentication
> Credentials in HTML Comment Tag.',tag:'LEAKAGE'"
>
> With this rule, it was able to catch all of the examples you showed.
>
> 3) In the debug log, it wasn't that Mod was choking on the RESPONSE_BODY
> data, but rather that it will only display up to a certain amount of
> data within the log file itself.  It will still parse the entire payload
> however.  It simply looks as though your example comment data was
> further down in the payload and thus was not displayed in the log.
>
> Hope this info helps.
>
> --
> Ryan C. Barnett
> ModSecurity Community Manager
> Breach Security: Director of Application Security
> Web Application Security Consortium (WASC) Member
> CIS Apache Benchmark Project Lead
> SANS Instructor, GCIA, GCFA, GCIH, GSNA, GCUX, GSEC
> Author: Preventing Web Attacks with Apache
>
>> -----Original Message-----
>> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-
>> security-users-bounces <at> lists.sourceforge.net] On Behalf Of Stephen
> Craig
>> Evans
>> Sent: Friday, June 27, 2008 8:01 AM
>> To: mod-security-users <at> lists.sourceforge.net
>> Subject: [mod-security-users] problem with my regex and single line
>> HTMLcomment in RESPONSE_BODY
>>
>> Hi,
>>
>> I'm having a problem with a regex.
>>
>> I want to prevent content in an HTTP response being returned as in:
>> <!--
>>       TODO admin:adminpw
>>   -->
>>
>> My regular expression:
>> <!--[ \r\n\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[
> \r\n\t]*
>> -->
>>
>> works both in Regex Coach and Expresso 3.0 and on variations such as:
>>     <!--
>>       Use Admin to regenerate database
>>   -->
>>
>> which is what I want.
>>
>> To try to make a long story short - without a bunch of debug stuff -
>> it works as designed on the 'bad' pages with multiline HTML comments
>> like above but when it processes a page with any HTML comment on one
>> line (e.g. <!-- Stop Instructions -->) it chokes and stops processing
>> the page.
>>
>>
>> My rule:
>> SecRule RESPONSE_BODY "<!--[
>> \r\n\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[ \r\n\t]*
>> -->"
>> "phase:4,ctl:auditLogParts=+E,log,deny,status:501,auditlog,msg:'HTML
>> comment source code
>> leakage',id:'8',tag:'LEAKAGE/SOURCE_CODE',severity:'4'"
>>
>> Adding 'capture' to the rule as in "phase:4,capture,ctl..." gets rid
>> of the debug message "Ignoring regex captures since "capture" action
>> is not enabled." but the result is the exact same.
>>
>> I also use "SecCacheTransformations=Off" so that transformations
>> aren't being cached.
>>
>> The end of the debug log file shows (with some sanitation):
>>
>> [26/Jun/2008:19:36:29 +0800]
>> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Starting phase
>> RESPONSE_BODY.
>> [26/Jun/2008:19:36:29 +0800]
>> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][9] This phase consists of
>> 5 rule(s).
>> [26/Jun/2008:19:36:29 +0800]
>> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Recipe: Invoking rule
>> 80c3c28; [file "/etc/modsecurity/rulefile_fail-open-auth2.conf"] [line
>> "268"] [id "8"].
>> [26/Jun/2008:19:36:29 +0800]
>> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][5] Rule 80c3c28: SecRule
>> "RESPONSE_BODY" "@rx <!--[
>> \\r\\n\\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[
>> \\r\\n\\t]* -->"
>>
> "phase:4,capture,ctl:auditLogParts=+E,log,deny,status:501,auditlog,msg:'
> HT
>> ML
>> comment source code leakage',id:8,tag:LEAKAGE/SOURCE_CODE,severity:4"
>> [26/Jun/2008:19:36:29 +0800]
>> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Transformation
>> completed in 4 usec.
>> [26/Jun/2008:19:36:29 +0800]
>> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Executing operator
>> "rx" with param "<!--[
>> \\r\\n\\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[
>> \\r\\n\\t]* -->" against RESPONSE_BODY.
>> [26/Jun/2008:19:36:29 +0800]
>> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][9] Target value:
>> "\r\n\r\n\r\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
>> Transitional//EN"
>>
> "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n\r\n<html
>> xmlns="http://www.w3.org/1999/xhtml">\r\n<head>\r\n<meta
>> http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"
>> />\r\n<title>Sample Page</title>\r\n<link rel="stylesheet"
>> href="css/draw.css" type="text/css" />\r\n<link rel="stylesheet"
>> href="css/page1.css" type="text/css" />\r\n<link rel="stylesheet"
>> href="css/menu.css" type="text/css" />\r\n<link rel="stylesheet"
>> href="css/layers.css" type="text/css" />\r\n<script
>> language="JavaScript1.2" src="javascript/javascript.js"
>> type="text/javascript"></script>\r\n<script language="JavaScript1.2"
>> src="javascript/menu_system.js"
>> type="text/javascript"></script>\r\n<script language="JavaScript1.2"
>> src="javascript/pageNav.js"
>> type="text/javascript"></script>\r\n<script language="JavaScript1.2"
>> src="javascript/makeWindow.js"
>> type="text/javascript"></script>\r\n<script language="Java
>>
>> And it stops there. This page below the output contains an HTML
>> comment: <!-- Stop Instructions -->
>>
>> I even added 'Instructions' to the rule to see if a one line HTML
>> comment with a match worked, but it doesn't. Same result.
>>
>> Can anybody help? I'm baffled.
>>
>> I'm using ModSecurity 2.5.1 with Apache 2.2 on Kubuntu 7.10.
>>
>> Thanks in advance,
>> Stephen
>>
>>
> ------------------------------------------------------------------------
> -
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://sourceforge.net/services/buy/index.php
>> _______________________________________________
>> mod-security-users mailing list
>> mod-security-users <at> lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
Permalink | Reply |

Gmane