28 Jun 13:29
Re: problem with my regex and single line HTMLcomment in RESPONSE_BODY
From: Stephen Craig Evans <stephencraig.evans <at> gmail.com>
Subject: Re: problem with my regex and single line HTMLcomment in RESPONSE_BODY
Newsgroups: gmane.comp.apache.mod-security.user
Date: 2008-06-28 11:29:57 GMT
Subject: Re: problem with my regex and single line HTMLcomment in RESPONSE_BODY
Newsgroups: gmane.comp.apache.mod-security.user
Date: 2008-06-28 11:29:57 GMT
Hi Ryan, Thanks. That's very close. Your regex works great except when the 'bad' words are between 2 HTML comments like this: <!-- start --> password <!-- stop --> I was getting a false match and it took me hours to get it whittled down to this; just put those 3 lines between the <body> tags of the barest HTML file and a false match will result. Both Expresso and RegEx Coach say that it is not a match. I might have to use regex capturing or something like that but I don't have the regex skills yet to do that. It's good enough for now and once I get the regex skills to figure it out, I'll revisit this. Thanks, Stephen On Fri, Jun 27, 2008 at 10:45 PM, Ryan Barnett <Ryan.Barnett <at> breach.com> wrote: > Stephen, > First off, I am sending you a "Virtual High-Five" as you are > demonstrating that you are using many of the various tools and > techniques that we have been discussing for trouble-shooting! Using > RegEx Coach/Expresso and also leveraging the debug log is outstanding. > > To your exact issues, I don't believe that Mod was "choking" on anything > but rather that your RegEx was not working correctly for the single line > comment. Here are a few comments - > > 1) While RegEx tools (such as Expresso) are invaluable for helping to > create or trouble-shoot regex issues, beware that you can also run into > issues if the tool is not configured identically to how ModSecurity's > pcre/transformations will be used. I just ran into this recently where > Expresso had the option checked to "Ignore Whitespace" and it screwed up > some of my tests. In Expresso 3.0, click on the "Design Mode" tab and > check at the bottom to see how it is handling some of these issues. > > 2) In order to account for both single or multi-line comments, I had to > update your RegEx to add a few quantifiers to allow for the > presence/absence of CR, LF, etc... I also made a few changes to catch > other "admin" or "password" variations. One other item, in order to > handle mixed-case issues you can use "t:lowercase" however keep in mind > that transformation functions against the RESPONSE_BODY are rather > expensive so if you can specify all of the variations in the RegEx you > might want to for performance reasons. You could also try and use the > regex embedded ignore case flag (?i:). Here is an updated rule that > works for me - > > SecRule RESPONSE_BODY "<!--[ > \r\n\t]*?(.*)?(?i:adm(in)?|pwd|passw(or)?d)(.*)?[ \r\n\t]*?-->" > "phase:4,t:none,log,auditlog,deny,status:503,msg:'Authentication > Credentials in HTML Comment Tag.',tag:'LEAKAGE'" > > With this rule, it was able to catch all of the examples you showed. > > 3) In the debug log, it wasn't that Mod was choking on the RESPONSE_BODY > data, but rather that it will only display up to a certain amount of > data within the log file itself. It will still parse the entire payload > however. It simply looks as though your example comment data was > further down in the payload and thus was not displayed in the log. > > Hope this info helps. > > -- > Ryan C. Barnett > ModSecurity Community Manager > Breach Security: Director of Application Security > Web Application Security Consortium (WASC) Member > CIS Apache Benchmark Project Lead > SANS Instructor, GCIA, GCFA, GCIH, GSNA, GCUX, GSEC > Author: Preventing Web Attacks with Apache > >> -----Original Message----- >> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod- >> security-users-bounces <at> lists.sourceforge.net] On Behalf Of Stephen > Craig >> Evans >> Sent: Friday, June 27, 2008 8:01 AM >> To: mod-security-users <at> lists.sourceforge.net >> Subject: [mod-security-users] problem with my regex and single line >> HTMLcomment in RESPONSE_BODY >> >> Hi, >> >> I'm having a problem with a regex. >> >> I want to prevent content in an HTTP response being returned as in: >> <!-- >> TODO admin:adminpw >> --> >> >> My regular expression: >> <!--[ \r\n\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[ > \r\n\t]* >> --> >> >> works both in Regex Coach and Expresso 3.0 and on variations such as: >> <!-- >> Use Admin to regenerate database >> --> >> >> which is what I want. >> >> To try to make a long story short - without a bunch of debug stuff - >> it works as designed on the 'bad' pages with multiline HTML comments >> like above but when it processes a page with any HTML comment on one >> line (e.g. <!-- Stop Instructions -->) it chokes and stops processing >> the page. >> >> >> My rule: >> SecRule RESPONSE_BODY "<!--[ >> \r\n\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[ \r\n\t]* >> -->" >> "phase:4,ctl:auditLogParts=+E,log,deny,status:501,auditlog,msg:'HTML >> comment source code >> leakage',id:'8',tag:'LEAKAGE/SOURCE_CODE',severity:'4'" >> >> Adding 'capture' to the rule as in "phase:4,capture,ctl..." gets rid >> of the debug message "Ignoring regex captures since "capture" action >> is not enabled." but the result is the exact same. >> >> I also use "SecCacheTransformations=Off" so that transformations >> aren't being cached. >> >> The end of the debug log file shows (with some sanitation): >> >> [26/Jun/2008:19:36:29 +0800] >> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Starting phase >> RESPONSE_BODY. >> [26/Jun/2008:19:36:29 +0800] >> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][9] This phase consists of >> 5 rule(s). >> [26/Jun/2008:19:36:29 +0800] >> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Recipe: Invoking rule >> 80c3c28; [file "/etc/modsecurity/rulefile_fail-open-auth2.conf"] [line >> "268"] [id "8"]. >> [26/Jun/2008:19:36:29 +0800] >> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][5] Rule 80c3c28: SecRule >> "RESPONSE_BODY" "@rx <!--[ >> \\r\\n\\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[ >> \\r\\n\\t]* -->" >> > "phase:4,capture,ctl:auditLogParts=+E,log,deny,status:501,auditlog,msg:' > HT >> ML >> comment source code leakage',id:8,tag:LEAKAGE/SOURCE_CODE,severity:4" >> [26/Jun/2008:19:36:29 +0800] >> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Transformation >> completed in 4 usec. >> [26/Jun/2008:19:36:29 +0800] >> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][4] Executing operator >> "rx" with param "<!--[ >> \\r\\n\\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[ >> \\r\\n\\t]* -->" against RESPONSE_BODY. >> [26/Jun/2008:19:36:29 +0800] >> [192.168.0.5/sid#82100e0][rid#8424118][/xxx][9] Target value: >> "\r\n\r\n\r\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 >> Transitional//EN" >> > "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n\r\n<html >> xmlns="http://www.w3.org/1999/xhtml">\r\n<head>\r\n<meta >> http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" >> />\r\n<title>Sample Page</title>\r\n<link rel="stylesheet" >> href="css/draw.css" type="text/css" />\r\n<link rel="stylesheet" >> href="css/page1.css" type="text/css" />\r\n<link rel="stylesheet" >> href="css/menu.css" type="text/css" />\r\n<link rel="stylesheet" >> href="css/layers.css" type="text/css" />\r\n<script >> language="JavaScript1.2" src="javascript/javascript.js" >> type="text/javascript"></script>\r\n<script language="JavaScript1.2" >> src="javascript/menu_system.js" >> type="text/javascript"></script>\r\n<script language="JavaScript1.2" >> src="javascript/pageNav.js" >> type="text/javascript"></script>\r\n<script language="JavaScript1.2" >> src="javascript/makeWindow.js" >> type="text/javascript"></script>\r\n<script language="Java >> >> And it stops there. This page below the output contains an HTML >> comment: <!-- Stop Instructions --> >> >> I even added 'Instructions' to the rule to see if a one line HTML >> comment with a match worked, but it doesn't. Same result. >> >> Can anybody help? I'm baffled. >> >> I'm using ModSecurity 2.5.1 with Apache 2.2 on Kubuntu 7.10. >> >> Thanks in advance, >> Stephen >> >> > ------------------------------------------------------------------------ > - >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for >> just about anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> mod-security-users mailing list >> mod-security-users <at> lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/mod-security-users > ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
RSS Feed