Re: problem with my regex and single lineHTMLcomment in RESPONSE_BODY

For what it's worth, I think that regexes are the best feature of 
ModSecurity and the most appropriate way to analyze a stream of text.  I 
couldn't imagine the headache of having to write rules in C.

Tom

Ryan Barnett wrote:
> Perhaps I missed it but have you tried testing your regexs with tools
> like Expresso before trying them in Mod?  They have a good description
> view that states what the regex components actually mean.
> 
> http://blog.modsecurity.org/2007/03/regular-express.html
> 
> The debug log doesn't really show you "how" the regex is being
> interpreted.
> 
> -Ryan
> 
>> -----Original Message-----
>> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-
>> security-users-bounces <at> lists.sourceforge.net] On Behalf Of Stephen
> Craig
>> Evans
>> Sent: Thursday, July 03, 2008 10:14 AM
>> To: Achim Hoffmann
>> Cc: mod-security-users <at> lists.sourceforge.net
>> Subject: Re: [mod-security-users] problem with my regex and single
>> lineHTMLcomment in RESPONSE_BODY
>>
>> Hi Achim,
>>
>> Pardon me if my rant is off-topic, but this seems to be a good place
>> for it for me now.
>>
>> In the 60+ hours in the last 5 days that I have spent writing
>> ModSecurity rules for WebGoat vulnerabilities, more than half of that
>> time has been spent on getting the regex's working. I am so tired of
>> reading the debug file to see how my regex is being interpreted.
>>
>> I feel like I am a slave to the PCRE engine instead of the opposite.
>>
>> It's not rocket science:
>> 1. I want an account number that has digits, characters and a hyphen,
>> but no spaces or special characters.
>> 2. I want a password that has alphanumeric and special chars, but has
>> no spaces or '>' and '<'.
>> 3. I want a user name with chars, ', -, and spaces but nothing else.
>>
>> I could do this much easier and faster writing Java, C#, or C (which
>> is why ModSecurity is written in C; check the source for
>> urlDecodeUni).
>>
>> I'm at the point where I think it's easier to write my own routines in
>> Lua and build my own library for reuse; disclaimer: I don't need
>> speed.
>>
>> (/end of rant)
>>
>> Stephen
>>
>>
>>
>> On Thu, Jul 3, 2008 at 7:10 PM, Achim Hoffmann <ah <at> securenet.de>
> wrote:
>>> !! Yes, we do use PCRE underneath. We don't do anything with the
> regular
>>> !! expression... we just pass it to the PCRE engine, compiling with
>>>
>>> thanks Ivan for this information (which could be found in the docs,
>>> I believe:)
>>>
>>> !!  "PCRE_DOTALL
>>>
>>> this means that the s modifier in the regex is obsolete, somehow
>>> As the core-rules set uses (?i:) modifiers, someone -who initially
>>> understands that- might think to use (?s:) also.
>>> On the other hand: does (?m:) change it back to "dot does not match
>>> newline"? This is not documented in http://www.pcre.org/pcre.txt
>>> However, perlre man-page is accurate in that behaviour.
>>>
>>> !! | PCRE_DOLLAR_ENDONLY".
>>>
>>> hmm, this causes some questions how ModSecurity handles "strings",
>>> for example:
>>>  is the whole HTTP header passed to the rules, or each line
>>>  (means what is separated by \r\n) individually?
>>>  That would make some difference, I guess.
>>>  You need to know that when writing rules.
>>>
>>> Before going deeper into that (and some more examples), I'd
>>> suggest to point this out in the docs. I mean to describe how
>>> the different parts of the request/response is handled by ModSec.
>>>
>>> Achim
>>>
>>>
>>
> ------------------------------------------------------------------------
> -
>> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
>> Studies have shown that voting for your favorite open source project,
>> along with a healthy diet, reduces your potential for chronic lameness
>> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
>> _______________________________________________
>> mod-security-users mailing list
>> mod-security-users <at> lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> 
> -------------------------------------------------------------------------
> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
> Studies have shown that voting for your favorite open source project,
> along with a healthy diet, reduces your potential for chronic lameness
> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
> _______________________________________________
> mod-security-users mailing list
> mod-security-users <at> lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> 
> 

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08