Re: problem with my regex and single lineHTMLcomment in RESPONSE_BODY

On Thu, Jul 3, 2008 at 3:50 PM, Stephen Craig Evans
<stephencraig.evans <at> gmail.com> wrote:
> Hi Ryan,
>
> Yes, I use Expresso 3.0 and The Regex Coach and as you know the
> results often don't jive with the results in ModSecurity.
>
>> The debug log doesn't really show you "how" the regex is being
>> interpreted.
> Yeah, it's not a good thing when I have to use the debug log to
> interpret how a regex is being processed.

But the debug log is not telling you how regular expressions are
processed. It is only telling you which regex ModSecurity is running,
and against what text. Are you frustrated with not being to able to
write regular expressions effortlessly, or by not being able to know
exactly how ModSecurity is executing the rules (and in which order,
etc).

> Again, please don't take this as anything against any of you guys at
> Breach or how ModSecurity is implemented. I'm just venting my current
> frustration at the regex stuff.
>
> Cheers,
> Stephen
>
>
> On Thu, Jul 3, 2008 at 10:33 PM, Ryan Barnett <Ryan.Barnett <at> breach.com> wrote:
>> Perhaps I missed it but have you tried testing your regexs with tools
>> like Expresso before trying them in Mod?  They have a good description
>> view that states what the regex components actually mean.
>>
>> http://blog.modsecurity.org/2007/03/regular-express.html
>>
>> The debug log doesn't really show you "how" the regex is being
>> interpreted.
>>
>> -Ryan
>>
>>> -----Original Message-----
>>> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-
>>> security-users-bounces <at> lists.sourceforge.net] On Behalf Of Stephen
>> Craig
>>> Evans
>>> Sent: Thursday, July 03, 2008 10:14 AM
>>> To: Achim Hoffmann
>>> Cc: mod-security-users <at> lists.sourceforge.net
>>> Subject: Re: [mod-security-users] problem with my regex and single
>>> lineHTMLcomment in RESPONSE_BODY
>>>
>>> Hi Achim,
>>>
>>> Pardon me if my rant is off-topic, but this seems to be a good place
>>> for it for me now.
>>>
>>> In the 60+ hours in the last 5 days that I have spent writing
>>> ModSecurity rules for WebGoat vulnerabilities, more than half of that
>>> time has been spent on getting the regex's working. I am so tired of
>>> reading the debug file to see how my regex is being interpreted.
>>>
>>> I feel like I am a slave to the PCRE engine instead of the opposite.
>>>
>>> It's not rocket science:
>>> 1. I want an account number that has digits, characters and a hyphen,
>>> but no spaces or special characters.
>>> 2. I want a password that has alphanumeric and special chars, but has
>>> no spaces or '>' and '<'.
>>> 3. I want a user name with chars, ', -, and spaces but nothing else.
>>>
>>> I could do this much easier and faster writing Java, C#, or C (which
>>> is why ModSecurity is written in C; check the source for
>>> urlDecodeUni).
>>>
>>> I'm at the point where I think it's easier to write my own routines in
>>> Lua and build my own library for reuse; disclaimer: I don't need
>>> speed.
>>>
>>> (/end of rant)
>>>
>>> Stephen
>>>
>>>
>>>
>>> On Thu, Jul 3, 2008 at 7:10 PM, Achim Hoffmann <ah <at> securenet.de>
>> wrote:
>>> > !! Yes, we do use PCRE underneath. We don't do anything with the
>> regular
>>> > !! expression... we just pass it to the PCRE engine, compiling with
>>> >
>>> > thanks Ivan for this information (which could be found in the docs,
>>> > I believe:)
>>> >
>>> > !!  "PCRE_DOTALL
>>> >
>>> > this means that the s modifier in the regex is obsolete, somehow
>>> > As the core-rules set uses (?i:) modifiers, someone -who initially
>>> > understands that- might think to use (?s:) also.
>>> > On the other hand: does (?m:) change it back to "dot does not match
>>> > newline"? This is not documented in http://www.pcre.org/pcre.txt
>>> > However, perlre man-page is accurate in that behaviour.
>>> >
>>> > !! | PCRE_DOLLAR_ENDONLY".
>>> >
>>> > hmm, this causes some questions how ModSecurity handles "strings",
>>> > for example:
>>> >  is the whole HTTP header passed to the rules, or each line
>>> >  (means what is separated by \r\n) individually?
>>> >  That would make some difference, I guess.
>>> >  You need to know that when writing rules.
>>> >
>>> > Before going deeper into that (and some more examples), I'd
>>> > suggest to point this out in the docs. I mean to describe how
>>> > the different parts of the request/response is handled by ModSec.
>>> >
>>> > Achim
>>> >
>>> >
>>>
>>>
>> ------------------------------------------------------------------------
>> -
>>> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
>>> Studies have shown that voting for your favorite open source project,
>>> along with a healthy diet, reduces your potential for chronic lameness
>>> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod-security-users <at> lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>
>
> -------------------------------------------------------------------------
> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
> Studies have shown that voting for your favorite open source project,
> along with a healthy diet, reduces your potential for chronic lameness
> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
> _______________________________________________
> mod-security-users mailing list
> mod-security-users <at> lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>

--

-- 
Ivan Ristic

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08