Re: problem with my regex and single lineHTMLcomment in RESPONSE_BODY
Ivan,
Of course I am "frustrated with not being to able to write regular
expressions effortlessly" 
I think when Ryan cleans the gunk from under his fingernails, there's
more regex knowledge dispelled than what I have now.
Hope you all can make it for the OWASP conference in Portugal in November.
Stephen
On Thu, Jul 3, 2008 at 10:57 PM, Ivan Ristic <ivan.ristic <at> gmail.com> wrote:
> On Thu, Jul 3, 2008 at 3:50 PM, Stephen Craig Evans
> <stephencraig.evans <at> gmail.com> wrote:
>> Hi Ryan,
>>
>> Yes, I use Expresso 3.0 and The Regex Coach and as you know the
>> results often don't jive with the results in ModSecurity.
>>
>>> The debug log doesn't really show you "how" the regex is being
>>> interpreted.
>> Yeah, it's not a good thing when I have to use the debug log to
>> interpret how a regex is being processed.
>
> But the debug log is not telling you how regular expressions are
> processed. It is only telling you which regex ModSecurity is running,
> and against what text. Are you frustrated with not being to able to
> write regular expressions effortlessly, or by not being able to know
> exactly how ModSecurity is executing the rules (and in which order,
> etc).
>
>
>> Again, please don't take this as anything against any of you guys at
>> Breach or how ModSecurity is implemented. I'm just venting my current
>> frustration at the regex stuff.
>>
>> Cheers,
>> Stephen
>>
>>
>> On Thu, Jul 3, 2008 at 10:33 PM, Ryan Barnett <Ryan.Barnett <at> breach.com> wrote:
>>> Perhaps I missed it but have you tried testing your regexs with tools
>>> like Expresso before trying them in Mod? They have a good description
>>> view that states what the regex components actually mean.
>>>
>>> http://blog.modsecurity.org/2007/03/regular-express.html
>>>
>>> The debug log doesn't really show you "how" the regex is being
>>> interpreted.
>>>
>>> -Ryan
>>>
>>>> -----Original Message-----
>>>> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-
>>>> security-users-bounces <at> lists.sourceforge.net] On Behalf Of Stephen
>>> Craig
>>>> Evans
>>>> Sent: Thursday, July 03, 2008 10:14 AM
>>>> To: Achim Hoffmann
>>>> Cc: mod-security-users <at> lists.sourceforge.net
>>>> Subject: Re: [mod-security-users] problem with my regex and single
>>>> lineHTMLcomment in RESPONSE_BODY
>>>>
>>>> Hi Achim,
>>>>
>>>> Pardon me if my rant is off-topic, but this seems to be a good place
>>>> for it for me now.
>>>>
>>>> In the 60+ hours in the last 5 days that I have spent writing
>>>> ModSecurity rules for WebGoat vulnerabilities, more than half of that
>>>> time has been spent on getting the regex's working. I am so tired of
>>>> reading the debug file to see how my regex is being interpreted.
>>>>
>>>> I feel like I am a slave to the PCRE engine instead of the opposite.
>>>>
>>>> It's not rocket science:
>>>> 1. I want an account number that has digits, characters and a hyphen,
>>>> but no spaces or special characters.
>>>> 2. I want a password that has alphanumeric and special chars, but has
>>>> no spaces or '>' and '<'.
>>>> 3. I want a user name with chars, ', -, and spaces but nothing else.
>>>>
>>>> I could do this much easier and faster writing Java, C#, or C (which
>>>> is why ModSecurity is written in C; check the source for
>>>> urlDecodeUni).
>>>>
>>>> I'm at the point where I think it's easier to write my own routines in
>>>> Lua and build my own library for reuse; disclaimer: I don't need
>>>> speed.
>>>>
>>>> (/end of rant)
>>>>
>>>> Stephen
>>>>
>>>>
>>>>
>>>> On Thu, Jul 3, 2008 at 7:10 PM, Achim Hoffmann <ah <at> securenet.de>
>>> wrote:
>>>> > !! Yes, we do use PCRE underneath. We don't do anything with the
>>> regular
>>>> > !! expression... we just pass it to the PCRE engine, compiling with
>>>> >
>>>> > thanks Ivan for this information (which could be found in the docs,
>>>> > I believe:)
>>>> >
>>>> > !! "PCRE_DOTALL
>>>> >
>>>> > this means that the s modifier in the regex is obsolete, somehow
>>>> > As the core-rules set uses (?i:) modifiers, someone -who initially
>>>> > understands that- might think to use (?s:) also.
>>>> > On the other hand: does (?m:) change it back to "dot does not match
>>>> > newline"? This is not documented in http://www.pcre.org/pcre.txt
>>>> > However, perlre man-page is accurate in that behaviour.
>>>> >
>>>> > !! | PCRE_DOLLAR_ENDONLY".
>>>> >
>>>> > hmm, this causes some questions how ModSecurity handles "strings",
>>>> > for example:
>>>> > is the whole HTTP header passed to the rules, or each line
>>>> > (means what is separated by \r\n) individually?
>>>> > That would make some difference, I guess.
>>>> > You need to know that when writing rules.
>>>> >
>>>> > Before going deeper into that (and some more examples), I'd
>>>> > suggest to point this out in the docs. I mean to describe how
>>>> > the different parts of the request/response is handled by ModSec.
>>>> >
>>>> > Achim
>>>> >
>>>> >
>>>>
>>>>
>>> ------------------------------------------------------------------------
>>> -
>>>> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
>>>> Studies have shown that voting for your favorite open source project,
>>>> along with a healthy diet, reduces your potential for chronic lameness
>>>> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
>>>> _______________________________________________
>>>> mod-security-users mailing list
>>>> mod-security-users <at> lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>
>>
>> -------------------------------------------------------------------------
>> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
>> Studies have shown that voting for your favorite open source project,
>> along with a healthy diet, reduces your potential for chronic lameness
>> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
>> _______________________________________________
>> mod-security-users mailing list
>> mod-security-users <at> lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>
>
>
>
> --
> Ivan Ristic
>
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
RSS Feed