Re: ongoing textsearch/string match - 3 patches

Re: ongoing textsearch/string match - 3 patches

[Pablo Neira Ayuso]

Hi Amir,

Amir Goldstein wrote:
> I apologize that I am sending these patches directly to you,
> but I was not sure what is the policy of netfiler-devel list regarding
> sending patches.
> 
> I was hoping that you could help me with contributing these patches
> or with instructing me about the HOWTO of sending patches.

No need to worry. Basically, the policy is: send well-formed patches
together with a description of what you want to do and why you want to
do that. I've cc'ed the reply to netfilter-devel so everyone could
follow the discussion.

> I made these 3 patches to enable the string match netfilter
> to find a string that crosses the single packet boundaries.
> most of the work was to add ongoing textsearch support to Thomas's
> infrastructure.
> 
> Thanks for your time,
> Amir.
> 
> patches:
> ------------
> linux-2.6.14_ongoing-textsearch.patch:
> - add ongoing textsearch API to infrastructure (new/init/final/delete)
> - implement ongoing support for KMP textsearch algo
> - add /proc/textsearch entry to proc fs (lists loaded algos)
    ^^^
this can be useful.

> netfilter-2.6.14_ongoing-string-match.patch:
> - add ongoing support to string match info struct
> - use ongoing textsearch API when configured to do so
> 
> iptables-1.3.4_ongoing-string-match.patch:
> - add --ongoing option to string match iptables extension
> - use new string match info struct to pass args to kernel
> 
> test:
> ------
> I have tested ongoing string match using netcat UDP connection
> and it successfully matched the string:
> s
> tr
> ing

So, in this example, my understanding is that you want to find matchings
 splitted in several packets.

> which was sent in different 3 packets.
> the netcat test required using --from 28 to skip the udp header and
> some test code to ignore the new line characters.

I'm curious: We implement matching through fragments with KMP (not for
BM yet). So, what kind of application would require such pattern
matching policy?

> Index: linux-2.6.14/include/linux/netfilter_ipv4/ipt_string.h
> ===================================================================
> --- linux-2.6.14.orig/include/linux/netfilter_ipv4/ipt_string.h	2005-11-23 19:13:42.000000000 +0200
> +++ linux-2.6.14/include/linux/netfilter_ipv4/ipt_string.h	2005-11-23 19:16:38.000000000 +0200
> @@ -11,8 +11,10 @@
> 	char	  algo[IPT_STRING_MAX_ALGO_NAME_SIZE];
> 	char 	  pattern[IPT_STRING_MAX_PATTERN_SIZE];
> 	u_int8_t  patlen;
> +	u_int8_t  ongoing;
> 	u_int8_t  invert;
> 	struct ts_config __attribute__((aligned(8))) *config;
> +	struct ts_state __attribute__((aligned(8))) *state;
>  };

Unfortunately, you can't do this. This would break backward
compatibility for previous versions of iptables. The correct way to do
this is using the matches/targets versioning (see ipt_MARK for an
example). We can revisit this later.

-- 
Pablo

Re: ongoing textsearch/string match - 3 patches

[Pablo Neira Ayuso]

Amir Goldstein wrote:
> On 12/23/05, Pablo Neira Ayuso <pablo@eurodev.net> wrote:
> 
>>>test:
>>>------
>>>I have tested ongoing string match using netcat UDP connection
>>>and it successfully matched the string:
>>>s
>>>tr
>>>ing
>>
>>So, in this example, my understanding is that you want to find matchings
>> splitted in several packets.
>>
>>>which was sent in different 3 packets.
>>>the netcat test required using --from 28 to skip the udp header and
>>>some test code to ignore the new line characters.
>>
>>I'm curious: We implement matching through fragments with KMP (not for
>>BM yet). So, what kind of application would require such pattern
>>matching policy?
>>
> 
> it's the basics for content filtering in netfilter.
> of course, more work needs to be done:
> - connect this code with connection tracking
> - handle out of order tcp packets

OK. But I'm still in doubt about the usefulness of matching a pattern
between packets. What kind of application splits a request, represented
in the form of a pattern, between packets? Going back to your example:

1st packet) s
2nd packet) tr
3rd packet) ing

Could you comment any realistic scenario?

>>>Index: linux-2.6.14/include/linux/netfilter_ipv4/ipt_string.h
>>>===================================================================
>>>--- linux-2.6.14.orig/include/linux/netfilter_ipv4/ipt_string.h       2005-11-23 19:13:42.000000000 +0200
>>>+++ linux-2.6.14/include/linux/netfilter_ipv4/ipt_string.h    2005-11-23 19:16:38.000000000 +0200
>>>@@ -11,8 +11,10 @@
>>>      char      algo[IPT_STRING_MAX_ALGO_NAME_SIZE];
>>>      char      pattern[IPT_STRING_MAX_PATTERN_SIZE];
>>>      u_int8_t  patlen;
>>>+     u_int8_t  ongoing;
>>>      u_int8_t  invert;
>>>      struct ts_config __attribute__((aligned(8))) *config;
>>>+     struct ts_state __attribute__((aligned(8))) *state;
>>> };
>>
>>Unfortunately, you can't do this. This would break backward
>>compatibility for previous versions of iptables. The correct way to do
>>this is using the matches/targets versioning (see ipt_MARK for an
>>example). We can revisit this later.
>>
> 
> it would take me some time before I can look into the match versioning.
> should I just post the patches as they are now?

Hold on with the versioning stuff. We can do this later if I finally
understand the target of your contribution. Thanks!

-- 
Pablo

Re: ongoing textsearch/string match - 3 patches

[amir73il]

On 12/26/05, Pablo Neira Ayuso <pablo@eurodev.net> wrote:
> Amir Goldstein wrote:
> > On 12/23/05, Pablo Neira Ayuso <pablo@eurodev.net> wrote:
> >
> >>>test:
> >>>------
> >>>I have tested ongoing string match using netcat UDP connection
> >>>and it successfully matched the string:
> >>>s
> >>>tr
> >>>ing
> >>
> >>So, in this example, my understanding is that you want to find matchings
> >> splitted in several packets.
> >>
> >>>which was sent in different 3 packets.
> >>>the netcat test required using --from 28 to skip the udp header and
> >>>some test code to ignore the new line characters.
> >>
> >>I'm curious: We implement matching through fragments with KMP (not for
> >>BM yet). So, what kind of application would require such pattern
> >>matching policy?
> >>
> >
> > it's the basics for content filtering in netfilter.
> > of course, more work needs to be done:
> > - connect this code with connection tracking
> > - handle out of order tcp packets
>
> OK. But I'm still in doubt about the usefulness of matching a pattern
> between packets. What kind of application splits a request, represented
> in the form of a pattern, between packets? Going back to your example:
>
> 1st packet) s
> 2nd packet) tr
> 3rd packet) ing
>
> Could you comment any realistic scenario?
>

Hi Pablo,

the usefulness of matching a pattern between packets is for network virus scan.
it took me some time to finalize my kclamav module, but here it is:
http://sourceforge.net/projects/kclamav/

I patched the string match netfilter because it was the closest filter
to what I needed, but as I said, there is still work to be done to
make this a useful virus scan filter.
perhaps a more generic "content filter" is more appropriate here.

I hope you will find this contribution useful and help me get my
patches into the kernel.

Amir.

Re: ongoing textsearch/string match - 3 patches

[Pablo Neira Ayuso]

Hi Amir,

amir73il@gmail.com wrote:
>>Could you comment any realistic scenario?
>>
> the usefulness of matching a pattern between packets is for network virus scan.
> it took me some time to finalize my kclamav module, but here it is:
> http://sourceforge.net/projects/kclamav/
> 
> I patched the string match netfilter because it was the closest filter
> to what I needed, but as I said, there is still work to be done to
> make this a useful virus scan filter.
> perhaps a more generic "content filter" is more appropriate here.
> 
> I hope you will find this contribution useful and help me get my
> patches into the kernel.

Sorry for not replying. I haven't forgotten this. I think I can improve
that patch. I'll give it a spin this weekend and I'll get back to you.

-- 
Pablo